scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-11-11 08:37:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Merge origin/main into pebble-agent-loop

Browse source 
Main merged the sandbox-driver adoption (#849) in a later form than this
branch was stacked on: the driver's own exec types replace fabro-sandbox's,
shell quoting moved to fabro-util, the sandbox lifecycle collapsed, and the
driver's events are stored as run events. This branch had deleted
`fabro-agent` and put the coding agent, the environment adapter, and the
steering hub on pebble.

The resolution takes main's sandbox API and re-applies pebble on top: the
`RunSandbox` `Environment` adapter moves to `pebble_environment.rs` (main's
`environment.rs` is the sandbox spec) and runs commands through `ExecSpec`
and `ExecControls`, feeding pebble's output sink from the driver's; the
driver-era `sandbox.*` names leave the known-event list, as on main, so a
stored event with that name and no driver shape is `Unknown` rather than an
error; `program_exit_code` matches pebble's non-exhaustive termination; the
Docker and Daytona smokes use main's constructor and credentials; the
remaining `fabro_agent` paths point at fabro-sandbox.

Pebble's `mcp` feature pins sandbox-driver, and the preview-url trait
objects only cross when both sides name one revision, so pebble moved to
main's `a92c0db6` (lithoscomputer/pebble#10) and fabro pins that pebble
revision until it lands on pebble main.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Bryan Helmkamp 2026-09-12 09:40:00 -06:00
commit 957fc97c5c
No known key found for this signature in database
133 changed files with 6429 additions and 10846 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
AGENTS.md
View file

@ -32,17 +32,19 @@ macOS note: if `cargo nextest run` fails with `Too many open files (os error 24)
- The packaged compose service mounts `/var/run/docker.sock` so the server can create sibling run containers on the host daemon. This is host-root-equivalent under Docker's security model; only use it in the trusted, single-tenant deployment model described by the sandbox code/docs.
- Docker and Daytona are clone-based providers. When a run manifest has a GitHub origin, they clone it into the provider workspace. Present non-GitHub origins fail unless the provider has `skip_clone = true`; absent origins or `skip_clone = true` create an empty workspace without repository files. For an exact commit, the submitted branch names the working branch and the syntactically valid SHA is requested directly. No layer proves branch/SHA ancestry: a fetchable commit is checked out, an unavailable commit fails setup, and branch HEAD is never substituted.
- The sandbox layer also accepts an optional exact commit for future admitted
runs. An exact commit always requires a non-empty branch. Docker initializes
an empty repository, shallow-fetches the SHA at the same depth as a branch
clone, and checks it out; Daytona uses its official SDK clone with both
`branch` and `commit_id`. Both providers then point the admitted branch at
the commit and verify HEAD, so the workspace still reports the admitted
branch name. Keep those provider transports distinct, never fall back to a
newer branch HEAD, and do not wire this capability directly from legacy
`GitContext.sha`. The sandbox layer does not verify that the commit is
reachable from the branch; admission owns that check. Current production
callers remain branch-only until the RunIntent admission cutover supplies a
validated branch/SHA pair.
runs. An exact commit always requires a non-empty branch. The sandbox driver
performs the pin the same way on every provider: it initializes an empty
repository, fetches the SHA directly at the requested depth, and attaches
the admitted branch to it, so the workspace reports the admitted branch
name. Daytona's native toolbox clone serves plain branch clones only; its
commit pin checks the branch head out first, so the driver does not use
it. A successful clone has the pin checked out; the driver's
conformance suite verifies that on every provider, and fabro does not
re-verify HEAD. Never fall back to a newer branch HEAD, and do not wire
this capability directly from legacy `GitContext.sha`. The sandbox layer
does not verify that the commit is reachable from the branch; admission
owns that check. Current production callers remain branch-only until the
RunIntent admission cutover supplies a validated branch/SHA pair.
### Release automation
- `cargo dev release` — creates the next stable release tag. Use `cargo dev release --nightly` for a nightly prerelease. Use `--dry-run` to print planned commands without mutating git or running Cargo, `--skip-tests` only after running the release-mode smoke yourself, and `--release-date YYYY-MM-DD` or `FABRO_RELEASE_DATE` for deterministic version computation.

152
Cargo.lock generated
View file

@ -2063,7 +2063,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -2190,7 +2190,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -2246,7 +2246,7 @@ dependencies = [
[[package]]
name = "fabro-acp"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"agent-client-protocol",
"agent-client-protocol-tokio",
@ -2266,7 +2266,7 @@ dependencies = [
[[package]]
name = "fabro-api"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"chrono",
"fabro-automation",
@ -2280,6 +2280,7 @@ dependencies = [
"progenitor-client",
"regress",
"reqwest 0.13.4",
"sandbox-driver",
"serde",
"serde_json",
"serde_yaml",
@ -2289,7 +2290,7 @@ dependencies = [
[[package]]
name = "fabro-auth"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -2314,7 +2315,7 @@ dependencies = [
[[package]]
name = "fabro-automation"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -2335,11 +2336,11 @@ dependencies = [
[[package]]
name = "fabro-build-support"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
[[package]]
name = "fabro-checkpoint"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"chrono",
"fabro-config",
@ -2355,7 +2356,7 @@ dependencies = [
[[package]]
name = "fabro-cli"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"assert_cmd",
@ -2433,6 +2434,7 @@ dependencies = [
"reqwest 0.13.4",
"ring",
"rustls",
"sandbox-driver",
"scopeguard",
"semver",
"serde",
@ -2458,7 +2460,7 @@ dependencies = [
[[package]]
name = "fabro-client"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"bytes",
@ -2487,7 +2489,7 @@ dependencies = [
[[package]]
name = "fabro-config"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -2516,7 +2518,7 @@ dependencies = [
[[package]]
name = "fabro-core"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"async-trait",
"fabro-types",
@ -2532,7 +2534,7 @@ dependencies = [
[[package]]
name = "fabro-db"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -2545,7 +2547,7 @@ dependencies = [
[[package]]
name = "fabro-dev"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"assert_cmd",
@ -2564,7 +2566,7 @@ dependencies = [
[[package]]
name = "fabro-dump"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"bytes",
@ -2578,7 +2580,7 @@ dependencies = [
[[package]]
name = "fabro-environment"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -2600,7 +2602,7 @@ dependencies = [
[[package]]
name = "fabro-github"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -2625,7 +2627,7 @@ dependencies = [
[[package]]
name = "fabro-graphviz"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"fabro-types",
@ -2640,7 +2642,7 @@ dependencies = [
[[package]]
name = "fabro-hooks"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"async-trait",
"fabro-auth",
@ -2665,7 +2667,7 @@ dependencies = [
[[package]]
name = "fabro-http"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"fabro-static",
"http 1.4.0",
@ -2675,7 +2677,7 @@ dependencies = [
[[package]]
name = "fabro-install"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"base64",
@ -2694,7 +2696,7 @@ dependencies = [
[[package]]
name = "fabro-interview"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"async-trait",
"dialoguer",
@ -2709,7 +2711,7 @@ dependencies = [
[[package]]
name = "fabro-llm"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -2738,7 +2740,7 @@ dependencies = [
[[package]]
name = "fabro-macros"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"clap",
"fabro-options-metadata",
@ -2749,7 +2751,7 @@ dependencies = [
[[package]]
name = "fabro-manifest"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"fabro-api",
@ -2773,7 +2775,7 @@ dependencies = [
[[package]]
name = "fabro-mcp"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"fabro-types",
@ -2786,7 +2788,7 @@ dependencies = [
[[package]]
name = "fabro-mcp-server"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -2813,7 +2815,7 @@ dependencies = [
[[package]]
name = "fabro-mcp-store"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"chrono",
"fabro-db",
@ -2831,7 +2833,7 @@ dependencies = [
[[package]]
name = "fabro-oauth"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"axum",
@ -2853,7 +2855,7 @@ dependencies = [
[[package]]
name = "fabro-options-metadata"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"serde",
"serde_json",
@ -2861,7 +2863,7 @@ dependencies = [
[[package]]
name = "fabro-proc"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"cc",
"libc",
@ -2870,7 +2872,7 @@ dependencies = [
[[package]]
name = "fabro-redact"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"aho-corasick",
"ref-cast",
@ -2886,7 +2888,7 @@ dependencies = [
[[package]]
name = "fabro-sandbox"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -2900,8 +2902,6 @@ dependencies = [
"fabro-types",
"fabro-util",
"futures",
"hex",
"hmac 0.12.1",
"pebble-coding-agent",
"reqwest 0.13.4",
"sandbox-driver",
@ -2914,7 +2914,6 @@ dependencies = [
"sandbox-driver-testing",
"serde",
"serde_json",
"sha2 0.10.9",
"strum 0.28.0",
"tempfile",
"thiserror 2.0.18",
@ -2927,7 +2926,7 @@ dependencies = [
[[package]]
name = "fabro-server"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -2995,6 +2994,7 @@ dependencies = [
"rand 0.9.4",
"regex",
"reqwest 0.12.28",
"sandbox-driver",
"semver",
"serde",
"serde_json",
@ -3024,7 +3024,7 @@ dependencies = [
[[package]]
name = "fabro-slack"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"fabro-http",
"fabro-interview",
@ -3046,18 +3046,18 @@ dependencies = [
[[package]]
name = "fabro-spa"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"rust-embed",
]
[[package]]
name = "fabro-static"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
[[package]]
name = "fabro-store"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"async-trait",
"bytes",
@ -3090,7 +3090,7 @@ dependencies = [
[[package]]
name = "fabro-telemetry"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"base64",
@ -3116,7 +3116,7 @@ dependencies = [
[[package]]
name = "fabro-template"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"fabro-types",
@ -3130,7 +3130,7 @@ dependencies = [
[[package]]
name = "fabro-test"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"assert_cmd",
@ -3155,7 +3155,7 @@ dependencies = [
[[package]]
name = "fabro-tool"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -3176,7 +3176,7 @@ dependencies = [
[[package]]
name = "fabro-tracker"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"async-trait",
@ -3190,7 +3190,7 @@ dependencies = [
[[package]]
name = "fabro-types"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"chrono",
"clap",
@ -3200,6 +3200,7 @@ dependencies = [
"hex",
"lithos-llm",
"pebble-coding-agent",
"sandbox-driver",
"serde",
"serde_json",
"sha2 0.10.9",
@ -3214,7 +3215,7 @@ dependencies = [
[[package]]
name = "fabro-util"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"console 0.15.11",
@ -3237,7 +3238,7 @@ dependencies = [
[[package]]
name = "fabro-validate"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"fabro-acp",
"fabro-graphviz",
@ -3250,7 +3251,7 @@ dependencies = [
[[package]]
name = "fabro-variable"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -3267,7 +3268,7 @@ dependencies = [
[[package]]
name = "fabro-vault"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"chrono",
@ -3286,7 +3287,7 @@ dependencies = [
[[package]]
name = "fabro-workflow"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"anyhow",
"assert_cmd",
@ -3358,7 +3359,7 @@ dependencies = [
[[package]]
name = "fabro-workflow-version"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"fabro-config",
"fabro-graphviz",
@ -4347,7 +4348,7 @@ dependencies = [
"js-sys",
"log",
"wasm-bindgen",
"windows-core 0.61.2",
"windows-core 0.62.2",
]
[[package]]
@ -5293,7 +5294,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -5858,7 +5859,7 @@ dependencies = [
[[package]]
name = "pebble-agent"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/pebble?rev=29da922597873afba3d9ae8c2fb73425685eda9e#29da922597873afba3d9ae8c2fb73425685eda9e"
source = "git+https://github.com/lithoscomputer/pebble?rev=9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f#9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f"
dependencies = [
"async-trait",
"futures-util",
@ -5875,7 +5876,7 @@ dependencies = [
[[package]]
name = "pebble-cli-core"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/pebble?rev=29da922597873afba3d9ae8c2fb73425685eda9e#29da922597873afba3d9ae8c2fb73425685eda9e"
source = "git+https://github.com/lithoscomputer/pebble?rev=9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f#9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f"
dependencies = [
"anyhow",
"async-trait",
@ -5904,7 +5905,7 @@ dependencies = [
[[package]]
name = "pebble-coding-agent"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/pebble?rev=29da922597873afba3d9ae8c2fb73425685eda9e#29da922597873afba3d9ae8c2fb73425685eda9e"
source = "git+https://github.com/lithoscomputer/pebble?rev=9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f#9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f"
dependencies = [
"async-trait",
"futures-util",
@ -6274,7 +6275,7 @@ dependencies = [
"once_cell",
"socket2",
"tracing",
"windows-sys 0.60.2",
"windows-sys 0.59.0",
]
[[package]]
@ -6751,7 +6752,7 @@ dependencies = [
"errno 0.3.14",
"libc",
"linux-raw-sys",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -6810,7 +6811,7 @@ dependencies = [
"security-framework",
"security-framework-sys",
"webpki-root-certs",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -6855,10 +6856,11 @@ dependencies = [
[[package]]
name = "sandbox-driver"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"async-trait",
"globset",
"humantime",
"rand 0.10.1",
"serde",
"serde_json",
@ -6871,13 +6873,14 @@ dependencies = [
[[package]]
name = "sandbox-driver-daytona"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"anyhow",
"async-trait",
"base64",
"daytona-api-client",
"daytona-sdk",
"hmac 0.12.1",
"rand 0.10.1",
"reqwest 0.13.4",
"sandbox-driver",
@ -6887,6 +6890,7 @@ dependencies = [
"sandbox-driver-protocol",
"serde",
"serde_json",
"sha2 0.10.9",
"tokio",
"tokio-util",
"tracing",
@ -6896,7 +6900,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-daytona-config"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"sandbox-driver-docker-config",
"serde",
@ -6906,7 +6910,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-docker"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"anyhow",
"async-trait",
@ -6927,7 +6931,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-docker-config"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"serde",
"serde_json",
@ -6936,7 +6940,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-host"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"anyhow",
"async-trait",
@ -6954,7 +6958,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-protocol"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"async-trait",
"base64",
@ -6971,7 +6975,7 @@ dependencies = [
[[package]]
name = "sandbox-driver-testing"
version = "0.1.0"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=23062b6ad62ff4665cbbcb7dce037ec9c4c34318#23062b6ad62ff4665cbbcb7dce037ec9c4c34318"
source = "git+https://github.com/lithoscomputer/sandbox-driver?rev=a92c0db6b6a122ca9b6df75de6615544f53c0d47#a92c0db6b6a122ca9b6df75de6615544f53c0d47"
dependencies = [
"async-trait",
"sandbox-driver",
@ -7454,7 +7458,7 @@ version = "1.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
dependencies = [
"errno 0.2.8",
"errno 0.3.14",
"libc",
]
@ -8019,7 +8023,7 @@ dependencies = [
"getrandom 0.4.1",
"once_cell",
"rustix",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -8054,7 +8058,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
dependencies = [
"rustix",
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@ -8547,7 +8551,7 @@ dependencies = [
[[package]]
name = "twin-github"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
dependencies = [
"axum",
"base64",
@ -9121,7 +9125,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]

46
Cargo.toml
View file

@ -10,7 +10,7 @@ resolver = "2"
[workspace.package]
edition = "2021"
version = "0.348.0-nightly.0"
version = "0.354.0-nightly.0"
license = "MIT"
[workspace.dependencies]
@ -101,28 +101,30 @@ futures-util = "0.3"
# sandbox-driver: the sandbox provider layer. Bundled Host, Docker, and
# Daytona providers link in-process; third-party providers run as stdio
# plugins through sandbox-driver-protocol. Pinned by rev; currently the head of
# the sandbox-driver PR stack #9-#15 (configured plugin kind, tag pins, classified
# git failures, stop grace, snapshot ensure, ownership scope, testing doubles), to
# move to main on merge. The CI plugin job installs the driver executables at the
# same rev, read from this file.
# the sandbox-driver `section-4-driver-items` branch (provider-owned scopes, the
# supervisor as provider, Host attach by directory, git retry and verbs in the
# driver, status image/snapshot/network, Daytona snapshot caching, services port
# wait and list, RFC 3339 timestamps), to move to main on merge. The CI plugin
# job installs the driver executables at the same rev, read from this file.
sandbox-driver = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-protocol = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-host = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-docker = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-docker-config = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-daytona = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-daytona-config = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
sandbox-driver-testing = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "a92c0db6b6a122ca9b6df75de6615544f53c0d47" }
# pebble: the coding agent loop fabro runs its agent stages, Ask Fabro
# sessions, hook evaluators, and `fabro exec` on. Pinned by rev to the pebble
# branch `embedder-concerns`, which merges the live exec output sink and
# `max_turns` into pebble's main (so fabro and petri pin one line); re-pin to
# the merge commit once it lands. Pebble pins the same lithos-llm rev as
# fabro, and its lockfile policy is that every shared crate resolves to the
# version lithos-llm locks.
pebble-agent = { git = "https://github.com/lithoscomputer/pebble", rev = "29da922597873afba3d9ae8c2fb73425685eda9e" }
pebble-coding-agent = { git = "https://github.com/lithoscomputer/pebble", rev = "29da922597873afba3d9ae8c2fb73425685eda9e", features = ["mcp", "search-providers"] }
pebble-cli-core = { git = "https://github.com/lithoscomputer/pebble", rev = "29da922597873afba3d9ae8c2fb73425685eda9e" }
sandbox-driver = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-protocol = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-host = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-docker = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-docker-config = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-daytona = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-daytona-config = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
sandbox-driver-testing = { git = "https://github.com/lithoscomputer/sandbox-driver", rev = "23062b6ad62ff4665cbbcb7dce037ec9c4c34318" }
# sessions, hook evaluators, and `fabro exec` on. Pinned by rev to pebble's
# `sandbox-driver-section-4` branch (lithoscomputer/pebble#10), which is
# pebble main plus the sandbox-driver pin below: the `PreviewUrls` trait
# objects fabro hands pebble's MCP servers only cross when both sides name
# one sandbox-driver revision. Re-pin to main once that lands. Pebble pins
# the same lithos-llm rev as fabro, and its lockfile policy is that every
# shared crate resolves to the version lithos-llm locks.
pebble-agent = { git = "https://github.com/lithoscomputer/pebble", rev = "9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f" }
pebble-coding-agent = { git = "https://github.com/lithoscomputer/pebble", rev = "9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f", features = ["mcp", "search-providers"] }
pebble-cli-core = { git = "https://github.com/lithoscomputer/pebble", rev = "9ec23d00d37faebcb2da5e3c5f2c1dfc5eec4d7f" }
sentry = { version = "0.35", default-features = false, features = ["backtrace", "contexts", "ureq", "rustls"] }
fork = "0.2"
exec = "0.3"

2
apps/fabro-web/app/components/run-summary-panel.test.tsx
View file

@ -174,7 +174,7 @@ describe("RunSummaryPanelView", () => {
const tree = render({
run: makeRun(),
sandboxState: "running",
sandboxResources: { cpu_cores: 4, memory_bytes: 8 * 1024 * 1024 * 1024 } as any,
sandboxResources: { cpu_cores: 4, memory_mb: 8 * 1024 },
});
expect(instanceText(cellAfterLabel(tree, "Sandbox"))).toBe("4 CPU · 8 GiB");
});

10
apps/fabro-web/app/components/run-summary-panel.tsx
View file

@ -80,10 +80,10 @@ function SandboxValue({
}) {
const display = SANDBOX_STATE_DISPLAY[state] ?? SANDBOX_STATE_DISPLAY.unknown;
const cpu = resources?.cpu_cores;
const memory = resources?.memory_bytes;
const memoryMb = resources?.memory_mb;
const valueText =
cpu != null && memory != null
? `${formatCpuCores(cpu)} CPU · ${formatBytesAsMemory(memory)}`
cpu != null && memoryMb != null
? `${formatCpuCores(cpu)} CPU · ${formatBytesAsMemory(memoryMb * 1024 * 1024)}`
: display.label;
return (
@ -221,8 +221,8 @@ export function RunSummaryPanel({ runId }: { runId: string }) {
<RunSummaryPanelView
run={runQuery.data ?? null}
runLoading={runQuery.isLoading && !runQuery.data}
sandboxState={sandboxQuery.data?.state ?? null}
sandboxResources={sandboxQuery.data?.resources ?? null}
sandboxState={sandboxQuery.data?.status.state ?? null}
sandboxResources={sandboxQuery.data?.status.resources ?? null}
sandboxLoading={sandboxReady && sandboxQuery.isLoading && !sandboxQuery.data}
artifactsCount={artifactsQuery.data?.data.length ?? null}
artifactsLoading={artifactsQuery.isLoading && !artifactsQuery.data}

88
apps/fabro-web/app/lib/sandbox-state.ts
View file

@ -11,29 +11,31 @@ export interface SandboxStateDisplay {
text: string;
}
const PENDING = { dot: "bg-amber", text: "text-amber" } as const;
const QUIET = { dot: "bg-fg-muted", text: "text-fg-muted" } as const;
const GONE = { dot: "bg-coral", text: "text-coral" } as const;
/**
* Display metadata for every normalized sandbox lifecycle state. Shared by the
* Display metadata for every sandbox driver lifecycle state. Shared by the
* run overview summary panel and the dedicated sandbox page so the dot color,
* label, and hover copy stay consistent.
* label, and hover copy stay consistent. A state this build does not know
* renders as `unknown`.
*/
export const SANDBOX_STATE_DISPLAY: Record<SandboxState, SandboxStateDisplay> = {
unknown: {
label: "Unknown",
description: "The sandbox state could not be determined.",
dot: "bg-fg-muted",
text: "text-fg-muted",
...QUIET,
},
provisioning: {
label: "Provisioning",
description: "The sandbox is being provisioned.",
dot: "bg-amber",
text: "text-amber",
creating: {
label: "Creating",
description: "The sandbox is being created.",
...PENDING,
},
starting: {
label: "Starting",
description: "The sandbox is starting up.",
dot: "bg-amber",
text: "text-amber",
...PENDING,
},
running: {
label: "Running",
@ -44,55 +46,71 @@ export const SANDBOX_STATE_DISPLAY: Record<SandboxState, SandboxStateDisplay> =
stopping: {
label: "Stopping",
description: "The sandbox is shutting down.",
dot: "bg-amber",
text: "text-amber",
...PENDING,
},
stopped: {
label: "Stopped",
description: "The sandbox is stopped.",
dot: "bg-fg-muted",
text: "text-fg-muted",
...QUIET,
},
pausing: {
label: "Pausing",
description: "The sandbox is being paused.",
...PENDING,
},
paused: {
label: "Paused",
description: "The sandbox is paused.",
dot: "bg-amber",
text: "text-amber",
...PENDING,
},
deleting: {
label: "Deleting",
description: "The sandbox is being deleted.",
dot: "bg-amber",
text: "text-amber",
resuming: {
label: "Resuming",
description: "The sandbox is resuming.",
...PENDING,
},
deleted: {
label: "Deleted",
description: "The sandbox has been deleted.",
dot: "bg-coral",
text: "text-coral",
archiving: {
label: "Archiving",
description: "The sandbox is being archived.",
...PENDING,
},
archived: {
label: "Archived",
description: "The sandbox has been archived.",
dot: "bg-fg-muted",
text: "text-fg-muted",
...QUIET,
},
restoring: {
label: "Restoring",
description: "The sandbox is being restored.",
dot: "bg-amber",
text: "text-amber",
...PENDING,
},
resizing: {
label: "Resizing",
description: "The sandbox resources are being resized.",
dot: "bg-amber",
text: "text-amber",
...PENDING,
},
forking: {
label: "Forking",
description: "The sandbox is being forked.",
...PENDING,
},
snapshotting: {
label: "Snapshotting",
description: "A snapshot of the sandbox is being taken.",
...PENDING,
},
deleting: {
label: "Deleting",
description: "The sandbox is being deleted.",
...PENDING,
},
deleted: {
label: "Deleted",
description: "The sandbox has been deleted.",
...GONE,
},
error: {
label: "Error",
description: "The sandbox encountered an error.",
dot: "bg-coral",
text: "text-coral",
...GONE,
},
};

128
apps/fabro-web/app/routes/run-sandbox.test.tsx
View file

@ -103,14 +103,15 @@ mock.restore();
const mountedRenderers: TestRenderer.ReactTestRenderer[] = [];
function sandboxDetails(
overrides: Partial<SandboxDetails> & {
overrides: {
sandbox?: Partial<SandboxDetails["sandbox"]> & {
runtime?: Partial<NonNullable<SandboxDetails["sandbox"]["runtime"]>>;
};
status?: Partial<SandboxDetails["status"]>;
} = {},
): SandboxDetails {
const sandbox = overrides.sandbox ?? {};
const { sandbox: _sandboxOverride, ...detailOverrides } = overrides;
const status = overrides.status ?? {};
return {
sandbox: {
provider: "docker",
@ -126,34 +127,27 @@ function sandboxDetails(
},
...sandbox,
},
state: "running",
native_state: null,
region: null,
resources: { cpu_cores: null, memory_bytes: null, disk_bytes: null },
network: networkDetails(),
labels: {},
timestamps: { created_at: null, last_activity_at: null },
...detailOverrides,
status: {
id: sandbox.runtime?.id ?? "",
state: "running",
provider_state: "",
error_reason: null,
resources: null,
sandbox_kind: null,
region: null,
labels: {},
image: null,
snapshot: null,
network: null,
workspace_ownership: null,
web_url: null,
created_at: null,
updated_at: null,
...status,
},
};
}
function networkDetails(
overrides: Partial<SandboxDetails["network"]> = {},
): SandboxDetails["network"] {
return {
egress: networkPolicy("unknown"),
ingress: networkPolicy("unknown"),
...overrides,
};
}
function networkPolicy(
mode: SandboxDetails["network"]["egress"]["mode"],
cidrs: string[] = [],
): SandboxDetails["network"]["egress"] {
return { mode, cidrs };
}
function textContent(renderer: TestRenderer.ReactTestRenderer): string {
return renderer.root
.findAll((node) => typeof node.type === "string")
@ -230,22 +224,18 @@ describe("RunSandbox route", () => {
working_directory: "/workspace",
},
},
state: "running",
native_state: "running",
region: undefined,
resources: {
cpu_cores: 2,
memory_bytes: 4 * 1024 * 1024 * 1024,
disk_bytes: undefined,
},
network: networkDetails({
egress: networkPolicy("open"),
ingress: networkPolicy("blocked"),
}),
labels: { run: "abc" },
timestamps: {
created_at: "2026-05-09T12:00:00Z",
last_activity_at: undefined,
status: {
state: "running",
provider_state: "running",
resources: {
cpu_cores: 2,
memory_mb: 4 * 1024,
disk_mb: null,
gpus: null,
},
network: "allow_all",
labels: { run: "abc" },
created_at: "2026-05-09T12:00:00Z",
},
});
const renderer = renderRoute();
@ -256,8 +246,8 @@ describe("RunSandbox route", () => {
.filter((text): text is string => typeof text === "string");
expect(panelHeadings).toEqual(["Overview", "Resources", "Network", "Labels", "Timestamps"]);
const copy = textContent(renderer);
expect(copy).toContain("Open");
expect(copy).toContain("Blocked");
expect(copy).toContain("Allow all");
expect(copy).toContain("4 GiB");
});
test("links to the provider dashboard when a sandbox web URL is present", () => {
@ -269,8 +259,10 @@ describe("RunSandbox route", () => {
working_directory: "/workspace",
},
},
web_url:
"https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9",
status: {
web_url:
"https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9",
},
});
const renderer = renderRoute();
@ -296,18 +288,12 @@ describe("RunSandbox route", () => {
working_directory: "/tmp/project",
},
},
state: "unknown",
native_state: undefined,
region: undefined,
resources: {
cpu_cores: undefined,
memory_bytes: undefined,
disk_bytes: undefined,
},
labels: {},
timestamps: {
created_at: undefined,
last_activity_at: undefined,
status: {
state: "unknown",
resources: { cpu_cores: null, memory_mb: null, disk_mb: null, gpus: null },
labels: {},
created_at: null,
updated_at: null,
},
});
const renderer = renderRoute();
@ -328,35 +314,29 @@ describe("RunSandbox route", () => {
expect(noLabelsCopy).toHaveLength(1);
});
test("renders unknown network policies", () => {
currentDetails = sandboxDetails({
network: networkDetails({
egress: networkPolicy("unknown"),
ingress: networkPolicy("unknown"),
}),
});
test("renders an unknown network policy", () => {
currentDetails = sandboxDetails({ status: { network: null } });
const renderer = renderRoute();
const copy = textContent(renderer);
expect(copy).toContain("Network");
expect(copy).toContain("Egress");
expect(copy).toContain("Ingress");
expect(copy).toContain("Policy");
expect(copy).toContain("Unknown");
});
test("renders blocked, essentials, and CIDR network policies", () => {
test("renders blocked and CIDR allow list network policies", () => {
currentDetails = sandboxDetails({
network: networkDetails({
egress: networkPolicy("cidr_allow_list", ["10.0.0.0/8", "192.168.0.0/16"]),
ingress: networkPolicy("essentials_only"),
}),
status: { network: { cidr_allow_list: { cidrs: ["10.0.0.0/8", "192.168.0.0/16"] } } },
});
const renderer = renderRoute();
const copy = textContent(renderer);
expect(copy).toContain("CIDR allow list");
expect(copy).toContain("10.0.0.0/8, 192.168.0.0/16");
expect(copy).toContain("Essentials only");
currentDetails = sandboxDetails({ status: { network: "block" } });
const blocked = renderRoute();
expect(textContent(blocked)).toContain("Blocked");
});
test("shows the empty state when no sandbox is reported", () => {

109
apps/fabro-web/app/routes/run-sandbox.tsx
View file

@ -22,7 +22,7 @@ import { SANDBOX_STATE_DISPLAY } from "../lib/sandbox-state";
import type {
RunSandbox,
SandboxDetails,
SandboxNetwork,
SandboxNetworkPolicy,
SandboxResources,
} from "@qltysh/fabro-api-client";
import FilesystemPanel from "./run-sandbox/filesystem-panel";
@ -57,27 +57,47 @@ function nullableTimestamp(value: string | null | undefined): string {
return value ? formatAbsoluteTs(value) : EMPTY_VALUE;
}
function nullableMemory(bytes: number | null | undefined): string {
return bytes != null ? formatBytesAsMemory(bytes) : EMPTY_VALUE;
function nullableMegabytes(megabytes: number | null | undefined): string {
return megabytes != null ? formatBytesAsMemory(megabytes * 1024 * 1024) : EMPTY_VALUE;
}
function nullableCpu(cores: number | null | undefined): string {
return cores != null ? formatCpuCores(cores) : EMPTY_VALUE;
}
type SandboxNetworkPolicy = SandboxNetwork["egress"];
type SandboxNetworkPolicyMode = SandboxNetworkPolicy["mode"];
function nullableCount(count: number | null | undefined): string {
return count != null ? String(count) : EMPTY_VALUE;
}
const NETWORK_POLICY_DISPLAY: Record<SandboxNetworkPolicyMode, string> = {
unknown: "Unknown",
open: "Open",
blocked: "Blocked",
cidr_allow_list: "CIDR allow list",
essentials_only: "Essentials only",
const NETWORK_POLICY_DISPLAY: Record<string, string> = {
provider_default: "Provider default",
allow_all: "Allow all",
block: "Blocked",
};
function networkPolicySummary(policy: SandboxNetworkPolicy): string {
return NETWORK_POLICY_DISPLAY[policy.mode] ?? policy.mode;
/** The policy's name, and the entries of an allow list when it carries one. */
function describeNetworkPolicy(
policy: SandboxNetworkPolicy | null | undefined,
): { summary: string; entries: { label: string; values: string[] } | null } {
if (policy == null) {
return { summary: "Unknown", entries: null };
}
if (typeof policy === "string") {
return { summary: NETWORK_POLICY_DISPLAY[policy] ?? policy, entries: null };
}
if ("cidr_allow_list" in policy) {
return {
summary: "CIDR allow list",
entries: { label: "Allowed CIDRs", values: policy.cidr_allow_list.cidrs },
};
}
if ("domain_allow_list" in policy) {
return {
summary: "Domain allow list",
entries: { label: "Allowed domains", values: policy.domain_allow_list.domains },
};
}
return { summary: "Unknown", entries: null };
}
interface RowProps {
@ -142,11 +162,12 @@ function Panel({ title, children }: PanelProps) {
}
function StatusStrip({ details }: { details: SandboxDetails }) {
const display = SANDBOX_STATE_DISPLAY[details.state] ?? SANDBOX_STATE_DISPLAY.unknown;
const status = details.status;
const display = SANDBOX_STATE_DISPLAY[status.state] ?? SANDBOX_STATE_DISPLAY.unknown;
const provider = details.sandbox.provider;
const providerState = status.provider_state ?? "";
const showNative =
details.native_state &&
details.native_state.toLowerCase() !== details.state.toLowerCase();
providerState.length > 0 && providerState.toLowerCase() !== status.state.toLowerCase();
return (
<div className="flex flex-wrap items-center gap-x-5 gap-y-2 rounded-md border border-line bg-panel/60 px-4 py-3 text-sm">
<span className="font-mono text-xs text-fg-muted uppercase tracking-wide">
@ -158,7 +179,7 @@ function StatusStrip({ details }: { details: SandboxDetails }) {
</span>
{showNative && (
<span className="font-mono text-xs text-fg-muted">
({details.native_state})
({providerState})
</span>
)}
</div>
@ -167,20 +188,25 @@ function StatusStrip({ details }: { details: SandboxDetails }) {
function OverviewPanel({ details }: { details: SandboxDetails }) {
const sandbox = details.sandbox;
const status = details.status;
const runtime = sandbox.runtime;
return (
<Panel title="Overview">
<Row label="ID" value={nullable(runtime?.id)} />
<Row label="ID" value={nullable(status.id || runtime?.id)} />
<Row label="Working directory" value={nullable(runtime?.working_directory)} />
<Row
label="Region"
value={details.region ? details.region : sandbox.provider === "docker" ? "local" : EMPTY_VALUE}
value={status.region ? status.region : sandbox.provider === "docker" ? "local" : EMPTY_VALUE}
/>
<Row label="Image" value={nullable(sandbox.image ?? sandbox.snapshot)} />
{details.web_url && (
<Row
label="Image"
value={nullable(status.image ?? status.snapshot ?? sandbox.image ?? sandbox.snapshot)}
/>
{status.sandbox_kind && <Row label="Kind" value={status.sandbox_kind} />}
{status.web_url && (
<LinkRow
label="Provider"
href={details.web_url}
href={status.web_url}
text={
sandbox.provider === "daytona"
? "Open in Daytona"
@ -192,29 +218,25 @@ function OverviewPanel({ details }: { details: SandboxDetails }) {
);
}
function ResourcesPanel({ resources }: { resources: SandboxResources }) {
function ResourcesPanel({ resources }: { resources: SandboxResources | null | undefined }) {
return (
<Panel title="Resources">
<Row label="CPU" value={nullableCpu(resources.cpu_cores)} />
<Row label="Memory" value={nullableMemory(resources.memory_bytes)} />
<Row label="Disk" value={nullableMemory(resources.disk_bytes)} />
<Row label="CPU" value={nullableCpu(resources?.cpu_cores)} />
<Row label="Memory" value={nullableMegabytes(resources?.memory_mb)} />
<Row label="Disk" value={nullableMegabytes(resources?.disk_mb)} />
{resources?.gpus != null && <Row label="GPUs" value={nullableCount(resources.gpus)} />}
</Panel>
);
}
function NetworkPanel({ network }: { network: SandboxNetwork }) {
const cidrRows: Array<{ label: string; policy: SandboxNetworkPolicy }> = [
{ label: "Egress CIDRs", policy: network.egress },
{ label: "Ingress CIDRs", policy: network.ingress },
].filter(({ policy }) => policy.mode === "cidr_allow_list");
function NetworkPanel({ network }: { network: SandboxNetworkPolicy | null | undefined }) {
const { summary, entries } = describeNetworkPolicy(network);
return (
<Panel title="Network">
<Row label="Egress" value={networkPolicySummary(network.egress)} />
<Row label="Ingress" value={networkPolicySummary(network.ingress)} />
{cidrRows.map(({ label, policy }) => (
<Row key={label} label={label} value={policy.cidrs.join(", ") || EMPTY_VALUE} />
))}
<Row label="Policy" value={summary} />
{entries && (
<Row label={entries.label} value={entries.values.join(", ") || EMPTY_VALUE} />
)}
</Panel>
);
}
@ -237,11 +259,8 @@ function LabelsPanel({ labels }: { labels: { [key: string]: string } | null | un
function TimestampsPanel({ details }: { details: SandboxDetails }) {
return (
<Panel title="Timestamps">
<Row label="Created" value={nullableTimestamp(details.timestamps.created_at)} />
<Row
label="Last activity"
value={nullableTimestamp(details.timestamps.last_activity_at)}
/>
<Row label="Created" value={nullableTimestamp(details.status.created_at)} />
<Row label="Last updated" value={nullableTimestamp(details.status.updated_at)} />
</Panel>
);
}
@ -259,9 +278,9 @@ function DetailsColumn({ details }: { details: SandboxDetails | null }) {
<div className="space-y-4">
<StatusStrip details={details} />
<OverviewPanel details={details} />
<ResourcesPanel resources={details.resources} />
<NetworkPanel network={details.network} />
<LabelsPanel labels={details.labels} />
<ResourcesPanel resources={details.status.resources} />
<NetworkPanel network={details.status.network} />
<LabelsPanel labels={details.status.labels} />
<TimestampsPanel details={details} />
</div>
);

45
apps/fabro-web/app/routes/run-sandbox/services-panel.test.tsx
View file

@ -26,10 +26,7 @@ function makeIdlePreview(): PreviewMutationShape {
}
function makeServicesData(data: SandboxService[]) {
return {
data,
meta: { source: "ss" as const },
};
return { data };
}
const mountedRenderers: TestRenderer.ReactTestRenderer[] = [];
@ -116,46 +113,6 @@ describe("ServicesPanelView", () => {
expect(titles).toHaveLength(1);
});
test("shows an iproute2 tip when services were discovered from procfs", () => {
const service: SandboxService = {
port: 3000,
addresses: ["0.0.0.0:3000"],
processes: [],
preview_supported: true,
};
const renderer = renderView({
servicesQuery: {
...makeIdleQuery(),
data: {
data: [service],
meta: { source: "procfs" },
},
},
previewMutation: makeIdlePreview(),
});
const tipLabels = renderer.root.findAll(
(node) =>
node.type === "span" &&
Array.isArray(node.children) &&
node.children.includes("Tip:"),
);
expect(tipLabels).toHaveLength(1);
const commands = renderer.root.findAll(
(node) =>
node.type === "code" &&
Array.isArray(node.children) &&
node.children.includes("apt-get install iproute2"),
);
expect(commands).toHaveLength(1);
const tipText = JSON.stringify(renderer.toJSON());
expect(tipText).toContain("Install ");
expect(tipText).toContain("ss");
expect(tipText).toContain(" in the sandbox for improved services listing:");
});
test("shows API error state with the error message", () => {
const renderer = renderView({
servicesQuery: {

13
apps/fabro-web/app/routes/run-sandbox/services-panel.tsx
View file

@ -77,7 +77,6 @@ export function ServicesPanelView({
const [previewError, setPreviewError] = useState<string | null>(null);
const services = servicesQuery.data?.data ?? [];
const discoverySource = servicesQuery.data?.meta.source;
const queryErrorMessage = describeQueryError(servicesQuery.error);
const showLoading = servicesQuery.isLoading && !servicesQuery.data;
const showError = queryErrorMessage !== null && !servicesQuery.data;
@ -150,7 +149,6 @@ export function ServicesPanelView({
<EmptyState title="No services" />
) : (
<>
{discoverySource === "procfs" ? <ProcfsDiscoveryTip /> : null}
<ServicesTable
services={services}
pendingPort={pendingPort}
@ -170,17 +168,6 @@ function describeQueryError(error: unknown): string | null {
return "Could not load services.";
}
function ProcfsDiscoveryTip() {
return (
<div className="mb-3 rounded-md border border-line bg-panel/60 px-3 py-2 text-xs leading-5 text-fg-3">
<span className="font-medium text-fg-2">Tip:</span>{" "}
Install <code className="font-mono text-fg-2">ss</code> in the sandbox
for improved services listing:{" "}
<code className="font-mono text-fg-2">apt-get install iproute2</code>
</div>
);
}
function ServicesTable({
services,
pendingPort,

13
docs/internal/events-strategy.md
View file

@ -126,10 +126,15 @@ Never build the same `RunEvent` twice if multiple sinks receive it.
### 1. Add the typed event
Add a variant to `Event`, `AgentEvent`, or `SandboxLifecycle` as appropriate. Sandbox
lifecycle facts come from two places: the pipeline emits `Initializing`, `Ready`, and
`InitializeFailed` around bringing the sandbox up, and `SandboxEventBridge` (in the
`fabro-workflow::event` module) translates the sandbox driver's own events — start,
stop, delete, image pulls, snapshot builds — into the rest. Fabro-sandbox emits no
facts come from two places: the pipeline emits `Initializing`, `Ready`, and
`InitializeFailed` around bringing the sandbox up, and the sandbox driver's own events
(operations and their outcome, progress inside a create such as an image pull, snapshot
builds, state observations, notices) are stored whole as `Event::SandboxDriver` by the
`DriverEventRecorder` in the `fabro-workflow::event` module. Their names derive from the
event (`fabro_types::sandbox_driver_event_name`): `<subject>.<action>.<phase>` such as
`sandbox.stop.completed` or `snapshot.create.started`, `<subject>.state`, and
`<subject>.notice`; their `properties` are the driver's event as the driver serializes
it, so the driver's `Event` is part of fabro's stored format. Fabro-sandbox emits no
events of its own.
### 2. Add tracing

96
docs/internal/events.md
View file

@ -1807,82 +1807,52 @@ Emitted after the engine completes sandbox initialization (distinct from `sandbo
| `provider` | string | Sandbox provider name |
| `error` | string | Error message |
### `sandbox.snapshot.pulling`
### Sandbox driver events
Emitted only when the Docker image cache misses and Fabro starts pulling the image.
Everything the sandbox driver reports about a run's sandbox is stored whole. The
event name derives from the driver's event: `<subject>.<action>.<phase>` for an
operation (`sandbox.start.started`, `sandbox.stop.completed`, `sandbox.delete.failed`,
`sandbox.create.progress` for an image pull inside the create, `snapshot.create.started`
and `snapshot.create.completed` for a snapshot build), `<subject>.state` for a state
observation, and `<subject>.notice` for a notice. `properties` is the driver's event as
the driver serializes it.
```json
{
"id": "...", "ts": "...", "run_id": "...",
"event": "sandbox.snapshot.pulling",
"event": "sandbox.stop.completed",
"properties": {
"name": "my-image:latest"
"id": {"source_id": "9b2f…", "sequence": 4},
"occurred_at": "2026-08-31T20:00:00Z",
"provider": "docker",
"subject": {"type": "sandbox", "id": "container-abc123"},
"operation_id": "58a1…",
"correlation_id": "01JQ…",
"type": "operation_completed",
"action": "stop",
"duration": {"secs": 1, "nanos": 250000000}
}
}
```
| Property | Type | Description |
|----------|------|-------------|
| `name` | string | Image/snapshot name |
| `id` | object | The driver's event id: `source_id` and `sequence` within that source |
| `occurred_at` | string | When the driver observed the event (RFC 3339) |
| `provider` | string | The driver's provider kind (`host`, `docker`, `daytona`, a plugin's kind) |
| `subject` | object | `type` (`sandbox`, `snapshot`, `volume`, `provider`) with the resource's `id` and `name` when known |
| `operation_id` | string | Groups the started, progress, and completed or failed events of one operation |
| `correlation_id` | string | The run id fabro attached |
| `type` | string | `operation_started`, `operation_progress`, `operation_completed`, `operation_failed`, `state_observed`, or `notice` |
| `action` | string | The operation (`create`, `start`, `stop`, `delete`, `snapshot`, …) on operation events |
| `progress` | object | `code` (`image.pull`, `snapshot.build`, …), `message`, and optional `completed`, `total`, `unit` on progress events |
| `duration` | object | `secs` and `nanos` on completed and failed events |
| `error` | object | `kind`, `message`, `retryable`, `causes` on failed events |
### `sandbox.snapshot.creating`
Emitted only when a Daytona snapshot cache miss or inactive snapshot requires Fabro to create or wait for the snapshot.
```json
{
"id": "...", "ts": "...", "run_id": "...",
"event": "sandbox.snapshot.creating",
"properties": {
"name": "my-snapshot"
}
}
```
| Property | Type | Description |
|----------|------|-------------|
| `name` | string | Snapshot name |
### `sandbox.snapshot.ready`
Emitted when an image or snapshot ensure step succeeds. Cache hits still emit this event with a near-zero `duration_ms`; explicit no-op paths such as Docker `auto_pull = false` and the Daytona default snapshot path do not.
```json
{
"id": "...", "ts": "...", "run_id": "...",
"event": "sandbox.snapshot.ready",
"properties": {
"name": "my-snapshot",
"duration_ms": 30000
}
}
```
| Property | Type | Description |
|----------|------|-------------|
| `name` | string | Snapshot name |
| `duration_ms` | number | Ensure duration |
### `sandbox.snapshot.failed`
Emitted when an image or snapshot ensure step fails.
```json
{
"id": "...", "ts": "...", "run_id": "...",
"event": "sandbox.snapshot.failed",
"properties": {
"name": "my-snapshot",
"error": "disk quota exceeded"
}
}
```
| Property | Type | Description |
|----------|------|-------------|
| `name` | string | Snapshot name |
| `error` | string | Error message |
| `causes` | string[] | Optional error cause chain |
Events stored under `sandbox.start.*`, `sandbox.stop.*`, `sandbox.delete.*`, and
`sandbox.snapshot.*` before the driver's events were kept whole carry fabro's earlier
`provider`, `name`, `duration_ms`, and `error` properties instead; readers treat them as
unknown bodies.
### `sandbox.git.started`

295
docs/public/api-reference/fabro-api.yaml
View file

@ -3907,7 +3907,7 @@ paths:
operationId: retrieveRunSandbox
tags: [Human-in-the-Loop]
summary: Retrieve Run Sandbox Details
description: Returns provider-neutral details about the sandbox owned by this run, including identity, normalized state, image/snapshot, resources, labels, and timestamps.
description: Returns the sandbox owned by this run as fabro's record of it plus the sandbox driver's status (identity, state, image or snapshot, resources, network policy, labels, and timestamps).
parameters:
- $ref: "#/components/parameters/RunId"
responses:
@ -13864,179 +13864,182 @@ components:
example: docker exec -it fabro-run-01HY0000000000000000000000 sh -lc 'cd /workspace/fabro && exec sh -l'
SandboxState:
description: Normalized sandbox lifecycle state used by the control plane and UI. The original provider-specific state string is preserved in `native_state`.
description: The sandbox driver's lifecycle state for a sandbox. The provider's own state string is preserved in `SandboxStatus.provider_state`. A reader must treat a value it does not know as `unknown`.
type: string
enum:
- unknown
- provisioning
- creating
- starting
- running
- stopping
- stopped
- pausing
- paused
- deleting
- deleted
- resuming
- archiving
- archived
- restoring
- resizing
- forking
- snapshotting
- deleting
- deleted
- error
- unknown
SandboxKind:
description: The kind of isolation a sandbox was provisioned with, as observed by the driver. Not an isolation guarantee.
type: string
enum:
- container
- virtual_machine
- unknown
SandboxWorkspaceOwnership:
description: Who owns a local sandbox's workspace directory. `designated` is a caller-owned directory that deleting the sandbox never touches; `managed` is a directory the driver created and removes.
type: string
enum:
- designated
- managed
SandboxResources:
description: Resource configuration for a sandbox. Fields are nullable when the provider does not surface a value or no limit is configured.
description: Compute resources of a sandbox, in the units the field names give. A field is null when the provider does not report a value or applies its default.
type: object
properties:
cpu_cores:
type: number
format: double
description: Configured CPU cores. Null when unavailable.
memory_bytes:
type: integer
type: ["integer", "null"]
format: int64
minimum: 0
description: Memory limit in bytes. Null when unavailable or unlimited.
disk_bytes:
type: integer
memory_mb:
type: ["integer", "null"]
format: int64
minimum: 0
disk_mb:
type: ["integer", "null"]
format: int64
minimum: 0
gpus:
type: ["integer", "null"]
format: int64
minimum: 0
description: Disk size in bytes. Null when unavailable.
SandboxNetworkPolicyMode:
description: Provider-neutral public-network policy for one direction.
type: string
enum:
- unknown
- open
- blocked
- cidr_allow_list
- essentials_only
SandboxNetworkPolicy:
description: Public-network policy for one direction.
description: The network policy in force for a sandbox. A policy without parameters is its name; an allow list carries its entries.
oneOf:
- type: string
enum:
- provider_default
- allow_all
- block
- type: object
required: [cidr_allow_list]
properties:
cidr_allow_list:
type: object
required: [cidrs]
properties:
cidrs:
type: array
items:
type: string
- type: object
required: [domain_allow_list]
properties:
domain_allow_list:
type: object
required: [domains]
properties:
domains:
type: array
items:
type: string
SandboxStatus:
description: What the sandbox driver reports about a sandbox. Only `id` and `state` are always present; every other field is null or empty when the provider does not report it.
type: object
required:
- mode
- cidrs
- id
- state
properties:
mode:
$ref: "#/components/schemas/SandboxNetworkPolicyMode"
cidrs:
type: array
items:
id:
type: string
description: The provider's stable identifier for the sandbox.
name:
type: ["string", "null"]
description: The provider's display name, which is not the stable identifier.
state:
$ref: "#/components/schemas/SandboxState"
provider_state:
type: string
default: ""
description: The provider's own state string, for display and debugging.
error_reason:
type: ["string", "null"]
resources:
oneOf:
- $ref: "#/components/schemas/SandboxResources"
- type: "null"
sandbox_kind:
oneOf:
- $ref: "#/components/schemas/SandboxKind"
- type: "null"
region:
type: ["string", "null"]
description: The provider region or target the sandbox runs in.
labels:
type: object
additionalProperties:
type: string
description: CIDR entries when `mode` is `cidr_allow_list`; empty for other modes.
SandboxNetwork:
description: Provider-neutral public-network policy for sandbox egress and ingress.
type: object
required:
- egress
- ingress
properties:
egress:
$ref: "#/components/schemas/SandboxNetworkPolicy"
ingress:
$ref: "#/components/schemas/SandboxNetworkPolicy"
SandboxTimestamps:
description: Lifecycle timestamps for a sandbox. Fields are nullable when the provider does not surface a value.
type: object
properties:
description: Provider-stored labels, including fabro's ownership labels.
image:
type: ["string", "null"]
description: The image the sandbox runs, when the provider knows it (a Docker container's image reference).
snapshot:
type: ["string", "null"]
description: The snapshot the sandbox was created from, when the provider knows it (a Daytona snapshot name).
network:
oneOf:
- $ref: "#/components/schemas/SandboxNetworkPolicy"
- type: "null"
description: The network policy in force, when the provider can read it back.
workspace_ownership:
oneOf:
- $ref: "#/components/schemas/SandboxWorkspaceOwnership"
- type: "null"
description: Local sandboxes only.
web_url:
type: ["string", "null"]
description: The provider's console page for the sandbox, when it has one.
created_at:
type: string
type: ["string", "null"]
format: date-time
description: When the sandbox was created.
last_activity_at:
type: string
updated_at:
type: ["string", "null"]
format: date-time
description: Most recent activity timestamp reported by the provider.
description: The provider's most recent activity or update timestamp for the sandbox.
SandboxDetails:
description: Provider-neutral details about the sandbox owned by a run.
description: The sandbox owned by a run, as fabro's record of it and the sandbox driver's status.
type: object
required:
- sandbox
- state
- resources
- network
- labels
- timestamps
- status
properties:
sandbox:
$ref: "#/components/schemas/RunSandboxInstance"
state:
$ref: "#/components/schemas/SandboxState"
native_state:
type: ["string", "null"]
description: Original provider state string before normalization. Display/debugging only; UI behavior keys off `state`.
region:
type: ["string", "null"]
description: Provider region or target. Null for local-style providers.
web_url:
type: ["string", "null"]
description: Provider dashboard URL for this sandbox when available.
resources:
$ref: "#/components/schemas/SandboxResources"
network:
$ref: "#/components/schemas/SandboxNetwork"
labels:
type: object
additionalProperties:
type: string
description: Provider-reported labels.
timestamps:
$ref: "#/components/schemas/SandboxTimestamps"
status:
$ref: "#/components/schemas/SandboxStatus"
SandboxInfo:
description: Provider-backed inventory record for a Fabro-managed sandbox.
description: One sandbox of fabro's provider-backed inventory, as the provider fabro connected it through and the sandbox driver's status.
type: object
required:
- provider
- id
- state
- resources
- network
- labels
- timestamps
- status
properties:
provider:
$ref: "#/components/schemas/SandboxProviderKind"
id:
type: string
description: Provider-native sandbox id.
display_name:
type: ["string", "null"]
description: Provider display name when distinct from the native id.
state:
$ref: "#/components/schemas/SandboxState"
native_state:
type: ["string", "null"]
description: Original provider state string before normalization. Display/debugging only; UI behavior keys off `state`.
image:
type: ["string", "null"]
description: Provider image when surfaced by the sandbox provider.
snapshot:
type: ["string", "null"]
description: Provider snapshot when surfaced by the sandbox provider.
region:
type: ["string", "null"]
description: Provider region or target. Null for local-style providers.
web_url:
type: ["string", "null"]
description: Provider dashboard URL for this sandbox when available.
working_directory:
type: ["string", "null"]
description: Provider-reported or Fabro-default working directory when available.
resources:
$ref: "#/components/schemas/SandboxResources"
network:
$ref: "#/components/schemas/SandboxNetwork"
labels:
type: object
additionalProperties:
type: string
description: Provider-reported labels.
timestamps:
$ref: "#/components/schemas/SandboxTimestamps"
status:
$ref: "#/components/schemas/SandboxStatus"
SandboxProviderLookupError:
description: Provider error captured during fail-soft sandbox inventory lookup.
@ -14105,7 +14108,7 @@ components:
$ref: "#/components/schemas/SandboxFileEntry"
SandboxService:
description: A listening TCP service discovered inside a run sandbox.
description: A TCP port a process inside a run sandbox listens on, as the sandbox driver reports it.
type: object
required:
- port
@ -14121,16 +14124,16 @@ components:
example: 3000
addresses:
type: array
description: Local bind addresses discovered from `ss` or `/proc/net/tcp*`.
description: Local bind addresses the sandbox reports for the port.
items:
type: string
example: ["127.0.0.1:3000", "[::]:3000"]
processes:
type: array
description: Visible process summaries when available. Empty when the sandbox only supports `/proc/net/tcp*` discovery.
description: The listening processes, when the sandbox can name them (`node`, or `pid=1234`). Empty when it cannot.
items:
type: string
example: ['users:(("node",pid=42,fd=23))']
example: ["node"]
preview_supported:
type: boolean
description: Whether the provider supports an external preview URL for this port.
@ -14141,30 +14144,11 @@ components:
type: object
required:
- data
- meta
properties:
data:
type: array
items:
$ref: "#/components/schemas/SandboxService"
meta:
$ref: "#/components/schemas/SandboxServiceListMeta"
SandboxServiceListMeta:
description: Metadata about sandbox service discovery.
type: object
required:
- source
properties:
source:
$ref: "#/components/schemas/SandboxServiceDiscoverySource"
SandboxServiceDiscoverySource:
description: Tool or kernel interface used to discover sandbox services.
type: string
enum:
- ss
- procfs
VncPreviewResponse:
description: Response containing a signed noVNC preview URL for a Daytona sandbox.
@ -14966,9 +14950,10 @@ components:
type: boolean
default: false
description: |
When true, Fabro-managed run-branch checkpoint commits bypass
local Git commit hooks. Does not affect Fabro `[[run.hooks]]`
or metadata-branch snapshots. Defaults to false.
Accepted for compatibility. Fabro-managed run-branch checkpoint
commits never run local Git commit hooks: the sandbox driver
disables repository hooks on every git command it runs. Does not
affect Fabro `[[run.hooks]]`. Defaults to false.
RunCloneSettings:
type: object

4
docs/public/execution/run-configuration.mdx
View file

@ -428,8 +428,8 @@ commit_timeout = "30s"
| Field | Description |
|---|---|
| `exclude_globs` | Glob patterns for files to exclude from checkpoint commits. Uses git pathspec `:(glob,exclude)` syntax. |
| `skip_git_hooks` | When `true`, Fabro-managed run-branch checkpoint commits bypass local Git commit hooks (e.g. `pre-commit`, `commit-msg`). Defaults to `false`. Does not affect Fabro workflow `[[run.hooks]]` or metadata-branch snapshots. |
| `commit_timeout` | Max duration for the per-node run-branch checkpoint commit (e.g. `"30s"`, `"10m"`). This commit runs repository commit hooks unless `skip_git_hooks` is `true`. Defaults to `"30s"`. |
| `skip_git_hooks` | Accepted for compatibility. Fabro-managed run-branch checkpoint commits never run local Git commit hooks (e.g. `pre-commit`, `commit-msg`); the sandbox driver disables repository hooks on every git command it runs. Does not affect Fabro workflow `[[run.hooks]]`. |
| `commit_timeout` | Accepted for compatibility. The per-node run-branch checkpoint commit runs under the sandbox driver's git command budget; no repository hook can prolong it. |
`exclude_globs` replaces across layers — the higher-precedence layer wins wholesale. `skip_git_hooks` and `commit_timeout` use normal override semantics: the highest layer that sets the field wins.

8
docs/public/integrations/daytona.mdx
View file

@ -126,9 +126,9 @@ Set either `image.docker` or `image.dockerfile`. `image.docker` can name any ima
dockerfile = "FROM node:20-slim\nRUN apt-get update && apt-get install -y git"
```
Fabro computes an internal snapshot name and looks up that snapshot in Daytona. If it does not exist, Fabro creates it automatically and polls until it reaches `Active` state for up to 30 minutes. A Dockerfile can be inline content or `{ path = "..." }`; paths are resolved relative to the TOML file that declares them and are bundled into run manifests. If the snapshot already exists, Fabro reuses it immediately.
The sandbox driver builds the image or Dockerfile into a Daytona snapshot named by its inputs (the image reference or Dockerfile text, the resources, and the Daytona API key) and creates the sandbox from it. If that snapshot already exists, it is reused immediately; otherwise the driver builds it and waits for it to reach `Active` state. A Dockerfile can be inline content or `{ path = "..." }`; paths are resolved relative to the TOML file that declares them and are bundled into run manifests.
The exact `image.docker` value is part of the snapshot identity. Prefer a digest such as `registry.example.com/team/image@sha256:...` when the image must be reproducible. If a mutable tag moves without its text changing, Fabro continues to reuse the existing snapshot.
The exact `image.docker` value is part of the snapshot identity. Prefer a digest such as `registry.example.com/team/image@sha256:...` when the image must be reproducible. If a mutable tag moves without its text changing, the existing snapshot continues to be reused.
<Note>
If neither image source is configured, sandboxes are created from the `daytona-medium` snapshot, which includes standard dev tools such as Git. To force a new Dockerfile snapshot, change the Dockerfile text, for example by adding a comment.
@ -237,11 +237,11 @@ If doctor reports missing scopes, regenerate the Daytona key with `write:snapsho
### Custom snapshot did not roll
Custom Daytona snapshot names are computed from the image reference or Dockerfile, resource hints, tenant scope, and Daytona API key. For `image.docker`, use an immutable digest and update it when the image changes. For `image.dockerfile`, change the Dockerfile text under the selected `[environments.<slug>.image]`.
Custom Daytona snapshot names (`sandbox-driver-<hex>`) are computed by the sandbox driver from the image reference or Dockerfile, the resources, and the Daytona API key. For `image.docker`, use an immutable digest and update it when the image changes. For `image.dockerfile`, change the Dockerfile text under the selected `[environments.<slug>.image]`.
### "Timed out waiting for snapshot to become active"
Snapshot creation took longer than 30 minutes. This can happen with large Dockerfiles. Check the snapshot status in the Daytona dashboard — it may still be building. Subsequent runs will reuse the snapshot once it's active.
Snapshot creation took longer than the sandbox driver's build budget. This can happen with large Dockerfiles. Check the snapshot status in the Daytona dashboard — it may still be building. Subsequent runs will reuse the snapshot once it's active.
### Git clone fails for private repositories

2
docs/public/integrations/github.mdx
View file

@ -303,7 +303,7 @@ Behavior notes:
Workflow authors may name any repository reachable by the server's GitHub App installation; Fabro applies no second server-side repository intersection. The token is scoped server-side to exactly the declared set — a request to an undeclared repository fails at GitHub, and Fabro never mints an unscoped installation-wide token. With `contents = "write"`, **any stage can push to any declared repository**. Declare the smallest repository set and the weakest permissions that work.
Installation Access Tokens are short-lived. Fabro refreshes its own credentials before checkpoint pushes. For ACP/CLI agent turns launched with GitHub App push credentials, Fabro also re-mints the token and rewrites the sandbox's `origin` URL before the ACP process starts, then every 45 minutes for the lifetime of that turn. Refresh failures are logged and do not fail the stage.
Installation Access Tokens are short-lived. Fabro's own pushes present a fresh token on each call. Git commands the agent runs inside the sandbox read the token through a credential store the sandbox driver configures for the checkout; the token never appears in the repository's remote URL or configuration. For ACP/CLI agent turns launched with GitHub App push credentials, Fabro re-mints the token and rewrites that store before the ACP process starts, then every 45 minutes for the lifetime of that turn. Refresh failures are logged and do not fail the stage.
`FABRO_PUSH_CRED_REFRESH_AHEAD` defaults to enabled; set it to `0`, `false`, `off`, `no`, or an empty value to disable both turn-entry and background refresh. `FABRO_PUSH_CRED_REFRESH_INTERVAL_SECONDS` overrides the background interval, and `0` disables only the background loop. This refresh loop is ACP-specific; command and native/API agent stages do not run it. Reconnected sandboxes for resumed or parked runs currently lack the App credentials needed for ACP refresh, so the refresh is skipped there.

2
lib/apps/fabro-cli/Cargo.toml
View file

@ -27,6 +27,7 @@ fabro-github = { path = "../../components/fabro-github" }
pebble-agent.workspace = true
pebble-coding-agent.workspace = true
pebble-cli-core.workspace = true
sandbox-driver.workspace = true
fabro-dump = { path = "../../components/fabro-dump" }
fabro-hooks = { path = "../../components/fabro-hooks" }
fabro-install = { path = "../../components/fabro-install" }
@ -120,6 +121,7 @@ assert_cmd = "2"
fabro-acp = { path = "../../components/fabro-acp", features = ["test-support"] }
fabro-mcp = { path = "../../components/fabro-mcp", features = ["test-support"] }
fabro-build-support = { path = "../../foundation/build-support" }
fabro-sandbox = { path = "../../components/fabro-sandbox", features = ["test-support"] }
fabro-server = { path = "../fabro-server", features = ["test-support"] }
fabro-workflow = { path = "../../components/fabro-workflow", features = ["test-support"] }
fabro-types = { path = "../../foundation/fabro-types", features = ["clap", "test-support"] }

65
lib/apps/fabro-cli/src/commands/run/events.rs
View file

@ -654,25 +654,35 @@ fn format_event_pretty_value(envelope: &serde_json::Value, styles: &Styles) -> O
styles.dim.apply_to(&duration),
))
}
"sandbox.snapshot.pulling" => {
let name = prop_str_field(envelope, "name").unwrap_or("?");
"sandbox.create.progress" => {
let code = envelope
.pointer("/properties/progress/code")
.and_then(serde_json::Value::as_str)?;
if code != "image.pull" {
return None;
}
let message = envelope
.pointer("/properties/progress/message")
.and_then(serde_json::Value::as_str)
.unwrap_or("image");
let name = message.strip_prefix("pulling image ").unwrap_or(message);
Some(format!(
"{} Sandbox: pulling {}",
styles.dim.apply_to(&ts),
name,
))
}
"sandbox.snapshot.creating" => {
let name = prop_str_field(envelope, "name").unwrap_or("?");
"snapshot.create.started" => {
let name = driver_subject_name(envelope);
Some(format!(
"{} Sandbox: building {}",
styles.dim.apply_to(&ts),
name,
))
}
"sandbox.snapshot.ready" => {
let name = prop_str_field(envelope, "name").unwrap_or("?");
let duration = format_duration_ms(prop_field(envelope, "duration_ms"));
"snapshot.create.completed" => {
let name = driver_subject_name(envelope);
let duration = format_duration_ms(driver_duration_ms(envelope).as_ref());
Some(format!(
"{} Sandbox snapshot: {} {}",
styles.dim.apply_to(&ts),
@ -680,9 +690,12 @@ fn format_event_pretty_value(envelope: &serde_json::Value, styles: &Styles) -> O
styles.dim.apply_to(&duration),
))
}
"sandbox.snapshot.failed" => {
let name = prop_str_field(envelope, "name").unwrap_or("?");
let error = prop_str_field(envelope, "error").unwrap_or("unknown error");
"snapshot.create.failed" => {
let name = driver_subject_name(envelope);
let error = envelope
.pointer("/properties/error/message")
.and_then(serde_json::Value::as_str)
.unwrap_or("unknown error");
Some(format!(
"{} {} Sandbox snapshot {} failed: {}",
styles.dim.apply_to(&ts),
@ -809,6 +822,30 @@ fn str_field<'a>(value: &'a serde_json::Value, key: &str) -> Option<&'a str> {
value.get(key)?.as_str()
}
/// The name of the resource a sandbox driver event is about, falling back
/// to its id.
fn driver_subject_name(envelope: &serde_json::Value) -> &str {
envelope
.pointer("/properties/subject/name")
.or_else(|| envelope.pointer("/properties/subject/id"))
.and_then(serde_json::Value::as_str)
.unwrap_or("?")
}
/// A sandbox driver operation's duration, in milliseconds, as the number
/// [`format_duration_ms`] reads.
fn driver_duration_ms(envelope: &serde_json::Value) -> Option<serde_json::Value> {
let duration = envelope.pointer("/properties/duration")?;
let secs = duration.get("secs").and_then(serde_json::Value::as_u64)?;
let nanos = duration
.get("nanos")
.and_then(serde_json::Value::as_u64)
.unwrap_or(0);
Some(serde_json::Value::from(
secs.saturating_mul(1000).saturating_add(nanos / 1_000_000),
))
}
fn prop_field<'a>(value: &'a serde_json::Value, key: &str) -> Option<&'a serde_json::Value> {
value.get("properties")?.get(key)
}
@ -1261,7 +1298,7 @@ mod tests {
#[test]
fn pretty_sandbox_snapshot_pulling() {
let styles = no_color_styles();
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"sandbox.snapshot.pulling","properties":{"name":"buildpack-deps:noble"}}"#;
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"sandbox.create.progress","properties":{"id":{"source_id":"t","sequence":1},"occurred_at":"2026-01-01T14:25:00Z","provider":"docker","subject":{"type":"sandbox"},"type":"operation_progress","action":"create","progress":{"code":"image.pull","message":"pulling image buildpack-deps:noble"}}}"#;
let result = format_event_pretty(line, &styles).unwrap();
assert!(result.contains("Sandbox: pulling"), "got: {result}");
assert!(result.contains("buildpack-deps:noble"), "got: {result}");
@ -1270,7 +1307,7 @@ mod tests {
#[test]
fn pretty_sandbox_snapshot_creating() {
let styles = no_color_styles();
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"sandbox.snapshot.creating","properties":{"name":"fabro-v9-test"}}"#;
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"snapshot.create.started","properties":{"id":{"source_id":"t","sequence":1},"occurred_at":"2026-01-01T14:25:00Z","provider":"daytona","subject":{"type":"snapshot","name":"fabro-v9-test"},"type":"operation_started","action":"create"}}"#;
let result = format_event_pretty(line, &styles).unwrap();
assert!(result.contains("Sandbox: building"), "got: {result}");
assert!(result.contains("fabro-v9-test"), "got: {result}");
@ -1279,7 +1316,7 @@ mod tests {
#[test]
fn pretty_sandbox_snapshot_ready() {
let styles = no_color_styles();
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"sandbox.snapshot.ready","properties":{"name":"buildpack-deps:noble","duration_ms":8200}}"#;
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"snapshot.create.completed","properties":{"id":{"source_id":"t","sequence":1},"occurred_at":"2026-01-01T14:25:00Z","provider":"daytona","subject":{"type":"snapshot","name":"buildpack-deps:noble"},"type":"operation_completed","action":"create","duration":{"secs":8,"nanos":200000000}}}"#;
let result = format_event_pretty(line, &styles).unwrap();
assert!(result.contains("Sandbox snapshot:"), "got: {result}");
assert!(result.contains("buildpack-deps:noble"), "got: {result}");
@ -1289,7 +1326,7 @@ mod tests {
#[test]
fn pretty_sandbox_snapshot_failed() {
let styles = no_color_styles();
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"sandbox.snapshot.failed","properties":{"name":"buildpack-deps:noble","error":"pull failed"}}"#;
let line = r#"{"ts":"2026-01-01T14:25:00Z","event":"snapshot.create.failed","properties":{"id":{"source_id":"t","sequence":1},"occurred_at":"2026-01-01T14:25:00Z","provider":"docker","subject":{"type":"snapshot","name":"buildpack-deps:noble"},"type":"operation_failed","action":"create","duration":{"secs":1,"nanos":0},"error":{"kind":"provider","message":"pull failed","retryable":false,"causes":[]}}}"#;
let result = format_event_pretty(line, &styles).unwrap();
assert!(
result.contains("Sandbox snapshot buildpack-deps:noble failed: pull failed"),

175
lib/apps/fabro-cli/src/commands/run/run_progress/event.rs
View file

@ -257,20 +257,7 @@ pub(super) fn from_run_event(stored: &RunEvent) -> Option<ProgressEvent> {
provider: props.provider.clone(),
error: props.error.clone(),
}),
EventBody::SnapshotPulling(props) => Some(ProgressEvent::SnapshotPulling {
name: props.name.clone(),
}),
EventBody::SnapshotCreating(props) => Some(ProgressEvent::SnapshotCreating {
name: props.name.clone(),
}),
EventBody::SnapshotReady(props) => Some(ProgressEvent::SnapshotReady {
name: props.name.clone(),
duration_ms: props.duration_ms,
}),
EventBody::SnapshotFailed(props) => Some(ProgressEvent::SnapshotFailed {
name: props.name.clone(),
error: props.error.clone(),
}),
EventBody::SandboxDriver { event, .. } => driver_progress_event(event),
EventBody::SshAccessReady(props) => Some(ProgressEvent::SshAccessReady {
ssh_command: props.ssh_command.clone(),
}),
@ -537,6 +524,65 @@ fn agent_progress_event(
}
}
/// The setup progress a sandbox driver event stands for: the image pull
/// inside the sandbox's create, or a snapshot build. Every other driver
/// event is stored on the run but renders nothing here.
fn driver_progress_event(event: &sandbox_driver::Event) -> Option<ProgressEvent> {
use sandbox_driver::{Action, EventBody as Body, EventSubject, ProgressCode};
match (&event.subject, &event.body) {
(
EventSubject::Sandbox { .. },
Body::OperationProgress {
action: Action::Create,
progress,
},
) if progress.code.as_str() == ProgressCode::IMAGE_PULL => {
Some(ProgressEvent::SnapshotPulling {
name: pulled_image_name(progress.message.as_deref()),
})
}
(EventSubject::Snapshot { id, name }, body) => {
let name = name
.clone()
.or_else(|| id.as_ref().map(ToString::to_string))
.unwrap_or_default();
match body {
Body::OperationStarted {
action: Action::Create,
} => Some(ProgressEvent::SnapshotCreating { name }),
Body::OperationCompleted {
action: Action::Create,
duration,
} => Some(ProgressEvent::SnapshotReady {
name,
duration_ms: u64::try_from(duration.as_millis()).unwrap_or(u64::MAX),
}),
Body::OperationFailed {
action: Action::Create,
error,
..
} => Some(ProgressEvent::SnapshotFailed {
name,
error: error.message.clone(),
}),
_ => None,
}
}
_ => None,
}
}
/// The image an image pull progress report names. The Docker provider
/// says `pulling image <reference>`; the reference alone reads better.
fn pulled_image_name(message: Option<&str>) -> String {
let message = message.unwrap_or("image");
message
.strip_prefix("pulling image ")
.unwrap_or(message)
.to_owned()
}
#[cfg(test)]
mod tests {
use fabro_types::{MetadataSnapshotFailureKind, MetadataSnapshotPhase, fixtures};
@ -810,32 +856,76 @@ mod tests {
));
}
#[test]
fn round_trip_snapshot_lifecycle_events() {
let pulling = to_run_event(&fixtures::RUN_1, &Event::Sandbox {
event: SandboxLifecycle::SnapshotPulling {
name: "buildpack-deps:noble".into(),
},
});
let creating = to_run_event(&fixtures::RUN_1, &Event::Sandbox {
event: SandboxLifecycle::SnapshotCreating {
name: "fabro-v9".into(),
},
});
let ready = to_run_event(&fixtures::RUN_1, &Event::Sandbox {
event: SandboxLifecycle::SnapshotReady {
name: "buildpack-deps:noble".into(),
duration_ms: 1200,
},
});
let failed = to_run_event(&fixtures::RUN_1, &Event::Sandbox {
event: SandboxLifecycle::SnapshotFailed {
name: "fabro-v9".into(),
error: "build failed".into(),
causes: Vec::new(),
},
});
fn driver_event(value: serde_json::Value) -> Event {
Event::SandboxDriver {
event: serde_json::from_value(value).expect("a driver event"),
}
}
#[test]
fn round_trip_driver_events_that_render_setup_progress() {
let pulling = to_run_event(
&fixtures::RUN_1,
&driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 1},
"occurred_at": "2026-01-01T00:00:00Z",
"provider": "docker",
"subject": {"type": "sandbox"},
"type": "operation_progress",
"action": "create",
"progress": {"code": "image.pull", "message": "pulling image buildpack-deps:noble"}
})),
);
let creating = to_run_event(
&fixtures::RUN_1,
&driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 2},
"occurred_at": "2026-01-01T00:00:00Z",
"provider": "daytona",
"subject": {"type": "snapshot", "name": "fabro-v9"},
"type": "operation_started",
"action": "create"
})),
);
let ready = to_run_event(
&fixtures::RUN_1,
&driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 3},
"occurred_at": "2026-01-01T00:00:01Z",
"provider": "daytona",
"subject": {"type": "snapshot", "name": "fabro-v9"},
"type": "operation_completed",
"action": "create",
"duration": {"secs": 1, "nanos": 200_000_000}
})),
);
let failed = to_run_event(
&fixtures::RUN_1,
&driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 4},
"occurred_at": "2026-01-01T00:00:02Z",
"provider": "daytona",
"subject": {"type": "snapshot", "name": "fabro-v9"},
"type": "operation_failed",
"action": "create",
"duration": {"secs": 2, "nanos": 0},
"error": {"kind": "provider", "message": "build failed", "retryable": false, "causes": []}
})),
);
let stopped = to_run_event(
&fixtures::RUN_1,
&driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 5},
"occurred_at": "2026-01-01T00:00:03Z",
"provider": "docker",
"subject": {"type": "sandbox", "id": "c1"},
"type": "operation_completed",
"action": "stop",
"duration": {"secs": 0, "nanos": 0}
})),
);
assert_eq!(pulling.event_name(), "sandbox.create.progress");
assert!(matches!(
from_run_event(&pulling).unwrap(),
ProgressEvent::SnapshotPulling { name } if name == "buildpack-deps:noble"
@ -847,13 +937,18 @@ mod tests {
assert!(matches!(
from_run_event(&ready).unwrap(),
ProgressEvent::SnapshotReady { name, duration_ms }
if name == "buildpack-deps:noble" && duration_ms == 1200
if name == "fabro-v9" && duration_ms == 1200
));
assert!(matches!(
from_run_event(&failed).unwrap(),
ProgressEvent::SnapshotFailed { name, error }
if name == "fabro-v9" && error == "build failed"
));
assert_eq!(stopped.event_name(), "sandbox.stop.completed");
assert!(
from_run_event(&stopped).is_none(),
"a stop is stored on the run but renders no setup progress"
);
}
#[test]

101
lib/apps/fabro-cli/src/commands/run/run_progress/mod.rs
View file

@ -507,6 +507,55 @@ mod tests {
.expect("valid utf-8")
}
fn driver_event(value: serde_json::Value) -> Event {
Event::SandboxDriver {
event: serde_json::from_value(value).expect("a driver event"),
}
}
/// A snapshot build reported by the driver: started, or completed after
/// `secs`.
fn snapshot_build_event(name: &str, kind: &str, secs: Option<u64>) -> Event {
let mut value = serde_json::json!({
"id": {"source_id": "test", "sequence": 1},
"occurred_at": "2026-01-01T00:00:00Z",
"provider": "daytona",
"subject": {"type": "snapshot", "name": name},
"type": kind,
"action": "create"
});
if let Some(secs) = secs {
value["duration"] = serde_json::json!({"secs": secs, "nanos": 0});
}
driver_event(value)
}
fn snapshot_build_failed_event(name: &str, error: &str) -> Event {
driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 1},
"occurred_at": "2026-01-01T00:00:00Z",
"provider": "docker",
"subject": {"type": "snapshot", "name": name},
"type": "operation_failed",
"action": "create",
"duration": {"secs": 1, "nanos": 0},
"error": {"kind": "provider", "message": error, "retryable": false, "causes": []}
}))
}
/// The Docker provider pulling the sandbox's image inside its create.
fn image_pull_event(image: &str) -> Event {
driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 1},
"occurred_at": "2026-01-01T00:00:00Z",
"provider": "docker",
"subject": {"type": "sandbox"},
"type": "operation_progress",
"action": "create",
"progress": {"code": "image.pull", "message": format!("pulling image {image}")}
}))
}
fn emit(ui: &mut ProgressUI, event: Event) {
let stored = to_run_event(&fixtures::RUN_1, &event);
ui.handle_event(&stored);
@ -1080,17 +1129,14 @@ mod tests {
provider: "daytona".into(),
},
});
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotCreating {
name: "fabro-v9-test".into(),
},
});
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotReady {
name: "fabro-v9-test".into(),
duration_ms: 210_000,
},
});
emit(
&mut ui,
snapshot_build_event("fabro-v9-test", "operation_started", None),
);
emit(
&mut ui,
snapshot_build_event("fabro-v9-test", "operation_completed", Some(210)),
);
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::Ready {
provider: "daytona".into(),
@ -1116,17 +1162,7 @@ mod tests {
provider: "docker".into(),
},
});
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotPulling {
name: "buildpack-deps:noble".into(),
},
});
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotReady {
name: "buildpack-deps:noble".into(),
duration_ms: 8_200,
},
});
emit(&mut ui, image_pull_event("buildpack-deps:noble"));
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::Ready {
provider: "docker".into(),
@ -1172,13 +1208,10 @@ mod tests {
provider: "docker".into(),
},
});
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotFailed {
name: "buildpack-deps:noble".into(),
error: "pull failed".into(),
causes: Vec::new(),
},
});
emit(
&mut ui,
snapshot_build_failed_event("buildpack-deps:noble", "pull failed"),
);
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::InitializeFailed {
provider: "docker".into(),
@ -1205,12 +1238,10 @@ mod tests {
});
assert!(ui.setup.sandbox_bar.is_some());
emit(&mut ui, Event::Sandbox {
event: SandboxLifecycle::SnapshotReady {
name: "buildpack-deps:noble".into(),
duration_ms: 10,
},
});
emit(
&mut ui,
snapshot_build_event("buildpack-deps:noble", "operation_completed", Some(0)),
);
assert!(ui.setup.sandbox_bar.is_some());
emit(&mut ui, Event::Sandbox {

92
lib/apps/fabro-cli/tests/it/cmd/attach.rs
View file

@ -1097,6 +1097,91 @@ fn attach_json_errors_without_prompting_for_human_input() {
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"
},
{
"actor": {
"kind": "worker",
"run_id": "[ULID]"
},
"event": "sandbox.create.started",
"id": "[EVENT_ID]",
"properties": {
"action": "create",
"correlation_id": "[ULID]",
"id": {
"sequence": 1,
"source_id": "[HEX]"
},
"occurred_at": "[TIMESTAMP]",
"operation_id": "[HEX]",
"provider": "host",
"subject": {
"id": "host-dir-[HEX]",
"type": "sandbox"
},
"type": "operation_started"
},
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"
},
{
"actor": {
"kind": "worker",
"run_id": "[ULID]"
},
"event": "sandbox.create.progress",
"id": "[EVENT_ID]",
"properties": {
"action": "create",
"correlation_id": "[ULID]",
"id": {
"sequence": 2,
"source_id": "[HEX]"
},
"occurred_at": "[TIMESTAMP]",
"operation_id": "[HEX]",
"progress": {
"code": "sandbox.provision"
},
"provider": "host",
"subject": {
"id": "host-dir-[HEX]",
"type": "sandbox"
},
"type": "operation_progress"
},
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"
},
{
"actor": {
"kind": "worker",
"run_id": "[ULID]"
},
"event": "sandbox.create.completed",
"id": "[EVENT_ID]",
"properties": {
"action": "create",
"correlation_id": "[ULID]",
"duration": {
"nanos": "[NANOS]",
"secs": 0
},
"id": {
"sequence": 3,
"source_id": "[HEX]"
},
"occurred_at": "[TIMESTAMP]",
"operation_id": "[HEX]",
"provider": "host",
"subject": {
"id": "host-dir-[HEX]",
"type": "sandbox"
},
"type": "operation_completed"
},
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"
},
{
"actor": {
"kind": "worker",
@ -1119,9 +1204,12 @@ fn attach_json_errors_without_prompting_for_human_input() {
"event": "sandbox.initialized",
"id": "[EVENT_ID]",
"properties": {
"id": "local:[ULID]",
"id": "host-dir-[HEX]",
"provider": "local",
"working_directory": "[TEMP_DIR]"
"repo_cloned": false,
"repos_root": "[TEMP_DIR]/.repos",
"working_directory": "[TEMP_DIR]",
"workspace_root": "[TEMP_DIR]"
},
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"

6
lib/apps/fabro-cli/tests/it/cmd/dump.rs
View file

@ -261,9 +261,9 @@ fn dump_exports_completed_run_snapshot() {
");
assert_snapshot!(dump_file_summary(&output_dir), @"
checkpoints/0014.json
checkpoints/0018.json
checkpoints/0022.json
checkpoints/0017.json
checkpoints/0021.json
checkpoints/0025.json
events.jsonl
graph.fabro
run.json

20
lib/apps/fabro-cli/tests/it/cmd/run.rs
View file

@ -993,8 +993,24 @@ fn dry_run_persists_event_history_in_store() {
"event": "sandbox.stop.completed",
"id": "[EVENT_ID]",
"properties": {
"duration_ms": "[DURATION_MS]",
"provider": "local"
"action": "stop",
"correlation_id": "[ULID]",
"duration": {
"nanos": "[NANOS]",
"secs": 0
},
"id": {
"sequence": 5,
"source_id": "[HEX]"
},
"occurred_at": "[TIMESTAMP]",
"operation_id": "[HEX]",
"provider": "host",
"subject": {
"id": "host-dir-[HEX]",
"type": "sandbox"
},
"type": "operation_completed"
},
"run_id": "[ULID]",
"ts": "[TIMESTAMP]"

4
lib/apps/fabro-cli/tests/it/cmd/support.rs
View file

@ -1107,7 +1107,7 @@ async fn append_seeded_simple_completion_events(
serde_json::json!({
"working_directory": context.temp_dir.display().to_string(),
"provider": "local",
"id": format!("local:{}", run.run_id),
"id": fabro_sandbox::test_support::local_sandbox_id(&context.temp_dir).await,
"repo_cloned": false,
"clone_origin_url": null,
"clone_branch": null,
@ -1276,7 +1276,7 @@ async fn append_seeded_git_completion_events(
serde_json::json!({
"working_directory": context.temp_dir.display().to_string(),
"provider": "local",
"id": format!("local:{}", run.run_id),
"id": fabro_sandbox::test_support::local_sandbox_id(&context.temp_dir).await,
"repo_cloned": false,
"clone_origin_url": null,
"clone_branch": null,

1
lib/apps/fabro-server/Cargo.toml
View file

@ -35,6 +35,7 @@ fabro-workflow = { path = "../../components/fabro-workflow" }
fabro-workflow-version = { path = "../../components/fabro-workflow-version" }
fabro-validate = { path = "../../components/fabro-validate" }
fabro-sandbox = { path = "../../components/fabro-sandbox" }
sandbox-driver.workspace = true
fabro-github = { path = "../../components/fabro-github" }
pebble-agent.workspace = true
pebble-coding-agent.workspace = true

8
lib/apps/fabro-server/src/demo/mod.rs
View file

@ -23,7 +23,6 @@ use fabro_api::types::{
RunFilesMeta, RunFilesMetaScope, RunFilesMetaSource, SandboxService,
SandboxServiceListResponse,
};
use fabro_types::{SandboxServiceDiscoverySource, SandboxServiceListMeta};
use serde_json::json;
use crate::error::ApiError;
@ -405,19 +404,16 @@ pub(crate) async fn list_sandbox_services_stub(
SandboxService {
port: 3000,
addresses: vec!["0.0.0.0:3000".to_string()],
processes: vec![r#"users:(("node",pid=42,fd=23))"#.to_string()],
processes: vec!["node".to_string()],
preview_supported: true,
},
SandboxService {
port: 2500,
addresses: vec!["127.0.0.1:2500".to_string()],
processes: vec![r#"users:(("debug",pid=84,fd=19))"#.to_string()],
processes: vec!["debug".to_string()],
preview_supported: false,
},
],
meta: SandboxServiceListMeta {
source: SandboxServiceDiscoverySource::Ss,
},
}),
)
.into_response()

2
lib/apps/fabro-server/src/diagnostics.rs
View file

@ -691,7 +691,7 @@ fn cloud_sandbox_probe_check(probe: anyhow::Result<daytona::DaytonaKeyCheck>) ->
remediation: Some(format!(
"Regenerate the Daytona API key with scopes: {}, then \
`fabro secret set DAYTONA_API_KEY`.",
daytona::required_perms_display()
check.required_display()
)),
},
Err(err) => {

13
lib/apps/fabro-server/src/install.rs
View file

@ -1005,13 +1005,12 @@ async fn check_install_daytona_api_key(
state: &InstallAppState,
api_key: String,
) -> anyhow::Result<daytona::DaytonaKeyCheck> {
let credentials = DaytonaCredentials {
api_key,
api_url: state.upstreams.daytona_api_base_url.clone(),
organization_id: state.upstreams.daytona_organization_id.clone(),
target: None,
http_client: Some(fabro_http::http_client().context("failed to build HTTP client")?),
};
let credentials = DaytonaCredentials::new(api_key)
.with_api_url(state.upstreams.daytona_api_base_url.clone())
.with_organization_id(state.upstreams.daytona_organization_id.clone())
.with_http_client(Some(
fabro_http::http_client().context("failed to build HTTP client")?,
));
daytona::check_daytona_api_key(&credentials, daytona::DAYTONA_CREDENTIAL_PROBE_TIMEOUT).await
}

225
lib/apps/fabro-server/src/run_files.rs
View file

@ -20,7 +20,7 @@ use std::future::Future;
use std::num::NonZeroU64;
use std::panic::AssertUnwindSafe;
use std::sync::Arc;
use std::time::Instant;
use std::time::{Duration, Instant};
use axum::Json;
use axum::extract::{Path, Query, State};
@ -35,13 +35,17 @@ use fabro_api::types::{
RunFilesMetaToSha,
};
use fabro_sandbox::reconnect::reconnect_for_run;
use fabro_sandbox::{RunSandbox, shell_quote};
use fabro_sandbox::{RunSandbox, Termination};
use fabro_types::RunId;
use fabro_util::shell;
use fabro_workflow::sandbox_git::{
DiffError, DiffNumstat, RawDiffEntry, SubmoduleChange, SymlinkChange, list_changed_files_raw,
list_diff_numstat, stream_blob_metadata, stream_blobs,
};
use futures_util::FutureExt;
use sandbox_driver::{
Git as _, GitCommit, GitDiffOptions, GitFacet, GitLogOptions, GitRevisionRange,
};
use serde::Deserialize;
use tokio::sync::{Mutex, watch};
@ -59,6 +63,7 @@ pub(crate) const AGGREGATE_BYTES_CAP: u64 = 5 * 1024 * 1024;
pub(crate) const FILE_COUNT_CAP: usize = 200;
/// Sandbox git timeout. Matches Unit 3 helpers (10 s).
const SANDBOX_GIT_TIMEOUT_MS: u64 = 10_000;
const SANDBOX_GIT_TIMEOUT: Duration = Duration::from_millis(SANDBOX_GIT_TIMEOUT_MS);
/// Below this SHA count the phase-1 `cat-file --batch-check` pre-filter is
/// skipped — its ~100 ms round-trip dominates for small diffs, and phase-2
@ -332,8 +337,7 @@ async fn materialize_run_commits(
.ok_or_else(|| ApiError::new(StatusCode::CONFLICT, "Run has no base SHA."))?;
let sandbox = reconnect_run_sandbox(state, run_id, &projection).await?;
let (head_sha, _) = resolve_ref_sha_and_time(&sandbox, "HEAD").await?;
let output = git_log_commits(&sandbox, &base_sha, &head_sha, limit + 1).await?;
let mut commits = parse_git_log_commits(&output)?;
let mut commits = git_log_commits(&sandbox, &base_sha, &head_sha, limit + 1).await?;
let truncated = commits.len() > usize::try_from(limit).unwrap_or(usize::MAX);
commits.truncate(usize::try_from(limit).unwrap_or(usize::MAX));
let total_returned = u64::try_from(commits.len()).unwrap_or(u64::MAX);
@ -356,57 +360,26 @@ async fn git_log_commits(
base_sha: &str,
head_sha: &str,
limit: u64,
) -> std::result::Result<String, ApiError> {
let base_q = shell_quote(base_sha);
let head_q = shell_quote(head_sha);
let format_q =
shell_quote("%H%x1f%T%x1f%P%x1f%an%x1f%ae%x1f%aI%x1f%cn%x1f%ce%x1f%cI%x1f%B%x1e");
sandbox_git_stdout(
sandbox,
&format!(
"git -c maintenance.auto=0 -c gc.auto=0 -c core.hooksPath=/dev/null -c core.fsmonitor=false -c core.quotePath=false log --first-parent --reverse --max-count={limit} --format={format_q} {base_q}..{head_q}"
),
"git log",
)
.await
) -> std::result::Result<Vec<RunCommit>, ApiError> {
let git = sandbox_git(sandbox)?;
let options = GitLogOptions::new(GitRevisionRange::new(base_sha).to(head_sha))
.first_parent()
.reverse()
.max_count(limit)
.timeout(SANDBOX_GIT_TIMEOUT);
let commits = git
.log(sandbox.working_directory(), &options)
.await
.map_err(|error| sandbox_git_error("git log", &error))?;
commits.iter().map(run_commit).collect()
}
fn parse_git_log_commits(stdout: &str) -> std::result::Result<Vec<RunCommit>, ApiError> {
stdout
.split('\x1e')
.filter_map(|record| {
let record = record.trim_matches('\n');
(!record.is_empty()).then_some(record)
})
.map(parse_git_log_commit)
.collect()
}
fn parse_git_log_commit(record: &str) -> std::result::Result<RunCommit, ApiError> {
let mut fields = record.splitn(10, '\x1f');
let sha = fields.next().unwrap_or_default();
let tree_sha = fields.next().unwrap_or_default();
let parents = fields.next().unwrap_or_default();
let author_name = fields.next().unwrap_or_default();
let author_email = fields.next().unwrap_or_default();
let author_date = fields.next().unwrap_or_default();
let committer_name = fields.next().unwrap_or_default();
let committer_email = fields.next().unwrap_or_default();
let committer_date = fields.next().unwrap_or_default();
let message = fields
.next()
.unwrap_or_default()
.trim_end_matches('\n')
.to_string();
if sha.is_empty() {
return Err(ApiError::bad_request(
"Malformed git log output: missing commit SHA.",
));
}
fn run_commit(commit: &GitCommit) -> std::result::Result<RunCommit, ApiError> {
let message = commit.message.trim_end_matches('\n').to_string();
let (subject, body) = split_commit_message(&message);
let parents = parents
.split_whitespace()
let parents = commit
.parents
.iter()
.map(|parent| {
Ok(RunCommitParent {
sha: sha_newtype::<RunCommitParentSha>(parent)?,
@ -416,27 +389,27 @@ fn parse_git_log_commit(record: &str) -> std::result::Result<RunCommit, ApiError
.collect::<std::result::Result<Vec<_>, ApiError>>()?;
Ok(RunCommit {
sha: sha_newtype::<RunCommitSha>(sha)?,
short_sha: short_sha_newtype::<RunCommitShortSha>(sha)?,
sha: sha_newtype::<RunCommitSha>(&commit.sha)?,
short_sha: short_sha_newtype::<RunCommitShortSha>(&commit.sha)?,
parents,
author: RunCommitPerson {
name: author_name.to_string(),
email: author_email.to_string(),
date: parse_git_date(author_date),
name: commit.author.name.clone(),
email: commit.author.email.clone(),
date: parse_git_date(&commit.author.date),
},
committer: RunCommitPerson {
name: committer_name.to_string(),
email: committer_email.to_string(),
date: parse_git_date(committer_date),
name: commit.committer.name.clone(),
email: commit.committer.email.clone(),
date: parse_git_date(&commit.committer.date),
},
subject,
body,
message: message.clone(),
trailers: parse_commit_trailers(&message),
tree_sha: if tree_sha.is_empty() {
tree_sha: if commit.tree.is_empty() {
None
} else {
Some(sha_newtype::<RunCommitTreeSha>(tree_sha)?)
Some(sha_newtype::<RunCommitTreeSha>(&commit.tree)?)
},
})
}
@ -732,15 +705,15 @@ async fn materialize_working_tree_sandbox_path(
start: Instant,
) -> ListRunFilesResult {
let (to_sha, to_sha_committed_at) = resolve_head_sha_and_time(sandbox).await?;
let base_q = shell_quote(base_ref);
let patch = sandbox_git_stdout(
sandbox,
&format!(
"git -c maintenance.auto=0 -c gc.auto=0 -c core.hooksPath=/dev/null -c core.fsmonitor=false -c core.quotePath=false diff --patch --find-renames=50% {base_q}"
),
"git diff --patch",
)
.await?;
let git = sandbox_git(sandbox)?;
// No head: the driver diffs `base_ref` against the working tree.
let options = GitDiffOptions::new(GitRevisionRange::new(base_ref))
.find_renames(50)
.timeout(SANDBOX_GIT_TIMEOUT);
let patch = git
.diff_patch(sandbox.working_directory(), &options)
.await
.map_err(|error| sandbox_git_error("git diff --patch", &error))?;
let entries: Vec<String> = split_patch_sections(&patch)
.into_iter()
@ -762,22 +735,25 @@ async fn materialize_working_tree_sandbox_path(
))
}
async fn sandbox_git_stdout(
sandbox: &RunSandbox,
command: &str,
op: &str,
) -> std::result::Result<String, ApiError> {
let res = sandbox
.exec_command(command, SANDBOX_GIT_TIMEOUT_MS, None, None, None)
.await
.map_err(|err| ApiError::new(StatusCode::SERVICE_UNAVAILABLE, err.display_with_causes()))?;
if res.is_timed_out() {
return Err(transient_503(op, "command timed out"));
/// The sandbox's git facet; a provider without git cannot serve files.
fn sandbox_git(sandbox: &RunSandbox) -> std::result::Result<GitFacet<'_>, ApiError> {
sandbox
.git()
.map_err(|err| ApiError::new(StatusCode::SERVICE_UNAVAILABLE, err.display_with_causes()))
}
/// A driver git failure as the endpoint's transient 503, so the client
/// retries; a command that timed out says so.
fn sandbox_git_error(op: &str, error: &sandbox_driver::Error) -> ApiError {
let timed_out = matches!(
error,
sandbox_driver::Error::Git(failure)
if failure.output().is_some_and(|output| output.termination() == Termination::TimedOut)
);
if timed_out {
return transient_503(op, "command timed out");
}
if !res.is_success() {
return Err(transient_503(op, res.stderr.trim()));
}
Ok(res.stdout)
transient_503(op, &fabro_sandbox::display_for_log(error))
}
/// Build the degraded response from the stored terminal diff patch.
@ -1204,7 +1180,7 @@ async fn reconnect_run_sandbox(
.provider_access()
.await
.map_err(|err| ApiError::new(StatusCode::INTERNAL_SERVER_ERROR, err.to_string()))?;
let sandbox = reconnect_for_run(&record, &access, Some(*run_id))
let sandbox = reconnect_for_run(&record, &access, Some(*run_id), None)
.await
.map_err(|err| ApiError::new(StatusCode::CONFLICT, err.to_string()))?;
sandbox
@ -1228,7 +1204,7 @@ async fn resolve_ref_sha_and_time(
sandbox: &RunSandbox,
git_ref: &str,
) -> std::result::Result<(String, Option<chrono::DateTime<chrono::Utc>>), ApiError> {
let ref_q = shell_quote(git_ref);
let ref_q = shell::shell_quote(git_ref);
let res = sandbox
.exec_command(
&format!("git -c core.hooksPath=/dev/null show -s --format=%H\\ %cI {ref_q}"),
@ -1239,13 +1215,13 @@ async fn resolve_ref_sha_and_time(
)
.await
.map_err(|err| ApiError::new(StatusCode::SERVICE_UNAVAILABLE, err.display_with_causes()))?;
if !res.is_success() {
if !res.success() {
return Err(ApiError::new(
StatusCode::SERVICE_UNAVAILABLE,
"Failed to resolve sandbox git ref.",
));
}
parse_head_show_output(&res.stdout).ok_or_else(|| {
parse_head_show_output(&res.stdout_lossy()).ok_or_else(|| {
ApiError::new(
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox HEAD resolved to an empty value.",
@ -1703,7 +1679,9 @@ fn count_flags(data: &[FileDiff]) -> (u64, u64, u64, u64) {
mod tests {
use std::sync::atomic::{AtomicUsize, Ordering};
use fabro_types::{CommandTermination, RunId, test_support};
use fabro_sandbox::Termination;
use fabro_sandbox::test_support::exec_result;
use fabro_types::{RunId, test_support};
use tokio::time::{Duration, sleep};
use super::*;
@ -1749,7 +1727,7 @@ mod tests {
sandbox.respond_with(|command| {
let stdout = if command.contains(" show -s --format=") {
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 2026-05-09T17:12:40Z\n".to_string()
} else if command.contains(" diff --patch --find-renames=50% ") {
} else if command.contains("'diff'") && command.contains("'--find-renames=50%'") {
"\
diff --git a/src/live.rs b/src/live.rs
--- a/src/live.rs
@ -1762,13 +1740,7 @@ diff --git a/src/live.rs b/src/live.rs
} else {
return None;
};
Some(fabro_sandbox::ExecResult {
stdout,
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 0,
})
Some(exec_result(&stdout, "", Some(0), Termination::Exited, 0))
});
let body = materialize_working_tree_sandbox_path(
@ -1784,15 +1756,21 @@ diff --git a/src/live.rs b/src/live.rs
assert_eq!(body.meta.source, RunFilesMetaSource::Sandbox);
assert_eq!(body.meta.scope, RunFilesMetaScope::Uncommitted);
assert_eq!(body.data.len(), 1);
let commands = sandbox.captured_commands();
let commands = sandbox.driver().scripted_exec().commands();
assert_eq!(commands.len(), 2);
assert!(commands[0].contains(" show -s --format="));
assert!(commands[1].contains(" diff --patch --find-renames=50% HEAD"));
assert!(
commands[1].contains("'diff'")
&& commands[1].contains("'--find-renames=50%'")
&& commands[1].contains("'HEAD'"),
"{}",
commands[1]
);
assert!(!commands.iter().any(|command| command.contains("ls-files")));
}
#[test]
fn parse_git_log_commits_keeps_external_and_fabro_metadata() {
#[tokio::test]
async fn git_log_commits_keeps_external_and_fabro_metadata() {
let stdout = concat!(
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\x1f",
"cccccccccccccccccccccccccccccccccccccccc\x1f",
@ -1807,8 +1785,26 @@ diff --git a/src/live.rs b/src/live.rs
"Alice\x1falice@example.com\x1f2026-05-09T18:00:00Z\x1f",
"external tool update\n\nLonger body.\n\x1e",
);
let sandbox = fabro_sandbox::test_support::MockSandbox::default();
sandbox
.driver()
.scripted_exec()
.push_result(fabro_sandbox::test_support::exec_result(
stdout,
"",
Some(0),
Termination::Exited,
1,
));
let commits = parse_git_log_commits(stdout).expect("git log should parse");
let commits = git_log_commits(
&sandbox.sandbox(),
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"dddddddddddddddddddddddddddddddddddddddd",
50,
)
.await
.expect("git log should parse");
assert_eq!(commits.len(), 2);
assert_eq!(commits[0].subject, "fabro(run_1): implement (succeeded)");
@ -1821,6 +1817,11 @@ diff --git a/src/live.rs b/src/live.rs
assert_eq!(commits[1].subject, "external tool update");
assert_eq!(commits[1].body.as_deref(), Some("Longer body."));
assert!(commits[1].trailers.is_empty());
let command = &sandbox.driver().scripted_exec().commands()[0];
assert!(
command.contains("'--first-parent'") && command.contains("'--max-count=50'"),
"{command}"
);
}
#[tokio::test]
@ -2873,23 +2874,11 @@ rename to .env.production
}
fn ok_exec(stdout: &str) -> ExecResult {
ExecResult {
stdout: stdout.to_string(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 0,
}
exec_result(stdout, "", Some(0), Termination::Exited, 0)
}
fn fail_exec(stderr: &str) -> ExecResult {
ExecResult {
stdout: String::new(),
stderr: stderr.to_string(),
exit_code: Some(1),
termination: CommandTermination::Exited,
duration_ms: 0,
}
exec_result("", stderr, Some(1), Termination::Exited, 0)
}
#[tokio::test]

88
lib/apps/fabro-server/src/run_manifest.rs
View file

@ -17,10 +17,8 @@ use fabro_graphviz::render::apply_direction;
use fabro_llm::FabroClient;
use fabro_llm::lithos_catalog::Catalog;
use fabro_llm::probe::{self, ModelTestStatus};
use fabro_sandbox::redact::redact_auth_url;
use fabro_sandbox::{
ProviderAccess, ProviderSandboxSpec, RunSandbox, SandboxSpec,
local_working_directory_from_environment, options_from_environment, unresolved_env,
CloneRequest, ProviderAccess, RunSandbox, SandboxSpec, sandbox_spec_for_environment,
};
use fabro_static::EnvVars;
use fabro_types::settings::ModelRef;
@ -874,7 +872,10 @@ async fn check_git_remote_ref(
run_ls_remote(command)
.await
.map_err(|message| redact_auth_url(&message, auth_url.as_ref()))
.map_err(|message| match &auth_url {
Some(auth_url) => auth_url.redact_in(&message),
None => message,
})
}
/// Run a prepared `git ls-remote` invocation with a 10s timeout, reducing a
@ -918,31 +919,36 @@ fn preflight_sandbox_spec(
let clone_branch = prepared.git.as_ref().map(|git| git.branch.clone());
if sandbox_provider.bundled() == Some(BundledProvider::Local) {
let working_directory = local_working_directory_from_environment(
&resolved_run.environment,
Some(&prepared.source_directory),
)?;
return Ok(SandboxSpec::Local { working_directory });
let working_directory = resolved_run
.environment
.local_working_directory(Some(&prepared.source_directory))
.map_err(|err| {
fabro_sandbox::Error::context(
"Failed to resolve local environment working directory",
err,
)
})?;
return Ok(SandboxSpec::local(working_directory, access.clone()));
}
// No vault is available on this path, so a `{{ secrets.* }}` value keeps
// its source form.
let mut options = options_from_environment(
// its source form. Preflight never clones.
let spec = sandbox_spec_for_environment(
&resolved_run.environment,
&resolved_run.clone,
unresolved_env(&resolved_run.environment),
resolved_run.environment.unresolved_env(),
)?;
options.skip_clone = true;
Ok(SandboxSpec::Provider(Box::new(ProviderSandboxSpec {
let clone = CloneRequest {
origin_url: clone_origin_url,
branch: clone_branch,
..CloneRequest::none()
};
Ok(SandboxSpec {
kind: sandbox_provider.clone(),
access: access.clone(),
options,
spec,
clone,
github_app,
run_id: None,
clone_origin_url,
clone_branch,
clone_tag: None,
clone_commit_sha: None,
})))
})
}
async fn run_sandbox_check(
@ -993,7 +999,7 @@ async fn run_sandbox_check(
warn: true,
});
}
if let Err(err) = sandbox.cleanup().await {
if let Err(err) = sandbox.delete().await {
checks.push(CheckResult {
name: "Sandbox".into(),
status: CheckStatus::Error,
@ -1013,7 +1019,7 @@ async fn run_sandbox_check(
true
}
Err(err) => {
let cleanup_error = sandbox.cleanup().await.err();
let cleanup_error = sandbox.delete().await.err();
checks.push(CheckResult {
name: "Sandbox".into(),
status: CheckStatus::Error,
@ -1494,23 +1500,21 @@ async fn probe_github_repository(
/// Retry auth-shaped failures with the SAME token: replication of a given
/// token only makes progress, while re-minting would restart the replication
/// clock. The sandbox git retry executor owns attempt limits,
/// classification, and pacing.
/// clock. The driver's git retry owns the decision and the pacing; fabro's
/// probe policy owns the attempt count.
async fn probe_with_replication_retry<F, Fut>(
snapshot: TokenSnapshot,
mut run: F,
run: F,
) -> std::result::Result<(), String>
where
F: FnMut() -> Fut,
Fut: Future<Output = std::result::Result<(), String>>,
{
let credential_context = fabro_sandbox::CredentialContext::from_snapshot(Some(&snapshot));
fabro_sandbox::retry_git_operation(
SandboxProviderKind::LOCAL,
fabro_sandbox::retry_git_messages(
&fabro_sandbox::repository_probe_policy(),
Some(&snapshot),
"repository probe",
&fabro_sandbox::RetryPlan::repository_probe(),
|_attempt| run(),
|message| fabro_sandbox::classify_failure(message, credential_context),
run,
)
.await
}
@ -2215,18 +2219,14 @@ provider = "local"
&ProviderAccess::default(),
);
match spec {
Ok(SandboxSpec::Provider(spec)) => {
assert_eq!(spec.kind, SandboxProviderKind::DOCKER);
assert!(spec.options.skip_clone);
assert_eq!(
spec.clone_origin_url.as_deref(),
Some("https://github.com/acme/widgets")
);
assert_eq!(spec.clone_branch.as_deref(), Some("main"));
}
_ => panic!("expected Docker preflight sandbox spec"),
}
let spec = spec.expect("Docker preflight sandbox spec");
assert_eq!(spec.kind, SandboxProviderKind::DOCKER);
assert!(spec.clone.skip);
assert_eq!(
spec.clone.origin_url.as_deref(),
Some("https://github.com/acme/widgets")
);
assert_eq!(spec.clone.branch.as_deref(), Some("main"));
}
#[test]

2
lib/apps/fabro-server/src/serve.rs
View file

@ -799,7 +799,7 @@ where
github_api_base_url: None,
active_config_path,
http_client: None,
sandbox_provider_registry: None,
sandbox_inventory: None,
shutdown: shutdown.clone(),
#[cfg(test)]
worker_control_bus: None,

62
lib/apps/fabro-server/src/server.rs
View file

@ -65,10 +65,7 @@ use fabro_redact::redact_jsonl_line;
use fabro_sandbox::details::sandbox_details;
use fabro_sandbox::driver::{DaytonaCredentials, ProviderAccess, ProviderConnectOptions};
use fabro_sandbox::reconnect::reconnect_for_run;
use fabro_sandbox::{
DriverInventoryProvider, LocalSandboxProvider, SandboxProvider, SandboxProviderRegistry,
daytona,
};
use fabro_sandbox::{SandboxInventory, daytona};
use fabro_slack::client::{PostedMessage as SlackPostedMessage, SlackClient};
use fabro_slack::config::{
SlackCredentialResolution,
@ -1137,7 +1134,7 @@ pub struct AppState {
pub(crate) github_api_base_url: String,
active_config_path: PathBuf,
http_client: Option<fabro_http::HttpClient>,
sandbox_provider_registry: SandboxProviderRegistry,
sandbox_inventory: SandboxInventory,
shutdown: CancellationToken,
shutting_down: AtomicBool,
registry_factory_override: Option<Box<RegistryFactoryOverride>>,
@ -1282,7 +1279,7 @@ pub(crate) struct AppStateConfig {
pub(crate) github_api_base_url: Option<String>,
pub(crate) active_config_path: PathBuf,
pub(crate) http_client: Option<fabro_http::HttpClient>,
pub(crate) sandbox_provider_registry: Option<SandboxProviderRegistry>,
pub(crate) sandbox_inventory: Option<SandboxInventory>,
pub(crate) shutdown: CancellationToken,
#[cfg(test)]
pub(crate) worker_control_bus: Option<Arc<dyn WorkerControlBus>>,
@ -1475,15 +1472,8 @@ impl AppState {
/// the server's HTTP client. The process environment is consulted only
/// through the configured lookup.
pub(crate) fn daytona_credentials(&self, api_key: String) -> DaytonaCredentials {
DaytonaCredentials {
api_key,
api_url: self
.config_env_lookup(EnvVars::DAYTONA_API_URL)
.or_else(|| self.config_env_lookup(EnvVars::DAYTONA_SERVER_URL)),
organization_id: self.config_env_lookup(EnvVars::DAYTONA_ORGANIZATION_ID),
target: None,
http_client: self.http_client().ok(),
}
DaytonaCredentials::from_api_key(api_key, |name| self.config_env_lookup(name))
.with_http_client(self.http_client().ok())
}
/// Everything a reconnect needs to reach a run's provider: the server's
@ -1540,8 +1530,8 @@ impl AppState {
&self.session_runtimes
}
pub(crate) fn sandbox_provider_registry(&self) -> &SandboxProviderRegistry {
&self.sandbox_provider_registry
pub(crate) fn sandbox_inventory(&self) -> &SandboxInventory {
&self.sandbox_inventory
}
pub(crate) fn server_secret(&self, name: &str) -> Option<String> {
@ -2338,51 +2328,45 @@ fn worker_token_keys_from_server_secrets(
.map_err(|err| jwt_auth::session_secret_key_error(&err))
}
fn build_sandbox_provider_registry(
fn build_sandbox_inventory(
server_settings: &ServerSettings,
daytona_api_key: Option<String>,
env_lookup: &EnvLookup,
http_client: Option<fabro_http::HttpClient>,
) -> SandboxProviderRegistry {
) -> SandboxInventory {
let provider_settings = &server_settings.server.sandbox.providers;
let mut providers: Vec<Arc<dyn SandboxProvider>> = Vec::new();
let mut inventory = SandboxInventory::empty();
if provider_settings.is_enabled(&SandboxProviderKind::LOCAL) {
providers.push(Arc::new(LocalSandboxProvider));
inventory = inventory.with_host_directories(SandboxProviderKind::LOCAL);
}
if let Some(docker) = provider_settings.get(&SandboxProviderKind::DOCKER) {
if docker.enabled {
providers.push(Arc::new(DriverInventoryProvider::lazy(
inventory = inventory.with_lazy(
SandboxProviderKind::DOCKER,
docker.clone(),
ProviderConnectOptions::default(),
)));
);
}
}
if let Some(daytona) = provider_settings.get(&SandboxProviderKind::DAYTONA) {
if let Some(api_key) = daytona_api_key.filter(|_| daytona.enabled) {
let credentials = DaytonaCredentials {
api_key,
api_url: env_lookup(EnvVars::DAYTONA_API_URL)
.or_else(|| env_lookup(EnvVars::DAYTONA_SERVER_URL)),
organization_id: env_lookup(EnvVars::DAYTONA_ORGANIZATION_ID),
target: None,
http_client,
};
providers.push(Arc::new(DriverInventoryProvider::lazy(
let credentials = DaytonaCredentials::from_api_key(api_key, |name| env_lookup(name))
.with_http_client(http_client);
inventory = inventory.with_lazy(
SandboxProviderKind::DAYTONA,
daytona.clone(),
ProviderConnectOptions {
host_registry_root: None,
daytona: Some(credentials),
},
)));
);
}
}
SandboxProviderRegistry::new(providers)
inventory
}
pub(crate) fn automation_dir_for_active_config(active_config_path: &std::path::Path) -> PathBuf {
@ -2435,7 +2419,7 @@ pub(crate) fn build_app_state(config: AppStateConfig) -> anyhow::Result<Arc<AppS
github_api_base_url,
active_config_path,
http_client,
sandbox_provider_registry,
sandbox_inventory,
shutdown,
#[cfg(test)]
worker_control_bus,
@ -2505,8 +2489,8 @@ pub(crate) fn build_app_state(config: AppStateConfig) -> anyhow::Result<Arc<AppS
fabro_llm::build_catalog(&resolved_settings.llm_overlay, &|name| env_lookup(name))
.context("building LLM model catalog")?,
);
let sandbox_provider_registry = sandbox_provider_registry.unwrap_or_else(|| {
build_sandbox_provider_registry(
let sandbox_inventory = sandbox_inventory.unwrap_or_else(|| {
build_sandbox_inventory(
current_server_settings.as_ref(),
daytona_api_key,
&env_lookup,
@ -2619,7 +2603,7 @@ pub(crate) fn build_app_state(config: AppStateConfig) -> anyhow::Result<Arc<AppS
github_api_base_url,
active_config_path,
http_client,
sandbox_provider_registry,
sandbox_inventory,
shutdown,
shutting_down: AtomicBool::new(false),
registry_factory_override,
@ -2795,7 +2779,7 @@ async fn delete_run_sandbox_resource(
.provider_access()
.await
.map_err(|err| ApiError::new(StatusCode::INTERNAL_SERVER_ERROR, err.to_string()))?;
let sandbox = match reconnect_for_run(&record, &access, Some(id)).await {
let sandbox = match reconnect_for_run(&record, &access, Some(id), None).await {
Ok(sandbox) => sandbox,
Err(err) if force || delete_started => {
tracing::warn!(

7
lib/apps/fabro-server/src/server/handler/automations.rs
View file

@ -302,10 +302,9 @@ pub(in crate::server) fn resolve_automation_environment(
));
}
if !state
.sandbox_provider_registry()
.providers()
.iter()
.any(|sandbox_provider| sandbox_provider.kind() == provider)
.sandbox_inventory()
.kinds()
.any(|kind| *kind == provider)
{
return Err(ApiError::with_code(
status,

455
lib/apps/fabro-server/src/server/handler/sandbox.rs
View file

@ -1,19 +1,16 @@
use std::collections::BTreeMap;
use std::net::{Ipv4Addr, Ipv6Addr};
use std::num::NonZeroU64;
use std::sync::Arc;
use std::time::Duration;
use axum::extract::ws::{Message as WsMessage, WebSocket, WebSocketUpgrade};
use fabro_sandbox::{
FileKind, ProviderAccess, RunSandbox, TerminalSize, open_terminal_for_run,
reconnect_driver_for_run,
};
use fabro_types::{
RunSandboxInstance, SandboxProviderKind, SandboxServiceDiscoverySource, SandboxServiceListMeta,
FileKind, ProviderAccess, PtySize, RunSandbox, open_terminal_for_run, reconnect_for_run,
};
use fabro_types::{RunSandboxInstance, SandboxProviderKind};
use futures_util::FutureExt;
use futures_util::future::BoxFuture;
use sandbox_driver::{ListeningPort, Services as _};
use super::super::{
ApiError, AppState, Bytes, HeaderMap, IntoResponse, Json, NamedTempFile, Path,
@ -29,20 +26,7 @@ const DEFAULT_VNC_NO_VNC_PORT: u16 = 6080;
const DEFAULT_VNC_TTL_SECS: i32 = 3600;
/// Header a Daytona unsigned preview needs; surfaced as the response token.
const PREVIEW_TOKEN_HEADER: &str = "x-daytona-preview-token";
const LIST_SANDBOX_SERVICES_COMMAND: &str = r#"if command -v ss >/dev/null 2>&1; then
ss -H -ltnp && exit 0
fi
printf 'FABRO_PROC_NET_TCP procfs\n'
for file in /proc/net/tcp /proc/net/tcp6; do
if [ -r "$file" ]; then
printf 'FABRO_PROC_NET_TCP %s\n' "$file"
while IFS= read -r line; do
printf '%s\n' "$line"
done < "$file"
fi
done"#;
const LIST_SANDBOX_SERVICES_FAILURE_LABEL: &str = "sandbox service discovery command";
const LIST_SANDBOX_SERVICES_TIMEOUT_MS: u64 = 5_000;
const LIST_SANDBOX_SERVICES_FAILURE_LABEL: &str = "sandbox service discovery";
// Daytona's signed preview points at the noVNC service root, which serves a
// directory listing. Force the iframe to the actual viewer page with
// autoconnect+scale so the user lands on the desktop, not a file index.
@ -133,7 +117,7 @@ struct SandboxFileParams {
#[derive(Debug, PartialEq, Eq)]
enum TerminalClientMessage {
Resize(TerminalSize),
Resize(PtySize),
Close,
}
@ -150,7 +134,7 @@ fn parse_terminal_control_message(text: &str) -> Result<TerminalClientMessage, &
}
match serde_json::from_str::<TerminalClientControl>(text) {
Ok(TerminalClientControl::Resize { cols, rows }) if cols > 0 && rows > 0 => {
Ok(TerminalClientMessage::Resize(TerminalSize { cols, rows }))
Ok(TerminalClientMessage::Resize(PtySize { cols, rows }))
}
Ok(TerminalClientControl::Resize { .. }) => {
Err("Terminal resize dimensions must be greater than zero.")
@ -235,19 +219,19 @@ async fn terminal_websocket(mut socket: WebSocket, state: Arc<AppState>, id: Run
return;
}
};
let session =
match open_terminal_for_run(&record, &access, Some(id), TerminalSize::default()).await {
Ok(session) => session,
Err(err) => {
let _ = socket
.send(terminal_server_text(
"error",
Some(&err.display_with_causes()),
))
.await;
return;
}
};
let session = match open_terminal_for_run(&record, &access, Some(id), PtySize::default()).await
{
Ok(session) => session,
Err(err) => {
let _ = socket
.send(terminal_server_text(
"error",
Some(&err.display_with_causes()),
))
.await;
return;
}
};
if socket
.send(terminal_server_text("ready", None))
@ -268,7 +252,7 @@ async fn terminal_websocket(mut socket: WebSocket, state: Arc<AppState>, id: Run
Ok(WsMessage::Binary(bytes)) => {
if let Err(err) = session.write_input(&bytes).await {
let _ = socket
.send(terminal_server_text("error", Some(&err.display_with_causes())))
.send(terminal_server_text("error", Some(&fabro_sandbox::display_for_log(&err))))
.await;
break;
}
@ -278,7 +262,7 @@ async fn terminal_websocket(mut socket: WebSocket, state: Arc<AppState>, id: Run
Ok(TerminalClientMessage::Resize(size)) => {
if let Err(err) = session.resize(size).await {
let _ = socket
.send(terminal_server_text("error", Some(&err.display_with_causes())))
.send(terminal_server_text("error", Some(&fabro_sandbox::display_for_log(&err))))
.await;
break;
}
@ -313,7 +297,7 @@ async fn terminal_websocket(mut socket: WebSocket, state: Arc<AppState>, id: Run
}
Err(err) => {
let _ = socket
.send(terminal_server_text("error", Some(&err.display_with_causes())))
.send(terminal_server_text("error", Some(&fabro_sandbox::display_for_log(&err))))
.await;
break;
}
@ -322,7 +306,7 @@ async fn terminal_websocket(mut socket: WebSocket, state: Arc<AppState>, id: Run
}
}
if let Err(err) = session.close().await {
tracing::warn!(error = %err.display_with_causes(), run_id = %id, "failed to close run terminal session");
tracing::warn!(error = %fabro_sandbox::display_for_log(&err), run_id = %id, "failed to close run terminal session");
}
}
@ -587,143 +571,43 @@ async fn list_sandbox_services(
Ok(sandbox) => sandbox,
Err(response) => return response,
};
let result = match sandbox
.exec_command(
LIST_SANDBOX_SERVICES_COMMAND,
LIST_SANDBOX_SERVICES_TIMEOUT_MS,
None,
None,
None,
)
.await
{
Ok(result) => result,
let services = match sandbox.services() {
Ok(services) => services,
Err(err) => {
return ApiError::new(StatusCode::CONFLICT, err.display_with_causes()).into_response();
return ApiError::new(StatusCode::NOT_IMPLEMENTED, err.display_with_causes())
.into_response();
}
};
let ports = match services.listening_ports().await {
Ok(ports) => ports,
Err(err) => {
return ApiError::new(
StatusCode::CONFLICT,
format!("{LIST_SANDBOX_SERVICES_FAILURE_LABEL} failed: {err}"),
)
.into_response();
}
};
if !result.is_success() {
return ApiError::new(
StatusCode::CONFLICT,
sandbox_service_command_failure_detail(&result),
)
.into_response();
}
let discovery = parse_sandbox_services(&result.stdout, &provider);
Json(SandboxServiceListResponse {
data: discovery.services,
meta: SandboxServiceListMeta {
source: discovery.source,
},
data: services_from_ports(ports, &provider),
})
.into_response()
}
fn sandbox_service_command_failure_detail(result: &fabro_sandbox::ExecResult) -> String {
let stderr = result.stderr.trim();
if !stderr.is_empty() {
return stderr.to_string();
}
let stdout = result.stdout.trim();
if !stdout.is_empty() {
return stdout.to_string();
}
format!("{LIST_SANDBOX_SERVICES_FAILURE_LABEL} failed")
}
struct SandboxServiceDiscovery {
services: Vec<SandboxService>,
source: SandboxServiceDiscoverySource,
}
fn parse_sandbox_services(output: &str, provider: &SandboxProviderKind) -> SandboxServiceDiscovery {
if output
.lines()
.any(|line| line.trim_start().starts_with("FABRO_PROC_NET_TCP "))
{
SandboxServiceDiscovery {
services: parse_proc_net_listening_services(output, provider),
source: SandboxServiceDiscoverySource::Procfs,
}
} else {
SandboxServiceDiscovery {
services: parse_ss_listening_services(output, provider),
source: SandboxServiceDiscoverySource::Ss,
}
}
}
fn parse_ss_listening_services(
output: &str,
/// The driver's listeners grouped by port, previewable ports first.
fn services_from_ports(
ports: Vec<ListeningPort>,
provider: &SandboxProviderKind,
) -> Vec<SandboxService> {
let mut services = BTreeMap::<u16, SandboxService>::new();
for line in output
.lines()
.map(str::trim)
.filter(|line| !line.is_empty())
{
let fields = line.split_whitespace().collect::<Vec<_>>();
let Some(address) = fields.get(3).copied() else {
continue;
};
let Some(port) = parse_ss_local_port(address) else {
continue;
};
let process = (fields.len() > 5).then(|| fields[5..].join(" "));
push_service(&mut services, provider, port, address.to_string(), process);
}
sorted_services(services)
}
fn parse_ss_local_port(address: &str) -> Option<u16> {
let port = address.rsplit_once(':')?.1.parse::<u16>().ok()?;
(port > 0).then_some(port)
}
#[derive(Clone, Copy)]
enum ProcNetFamily {
Ipv4,
Ipv6,
}
fn parse_proc_net_listening_services(
output: &str,
provider: &SandboxProviderKind,
) -> Vec<SandboxService> {
let mut services = BTreeMap::<u16, SandboxService>::new();
let mut family = None;
for line in output
.lines()
.map(str::trim)
.filter(|line| !line.is_empty())
{
if let Some(path) = line.strip_prefix("FABRO_PROC_NET_TCP ") {
family = if path.ends_with("/tcp6") {
Some(ProcNetFamily::Ipv6)
} else {
Some(ProcNetFamily::Ipv4)
};
continue;
}
if line.starts_with("sl") {
continue;
}
let Some(family) = family else {
continue;
};
let fields = line.split_whitespace().collect::<Vec<_>>();
let (Some(local_address), Some(state)) = (fields.get(1), fields.get(3)) else {
continue;
};
if *state != "0A" {
continue;
}
let Some((address, port)) = parse_proc_net_local_address(local_address, family) else {
continue;
};
push_service(&mut services, provider, port, address, None);
for listener in ports {
push_service(
&mut services,
provider,
listener.port,
listener.address,
listener.process,
);
}
sorted_services(services)
}
@ -734,40 +618,6 @@ fn sorted_services(services: BTreeMap<u16, SandboxService>) -> Vec<SandboxServic
services
}
fn parse_proc_net_local_address(value: &str, family: ProcNetFamily) -> Option<(String, u16)> {
let (address_hex, port_hex) = value.split_once(':')?;
let port = u16::from_str_radix(port_hex, 16).ok()?;
if port == 0 {
return None;
}
let address = match family {
ProcNetFamily::Ipv4 => format!("{}:{port}", parse_proc_net_ipv4(address_hex)?),
ProcNetFamily::Ipv6 => format!("[{}]:{port}", parse_proc_net_ipv6(address_hex)?),
};
Some((address, port))
}
fn parse_proc_net_ipv4(value: &str) -> Option<Ipv4Addr> {
if value.len() != 8 {
return None;
}
let raw = u32::from_str_radix(value, 16).ok()?;
Some(Ipv4Addr::from(raw.to_le_bytes()))
}
fn parse_proc_net_ipv6(value: &str) -> Option<Ipv6Addr> {
if value.len() != 32 {
return None;
}
let mut bytes = [0_u8; 16];
for (chunk_index, chunk) in value.as_bytes().chunks_exact(8).enumerate() {
let chunk = std::str::from_utf8(chunk).ok()?;
let raw = u32::from_str_radix(chunk, 16).ok()?;
bytes[chunk_index * 4..chunk_index * 4 + 4].copy_from_slice(&raw.to_le_bytes());
}
Some(Ipv6Addr::from(bytes))
}
fn push_service(
services: &mut BTreeMap<u16, SandboxService>,
provider: &SandboxProviderKind,
@ -885,7 +735,7 @@ async fn reconnect_run_sandbox_instance(
record: &RunSandboxInstance,
) -> Result<RunSandbox, Response> {
let access = load_provider_access(state).await?;
let sandbox = reconnect_driver_for_run(record, &access, Some(*run_id), None)
let sandbox = reconnect_for_run(record, &access, Some(*run_id), None)
.await
.map_err(|err| {
let detail = render_with_causes(&err.to_string(), &collect_causes(err.as_ref()));
@ -934,7 +784,7 @@ mod tests {
fn terminal_control_accepts_resize_and_close() {
assert_eq!(
parse_terminal_control_message(r#"{"type":"resize","cols":120,"rows":32}"#),
Ok(TerminalClientMessage::Resize(TerminalSize {
Ok(TerminalClientMessage::Resize(PtySize {
cols: 120,
rows: 32,
}))
@ -1014,104 +864,28 @@ mod tests {
}
#[test]
fn ss_parser_extracts_addresses_processes_and_preview_support() {
let services = parse_ss_listening_services(
r#"
LISTEN 0 4096 127.0.0.1:3000 0.0.0.0:* users:(("node",pid=42,fd=23))
LISTEN 0 4096 0.0.0.0:5173 0.0.0.0:* users:(("vite",pid=84,fd=19))
LISTEN 0 4096 [::]:8080 [::]:* users:(("server",pid=126,fd=9))
LISTEN 0 4096 [::1]:2500 [::]:* users:(("debug",pid=168,fd=7))
"#,
&SandboxProviderKind::DAYTONA,
);
assert_eq!(services.len(), 4);
assert_eq!(services[0].port, 3000);
assert_eq!(services[0].addresses, vec!["127.0.0.1:3000"]);
assert_eq!(services[0].processes, vec![
r#"users:(("node",pid=42,fd=23))"#
]);
assert!(services[0].preview_supported);
assert_eq!(services[1].port, 5173);
assert_eq!(services[1].addresses, vec!["0.0.0.0:5173"]);
assert!(services[1].preview_supported);
assert_eq!(services[2].port, 8080);
assert_eq!(services[2].addresses, vec!["[::]:8080"]);
assert!(services[2].preview_supported);
assert_eq!(services[3].port, 2500);
assert_eq!(services[3].addresses, vec!["[::1]:2500"]);
assert_eq!(services[3].processes, vec![
r#"users:(("debug",pid=168,fd=7))"#
]);
assert!(!services[3].preview_supported);
}
#[test]
fn ss_parser_ignores_malformed_and_non_numeric_ports() {
let services = parse_ss_listening_services(
r#"
LISTEN 0 4096 127.0.0.1:not-a-port 0.0.0.0:* users:(("node",pid=42,fd=23))
LISTEN 0 4096 missing-peer
not enough fields
LISTEN 0 4096 127.0.0.1:0 0.0.0.0:* users:(("zero",pid=1,fd=2))
LISTEN 0 4096 127.0.0.1:65536 0.0.0.0:* users:(("large",pid=1,fd=2))
"#,
&SandboxProviderKind::DAYTONA,
);
assert!(services.is_empty());
}
#[test]
fn ss_parser_groups_duplicate_ports_and_deduplicates_values() {
let services = parse_ss_listening_services(
r#"
LISTEN 0 4096 127.0.0.1:3000 0.0.0.0:* users:(("node",pid=42,fd=23))
LISTEN 0 4096 0.0.0.0:3000 0.0.0.0:* users:(("node",pid=42,fd=23))
LISTEN 0 4096 127.0.0.1:3000 0.0.0.0:* users:(("node",pid=42,fd=23))
LISTEN 0 4096 [::]:3000 [::]:* users:(("vite",pid=84,fd=19))
"#,
&SandboxProviderKind::DAYTONA,
);
assert_eq!(services, vec![SandboxService {
port: 3000,
addresses: vec![
"127.0.0.1:3000".to_string(),
"0.0.0.0:3000".to_string(),
"[::]:3000".to_string(),
fn listening_ports_group_by_port_and_sort_previewable_first() {
let mut node = ListeningPort::new(3000, "127.0.0.1:3000");
node.process = Some("node".to_string());
let mut node_v6 = ListeningPort::new(3000, "[::]:3000");
node_v6.process = Some("node".to_string());
let mut debug = ListeningPort::new(2500, "[::1]:2500");
debug.process = Some("pid=168".to_string());
let services = services_from_ports(
vec![
debug,
node,
node_v6,
ListeningPort::new(5173, "0.0.0.0:5173"),
],
processes: vec![
r#"users:(("node",pid=42,fd=23))"#.to_string(),
r#"users:(("vite",pid=84,fd=19))"#.to_string(),
],
preview_supported: true,
}]);
}
#[test]
fn proc_net_parser_extracts_listening_tcp_services_without_processes() {
let discovery = parse_sandbox_services(
r"
FABRO_PROC_NET_TCP /proc/net/tcp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 0100007F:0BB8 00000000:0000 0A 00000000:00000000 00:00000000 00000000 501 0 11111
1: 00000000:1435 00000000:0000 0A 00000000:00000000 00:00000000 00000000 501 0 22222
2: 0100007F:2328 00000000:0000 01 00000000:00000000 00:00000000 00000000 501 0 33333
FABRO_PROC_NET_TCP /proc/net/tcp6
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000000000000000000000000000:1F90 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 501 0 44444
1: 00000000000000000000000001000000:09C4 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 501 0 55555
",
&SandboxProviderKind::DAYTONA,
);
assert_eq!(discovery.source, SandboxServiceDiscoverySource::Procfs);
assert_eq!(discovery.services, vec![
assert_eq!(services, vec![
SandboxService {
port: 3000,
addresses: vec!["127.0.0.1:3000".to_string()],
processes: vec![],
addresses: vec!["127.0.0.1:3000".to_string(), "[::]:3000".to_string()],
processes: vec!["node".to_string()],
preview_supported: true,
},
SandboxService {
@ -1120,16 +894,10 @@ FABRO_PROC_NET_TCP /proc/net/tcp6
processes: vec![],
preview_supported: true,
},
SandboxService {
port: 8080,
addresses: vec!["[::]:8080".to_string()],
processes: vec![],
preview_supported: true,
},
SandboxService {
port: 2500,
addresses: vec!["[::1]:2500".to_string()],
processes: vec![],
processes: vec!["pid=168".to_string()],
preview_supported: false,
},
]);
@ -1144,33 +912,6 @@ FABRO_PROC_NET_TCP /proc/net/tcp6
assert!(!preview_supported(&SandboxProviderKind::DOCKER, 3000));
}
#[test]
fn sandbox_service_command_failure_prefers_stderr_then_stdout() {
let mut result = fabro_sandbox::ExecResult {
stdout: "stdout detail".to_string(),
stderr: "stderr detail".to_string(),
exit_code: Some(127),
termination: fabro_types::CommandTermination::Exited,
duration_ms: 10,
};
assert_eq!(
sandbox_service_command_failure_detail(&result),
"stderr detail"
);
result.stderr.clear();
assert_eq!(
sandbox_service_command_failure_detail(&result),
"stdout detail"
);
result.stdout.clear();
assert_eq!(
sandbox_service_command_failure_detail(&result),
"sandbox service discovery command failed"
);
}
struct FakeVncSandbox {
error: Option<&'static str>,
viewer_url: &'static str,
@ -1270,6 +1011,7 @@ FABRO_PROC_NET_TCP /proc/net/tcp6
mod retrieve_sandbox_tests {
use axum::body::{Body, to_bytes};
use axum::http::{Request, StatusCode};
use fabro_sandbox::test_support::local_sandbox_id;
use fabro_types::{Graph, RunId, WorkflowSettings, test_support};
use serde_json::{Value, json};
use tower::ServiceExt;
@ -1322,6 +1064,26 @@ mod retrieve_sandbox_tests {
run_store: &fabro_store::RunDatabase,
run_id: &RunId,
provider: &str,
) {
append_sandbox_initialized_in(
run_store,
run_id,
provider,
&format!("{provider}:sandbox-id"),
"/workspace",
)
.await;
}
/// A local sandbox reconnects by the id the Host provider derives from
/// its working directory, so a test that reaches one records an
/// existing directory under the id fabro would have written for it.
async fn append_sandbox_initialized_in(
run_store: &fabro_store::RunDatabase,
run_id: &RunId,
provider: &str,
id: &str,
working_directory: &str,
) {
let payload = fabro_store::EventPayload::new(
json!({
@ -1331,8 +1093,8 @@ mod retrieve_sandbox_tests {
"event": "sandbox.initialized",
"properties": {
"provider": provider,
"id": format!("{provider}:sandbox-id"),
"working_directory": "/workspace",
"id": id,
"working_directory": working_directory,
},
}),
run_id,
@ -1466,7 +1228,10 @@ mod retrieve_sandbox_tests {
.await
.expect("test run should be creatable");
append_run_created(&run_store, &run_id).await;
append_sandbox_initialized(&run_store, &run_id, "local").await;
let workspace = tempfile::tempdir().expect("scratch directory");
let working_directory = workspace.path().to_str().expect("utf-8").to_owned();
let id = local_sandbox_id(workspace.path()).await;
append_sandbox_initialized_in(&run_store, &run_id, "local", &id, &working_directory).await;
let response = app
.oneshot(req_get(&format!("/api/v1/runs/{run_id}/sandbox")))
@ -1475,18 +1240,22 @@ mod retrieve_sandbox_tests {
assert_eq!(response.status(), StatusCode::OK);
let body = body_json(response).await;
assert_eq!(body["sandbox"]["provider"], "local");
assert_eq!(body["sandbox"]["runtime"]["id"], "local:sandbox-id");
assert_eq!(body["sandbox"]["runtime"]["id"], id);
assert_eq!(
body["sandbox"]["runtime"]["working_directory"],
"/workspace"
working_directory
);
assert_eq!(body["state"], "running");
assert!(body.get("name").is_none());
assert_eq!(body["status"]["state"], "running");
assert_eq!(body["status"]["workspace_ownership"], "designated");
assert!(
body["status"]["id"]
.as_str()
.is_some_and(|id| id.starts_with("host-dir-")),
"{}",
body["status"]["id"]
);
assert!(body.get("state").is_none(), "the status is not flattened");
assert!(body.get("identifier").is_none());
assert!(body["resources"].is_object());
assert_eq!(body["network"]["egress"]["mode"], "unknown");
assert_eq!(body["network"]["ingress"]["mode"], "unknown");
assert!(body["timestamps"].is_object());
}
#[tokio::test]
@ -1500,7 +1269,15 @@ mod retrieve_sandbox_tests {
.await
.expect("test run should be creatable");
append_run_created(&run_store, &run_id).await;
append_sandbox_initialized(&run_store, &run_id, "local").await;
let workspace = tempfile::tempdir().expect("scratch directory");
append_sandbox_initialized_in(
&run_store,
&run_id,
"local",
&local_sandbox_id(workspace.path()).await,
workspace.path().to_str().expect("utf-8"),
)
.await;
let response = app
.oneshot(req_post(&format!("/api/v1/runs/{run_id}/sandbox/vnc")))

133
lib/apps/fabro-server/src/server/handler/sandboxes.rs
View file

@ -21,7 +21,7 @@ async fn list_sandboxes(
State(state): State<Arc<AppState>>,
_auth: RequiredRunManagementActor,
) -> Json<SandboxListResponse> {
Json(state.sandbox_provider_registry().list_managed().await)
Json(state.sandbox_inventory().list_managed().await)
}
async fn retrieve_sandbox(
@ -30,7 +30,7 @@ async fn retrieve_sandbox(
_auth: RequiredRunManagementActor,
) -> Result<Json<SandboxInfo>, ApiError> {
state
.sandbox_provider_registry()
.sandbox_inventory()
.get_managed_by_native_id(&id)
.await
.map(Json)
@ -79,23 +79,49 @@ fn provider_list(providers: &[SandboxProviderKind]) -> String {
mod tests {
use axum::body::{Body, to_bytes};
use axum::http::{Request, StatusCode};
use fabro_sandbox::SandboxProviderRegistry;
use fabro_sandbox::test_support::{
FakeGet, FakeList, FakeSandboxProvider, fake_registry, fake_sandbox_info,
};
use fabro_sandbox::SandboxInventory;
use fabro_sandbox::driver::{ConnectedProvider, ProviderConnectOptions};
use fabro_sandbox::test_support::{managed_scripted_sandbox, scripted_inventory_provider};
use fabro_types::SandboxProviderKind;
use fabro_types::settings::server::{SandboxPluginSettings, ServerSandboxProviderSettings};
use serde_json::{Value, json};
use tower::ServiceExt;
use crate::test_support::{TestAppStateBuilder, build_test_router};
fn app_with_registry(registry: SandboxProviderRegistry) -> axum::Router {
fn app_with_inventory(inventory: SandboxInventory) -> axum::Router {
let state = TestAppStateBuilder::new()
.sandbox_provider_registry(registry)
.sandbox_inventory(inventory)
.build();
build_test_router(state)
}
/// A connected provider of `kind` holding fabro-managed sandboxes `ids`.
fn provider(kind: SandboxProviderKind, ids: &[&str]) -> ConnectedProvider {
scripted_inventory_provider(
kind,
ids.iter().map(|id| managed_scripted_sandbox(id)).collect(),
)
}
/// A plugin kind whose executable does not exist, so every lookup fails
/// to connect.
fn with_unreachable_plugin(inventory: SandboxInventory, name: &str) -> SandboxInventory {
let settings = ServerSandboxProviderSettings {
enabled: true,
plugin: Some(SandboxPluginSettings {
path: Some(format!("/nonexistent/fabro-sandbox-{name}")),
dev: true,
..SandboxPluginSettings::default()
}),
};
inventory.with_lazy(
SandboxProviderKind::try_new(name).expect("valid kind"),
settings,
ProviderConnectOptions::default(),
)
}
fn req_get(uri: &str) -> Request<Body> {
Request::builder()
.method("GET")
@ -113,37 +139,28 @@ mod tests {
#[tokio::test]
async fn list_returns_provider_backed_data_without_run_projection_state() {
let docker = fake_sandbox_info(SandboxProviderKind::DOCKER, "docker-native-id");
let app = app_with_registry(fake_registry(vec![FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(vec![docker]),
FakeGet::Missing,
)]));
let app = app_with_inventory(
SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &["docker-native-id"])),
);
let response = app.oneshot(req_get("/api/v1/sandboxes")).await.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let body = body_json(response).await;
assert_eq!(body["data"][0]["id"], "docker-native-id");
assert_eq!(body["data"][0]["status"]["id"], "docker-native-id");
assert_eq!(body["data"][0]["provider"], "docker");
assert_eq!(body["data"][0]["status"]["state"], "running");
assert_eq!(body["meta"]["provider_errors"], json!([]));
}
#[tokio::test]
async fn retrieve_searches_all_configured_providers() {
let daytona = fake_sandbox_info(SandboxProviderKind::DAYTONA, "native-id");
let app = app_with_registry(fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(daytona)),
),
]));
let app = app_with_inventory(
SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &[]))
.with_connected(provider(SandboxProviderKind::DAYTONA, &["native-id"])),
);
let response = app
.oneshot(req_get("/api/v1/sandboxes/native-id"))
@ -152,24 +169,17 @@ mod tests {
assert_eq!(response.status(), StatusCode::OK);
let body = body_json(response).await;
assert_eq!(body["id"], "native-id");
assert_eq!(body["status"]["id"], "native-id");
assert_eq!(body["provider"], "daytona");
}
#[tokio::test]
async fn no_matching_sandbox_returns_404() {
let app = app_with_registry(fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
]));
let app = app_with_inventory(
SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &[]))
.with_connected(provider(SandboxProviderKind::DAYTONA, &[])),
);
let response = app
.oneshot(req_get("/api/v1/sandboxes/missing"))
@ -181,24 +191,11 @@ mod tests {
#[tokio::test]
async fn duplicate_native_ids_return_409() {
let app = app_with_registry(fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(fake_sandbox_info(
SandboxProviderKind::DOCKER,
"same-id",
))),
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(fake_sandbox_info(
SandboxProviderKind::DAYTONA,
"same-id",
))),
),
]));
let app = app_with_inventory(
SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &["same-id"]))
.with_connected(provider(SandboxProviderKind::DAYTONA, &["same-id"])),
);
let response = app
.oneshot(req_get("/api/v1/sandboxes/same-id"))
@ -217,18 +214,10 @@ mod tests {
#[tokio::test]
async fn provider_lookup_uncertainty_returns_502() {
let app = app_with_registry(fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Err("daytona unavailable"),
),
]));
let app = app_with_inventory(with_unreachable_plugin(
SandboxInventory::empty().with_connected(provider(SandboxProviderKind::DOCKER, &[])),
"e2b",
));
let response = app
.oneshot(req_get("/api/v1/sandboxes/maybe-missing"))
@ -241,7 +230,7 @@ mod tests {
body["errors"][0]["detail"]
.as_str()
.unwrap_or_default()
.contains("daytona unavailable")
.contains("e2b: Failed to connect to the e2b provider")
);
}
}

2
lib/apps/fabro-server/src/server/handler/sessions.rs
View file

@ -723,7 +723,7 @@ async fn build_agent(
.provider_access()
.await
.map_err(|err| AskFabroBuildError::Agent(anyhow::Error::new(err)))?;
let sandbox = reconnect_for_run(sandbox_instance, &access, Some(run_id))
let sandbox = reconnect_for_run(sandbox_instance, &access, Some(run_id), None)
.await
.map_err(AskFabroBuildError::SandboxUnavailable)?;
sandbox

8
lib/apps/fabro-server/src/server/tests.rs
View file

@ -2107,7 +2107,7 @@ fn slack_app_state_with_settings_and_secret_sources(
github_api_base_url: None,
active_config_path: tempfile::tempdir().unwrap().path().join("settings.toml"),
http_client: Some(fabro_http::test_http_client().expect("test HTTP client should build")),
sandbox_provider_registry: None,
sandbox_inventory: None,
shutdown: tokio_util::sync::CancellationToken::new(),
worker_control_bus: None,
worker_runtime: None,
@ -2268,7 +2268,7 @@ fn slack_service_respects_disabled_server_config_even_with_vault_tokens() {
github_api_base_url: None,
active_config_path: tempfile::tempdir().unwrap().path().join("settings.toml"),
http_client: Some(fabro_http::test_http_client().expect("test HTTP client should build")),
sandbox_provider_registry: None,
sandbox_inventory: None,
shutdown: tokio_util::sync::CancellationToken::new(),
worker_control_bus: None,
worker_runtime: None,
@ -2622,7 +2622,7 @@ methods = ["dev-token"]
github_api_base_url: None,
active_config_path: tempfile::tempdir().unwrap().path().join("settings.toml"),
http_client: Some(fabro_http::test_http_client().expect("test HTTP client should build")),
sandbox_provider_registry: None,
sandbox_inventory: None,
shutdown: tokio_util::sync::CancellationToken::new(),
worker_control_bus: None,
worker_runtime: None,
@ -8457,7 +8457,7 @@ fn create_github_token_app_state_with_env_lookup_and_llm_catalog_settings(
github_api_base_url,
active_config_path,
http_client: Some(fabro_http::test_http_client().expect("test HTTP client should build")),
sandbox_provider_registry: None,
sandbox_inventory: None,
shutdown: tokio_util::sync::CancellationToken::new(),
worker_control_bus: None,
worker_runtime: None,

15
lib/apps/fabro-server/src/test_support.rs
View file

@ -19,7 +19,7 @@ use fabro_config::{LlmLayer, RunLayer, ServerSettingsBuilder, Storage, envfile};
use fabro_db::DbPool;
use fabro_interview::Interviewer;
use fabro_llm::lithos_catalog::Catalog;
use fabro_sandbox::SandboxProviderRegistry;
use fabro_sandbox::SandboxInventory;
use fabro_static::EnvVars;
use fabro_store::{ArtifactStore, Database, test_support as store_test_support};
use fabro_types::settings::ServerAuthMethod;
@ -90,7 +90,7 @@ pub struct TestAppStateBuilder {
manifest_run_defaults: RunLayer,
max_concurrent_runs: usize,
registry_factory_override: Option<Box<RegistryFactoryOverride>>,
sandbox_provider_registry: Option<SandboxProviderRegistry>,
sandbox_inventory: Option<SandboxInventory>,
store_bundle: Option<(Arc<Database>, ArtifactStore)>,
vault_path: Option<PathBuf>,
vault_entries: Vec<(String, String)>,
@ -112,7 +112,7 @@ impl Default for TestAppStateBuilder {
manifest_run_defaults: RunLayer::default(),
max_concurrent_runs: 5,
registry_factory_override: None,
sandbox_provider_registry: None,
sandbox_inventory: None,
store_bundle: None,
vault_path: None,
vault_entries: Vec::new(),
@ -160,11 +160,8 @@ impl TestAppStateBuilder {
self
}
pub fn sandbox_provider_registry(
mut self,
sandbox_provider_registry: SandboxProviderRegistry,
) -> Self {
self.sandbox_provider_registry = Some(sandbox_provider_registry);
pub fn sandbox_inventory(mut self, sandbox_inventory: SandboxInventory) -> Self {
self.sandbox_inventory = Some(sandbox_inventory);
self
}
@ -312,7 +309,7 @@ impl TestAppStateBuilder {
http_client: Some(
fabro_http::test_http_client().expect("test HTTP client should build"),
),
sandbox_provider_registry: self.sandbox_provider_registry,
sandbox_inventory: self.sandbox_inventory,
shutdown: CancellationToken::new(),
#[cfg(test)]
worker_control_bus: None,

49
lib/components/fabro-acp/src/transport.rs
View file

@ -10,9 +10,9 @@ use agent_client_protocol::{
};
use fabro_sandbox::{
DEFAULT_EXEC_OUTPUT_TAIL_BYTES, Error as SandboxError, Result as SandboxResult, RunSandbox,
StderrCollector, StdioProcessHandle, StdioProcessTermination,
StderrTail, StdioProcessHandle, Termination, command_termination, program_exit_code,
};
use fabro_types::{CommandTermination, ExecOutputTail};
use fabro_types::ExecOutputTail;
use futures::io::BufReader;
use futures::sink::unfold;
use futures::{AsyncBufReadExt, AsyncWriteExt, Stream};
@ -27,8 +27,8 @@ const CLEAN_EXIT_PROTOCOL_GRACE: Duration = Duration::from_millis(500);
#[derive(Clone)]
pub(crate) struct TransportState {
handle: Arc<TokioMutex<Option<StdioProcessHandle>>>,
stderr: Arc<TokioMutex<Option<StderrCollector>>>,
handle: Arc<TokioMutex<Option<Arc<dyn StdioProcessHandle>>>>,
stderr: Arc<TokioMutex<Option<StderrTail>>>,
startup_error: Arc<TokioMutex<Option<SandboxError>>>,
process_exit: Arc<TokioMutex<Option<AcpProcessExit>>>,
}
@ -43,7 +43,7 @@ impl TransportState {
}
}
async fn set_process(&self, handle: StdioProcessHandle, stderr: StderrCollector) {
async fn set_process(&self, handle: Arc<dyn StdioProcessHandle>, stderr: StderrTail) {
*self.handle.lock().await = Some(handle);
*self.stderr.lock().await = Some(stderr);
}
@ -52,10 +52,15 @@ impl TransportState {
*self.startup_error.lock().await = Some(error);
}
async fn set_process_exit(&self, termination: StdioProcessTermination, stderr: &str) {
async fn set_process_exit(
&self,
termination: Termination,
exit_code: Option<i32>,
stderr: &str,
) {
*self.process_exit.lock().await = Some(AcpProcessExit {
termination: termination.termination,
exit_code: termination.exit_code,
termination: command_termination(termination),
exit_code: program_exit_code(termination, exit_code),
exec_output_tail: redacted_stderr_tail(stderr),
});
}
@ -70,14 +75,14 @@ impl TransportState {
pub(crate) async fn terminate(&self) -> SandboxResult<()> {
if let Some(handle) = self.handle.lock().await.as_ref().cloned() {
handle.terminate().await?;
handle.terminate().await;
}
Ok(())
}
pub(crate) async fn stderr_tail(&self) -> String {
if let Some(stderr) = self.stderr.lock().await.as_ref().cloned() {
return stderr.tail_string().await;
return stderr.to_string_lossy();
}
String::new()
}
@ -125,7 +130,6 @@ impl ConnectTo<Client> for SandboxAcpTransport {
&self.command.to_shell_command(),
Some(&self.cwd),
Some(&env),
None,
)
.await
{
@ -136,9 +140,11 @@ impl ConnectTo<Client> for SandboxAcpTransport {
}
};
let handle = process.handle.clone();
let stderr = process.stderr.clone();
self.state.set_process(handle.clone(), stderr.clone()).await;
let handle: Arc<dyn StdioProcessHandle> = Arc::from(process.handle);
let stderr = process.stderr_tail.clone();
self.state
.set_process(Arc::clone(&handle), stderr.clone())
.await;
let incoming_lines = Box::pin(BufReader::new(process.stdout.compat()).lines())
as Pin<Box<dyn Stream<Item = IoResult<String>> + Send>>;
@ -158,25 +164,20 @@ impl ConnectTo<Client> for SandboxAcpTransport {
));
tokio::select! {
result = &mut protocol => {
if let Err(err) = handle.terminate().await {
tracing::warn!(error = %err, "Failed to terminate ACP process after protocol completion");
}
handle.terminate().await;
let _ = timeout(Duration::from_millis(500), handle.wait()).await;
result
}
termination = handle.wait() => {
let termination = termination.map_err(ProtocolError::into_internal_error)?;
let stderr = stderr.tail_string().await;
if termination.termination == CommandTermination::Exited
&& termination.exit_code == Some(0)
{
(termination, exit_code) = handle.wait() => {
let stderr = stderr.to_string_lossy();
if termination == Termination::Exited && exit_code == Some(0) {
// Stdio agents commonly exit immediately after writing their final response.
// Process wait can observe that exit before the line reader drains stdout.
if let Ok(result) = timeout(CLEAN_EXIT_PROTOCOL_GRACE, &mut protocol).await {
return result;
}
}
self.state.set_process_exit(termination, &stderr).await;
self.state.set_process_exit(termination, exit_code, &stderr).await;
Err(process_exited_before_protocol_completed())
}
}

28
lib/components/fabro-acp/tests/session.rs
View file

@ -10,8 +10,9 @@ use fabro_acp::{
run_acp_turn,
};
use fabro_sandbox::test_support::{MockSandbox, MockStdioProcess};
use fabro_sandbox::{RunSandbox, local_sandbox, shell_quote};
use fabro_sandbox::{RunSandbox, local_sandbox};
use fabro_util::error::collect_chain;
use fabro_util::shell;
use pebble_coding_agent::SteeringMessage;
use tokio::fs::{read_to_string, write};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader, DuplexStream};
@ -104,7 +105,10 @@ async fn session_lifecycle_initializes_sends_prompt_and_aggregates_text() {
.await
.expect("write fake ACP agent");
let raw_command = format!("python3 {}", shell_quote(&script_path.to_string_lossy()));
let raw_command = format!(
"python3 {}",
shell::shell_quote(&script_path.to_string_lossy())
);
let command = AcpProcessSpec::from_command_attr(&raw_command).expect("parse ACP command");
let sandbox: Arc<RunSandbox> = Arc::new(
local_sandbox(tempdir.path().to_path_buf())
@ -149,7 +153,10 @@ async fn steering_sends_followup_session_prompt_over_acp() {
.await
.expect("write fake ACP agent");
let raw_command = format!("python3 {}", shell_quote(&script_path.to_string_lossy()));
let raw_command = format!(
"python3 {}",
shell::shell_quote(&script_path.to_string_lossy())
);
let command = AcpProcessSpec::from_command_attr(&raw_command).expect("parse ACP command");
let sandbox: Arc<RunSandbox> = Arc::new(
local_sandbox(tempdir.path().to_path_buf())
@ -215,7 +222,10 @@ async fn interrupt_then_steer_sends_cancel_then_followup_session_prompt_over_acp
.await
.expect("write fake ACP agent");
let raw_command = format!("python3 {}", shell_quote(&script_path.to_string_lossy()));
let raw_command = format!(
"python3 {}",
shell::shell_quote(&script_path.to_string_lossy())
);
let command = AcpProcessSpec::from_command_attr(&raw_command).expect("parse ACP command");
let sandbox: Arc<RunSandbox> = Arc::new(
local_sandbox(tempdir.path().to_path_buf())
@ -292,7 +302,10 @@ async fn inline_interrupt_terminates_agent_that_ignores_cancel() {
.await
.expect("write fake ACP agent");
let raw_command = format!("python3 {}", shell_quote(&script_path.to_string_lossy()));
let raw_command = format!(
"python3 {}",
shell::shell_quote(&script_path.to_string_lossy())
);
let command = AcpProcessSpec::from_command_attr(&raw_command).expect("parse ACP command");
let sandbox: Arc<RunSandbox> = Arc::new(
local_sandbox(tempdir.path().to_path_buf())
@ -683,7 +696,10 @@ async fn run_fake_agent_with_activity(
write(&script_path, fake_acp_agent_script())
.await
.expect("write fake ACP agent");
let raw_command = format!("python3 {}", shell_quote(&script_path.to_string_lossy()));
let raw_command = format!(
"python3 {}",
shell::shell_quote(&script_path.to_string_lossy())
);
let command = AcpProcessSpec::from_command_attr(&raw_command).expect("parse ACP command");
let sandbox: Arc<RunSandbox> = Arc::new(
local_sandbox(tempdir.to_path_buf())

7
lib/components/fabro-hooks/src/executor.rs
View file

@ -8,7 +8,7 @@ use fabro_llm::credentials::CredentialProvider;
use fabro_llm::lithos_catalog::Catalog;
use fabro_llm::{Client, ClientOptions, Request};
use fabro_redact::redacted_url_for_log;
use fabro_sandbox::{RunSandbox, SecretRedactor};
use fabro_sandbox::{ExecResultExt as _, RunSandbox, SecretRedactor};
use fabro_types::PermissionLevel;
use fabro_types::settings::{InterpString, ResolveCtx, ResolveError};
use pebble_coding_agent::extensions::{
@ -189,7 +189,10 @@ impl HookExecutorImpl {
)
.await
{
Ok(result) => Self::parse_decision(result.exit_code.unwrap_or(-1), &result.stdout),
Ok(result) => Self::parse_decision(
result.program_exit_code().unwrap_or(-1),
&result.stdout_lossy(),
),
Err(e) => HookDecision::Block {
reason: Some(format!("sandbox exec failed: {e}")),
},

3
lib/components/fabro-sandbox/Cargo.toml
View file

@ -38,9 +38,6 @@ strum.workspace = true
tracing.workspace = true
reqwest.workspace = true
base64.workspace = true
hmac.workspace = true
sha2.workspace = true
hex.workspace = true
uuid.workspace = true
fabro-proc = { path = "../../foundation/fabro-proc" }
fabro-static.workspace = true

394
lib/components/fabro-sandbox/src/clone.rs
View file

@ -6,29 +6,33 @@
//! the repository checks out under `<repos_root>/<owner>/<repo>` and the
//! run works in `<workspace_root>/<repo>`, a symlink to the checkout. An
//! exact commit or a tag is pinned by the driver's clone options, which
//! fetch a tag by its fully qualified ref so a same-named branch is never
//! consulted; fabro verifies the checked-out head afterwards. Neither path
//! ever falls back to the branch head.
//! fetch the pin directly and attach the branch to it; an unavailable pin
//! fails the clone and never falls back to the branch head. The GitHub App
//! token travels with the clone per call and is then installed as the
//! checkout's ambient credentials, so the agent's own git commands can
//! push; the remote URL never carries it.
use std::time::Duration;
use fabro_github::token_source::ResolvedToken;
use fabro_redact::DisplaySafeUrl;
use fabro_types::SandboxProviderKind;
use sandbox_driver::{Git as _, GitCloneOptions, GitCredentials, Sandbox as DriverHandle};
use sandbox_driver::{
ExecResult, Git as _, GitCloneOptions, GitFailureKind, Sandbox as DriverHandle,
};
use tokio::time;
use crate::ExecResult;
use crate::clone_source::{self, GitHubRepoLayout, PinnedRevision};
use crate::exec::SandboxExec;
use crate::git_retry::{self, CredentialContext, GitRetryReason, RetryPlan};
use crate::push_credentials::PushCredentialState;
use crate::redact::redact_auth_url;
use crate::sandbox::shell_quote;
use crate::clone_source::{self, GitHubRepoLayout};
use crate::credentials::{self, RepoCredentials};
use crate::exec::{ExecResultExt, SandboxExec};
use crate::git_policy;
/// Whole-clone budget, shared by every network and local step.
pub(crate) const GIT_CLONE_TIMEOUT: Duration = Duration::from_mins(5);
const STEP_TIMEOUT: Duration = Duration::from_secs(10);
/// What the operator hears when the image has no `git`: the driver classifies
/// the failing command, and fabro names the fix.
const GIT_UNAVAILABLE_MESSAGE: &str = "The sandbox image must include git for repository \
clone and git lifecycle operations. Use an image with \
bash and git, such as buildpack-deps:noble.";
/// A GitHub clone fabro decided to perform.
#[derive(Clone, Debug, PartialEq, Eq)]
@ -40,8 +44,7 @@ pub(crate) struct GitHubClone {
pub(crate) depth: Option<u32>,
}
/// What the clone left behind: the layout and the token now embedded in
/// `origin`, if any.
/// What the clone left behind: the layout it checked out into.
pub(crate) struct CloneOutcome {
pub(crate) layout: GitHubRepoLayout,
}
@ -54,14 +57,10 @@ enum CloneStep {
Local,
}
struct CloneFailure {
error: crate::Error,
retry_reason: Option<GitRetryReason>,
}
/// Clone `plan` into `handle`, laid out under `workspace_root` and
/// `repos_root`, embedding a GitHub App token from `credentials` when one
/// is available.
/// `repos_root`, with a GitHub App token from `credentials` when one is
/// available: the clone carries it per call, and the checkout keeps it as
/// ambient credentials afterwards.
pub(crate) async fn clone_github_repo(
kind: &SandboxProviderKind,
handle: &dyn DriverHandle,
@ -69,34 +68,10 @@ pub(crate) async fn clone_github_repo(
plan: &GitHubClone,
workspace_root: &str,
repos_root: &str,
credentials: &PushCredentialState,
credentials: &RepoCredentials,
) -> crate::Result<CloneOutcome> {
verify_git_available(exec).await?;
let layout = clone_source::github_repo_layout(&plan.origin_url, workspace_root, repos_root)?;
// The clone mints its own token (never a warm-cache reuse) and seeds the
// shared source, so the first refresh compares against the clone token
// instead of believing nothing was ever embedded.
let resolved_token = match credentials.source() {
Some(source) => Some(source.mint_for_clone().await.map_err(|err| {
crate::Error::context_anyhow("Failed to get GitHub App credentials for clone", err)
})?),
None => None,
};
let credential_context =
CredentialContext::from_snapshot(resolved_token.as_ref().map(|token| &token.snapshot));
let auth_url = match &resolved_token {
Some(token) => Some(
fabro_github::embed_token_in_url(&plan.origin_url, token.token.expose()).map_err(
|err| {
crate::Error::context_anyhow(
"Failed to build authenticated GitHub clone URL",
err,
)
},
)?,
),
None => None,
};
let token = credentials.mint_for_clone().await?;
let fs = handle.fs();
for dir in [workspace_root, layout.repos_owner_path.as_str()] {
@ -106,7 +81,7 @@ pub(crate) async fn clone_github_repo(
}
let deadline = time::Instant::now() + GIT_CLONE_TIMEOUT;
let has_app = credentials.source().is_some();
let has_app = credentials.managed();
let git = handle.git().ok_or_else(|| {
crate::Error::message(format!(
"sandbox provider `{kind}` does not support git operations"
@ -123,84 +98,47 @@ pub(crate) async fn clone_github_repo(
options.commit = plan.commit_sha.clone();
options.tag = plan.tag.clone().filter(|_| plan.commit_sha.is_none());
options.depth = plan.depth;
options.credentials = resolved_token
.as_ref()
.map(|token| GitCredentials::new("x-access-token", token.token.expose()));
let retry_plan = RetryPlan::clone_default(Some(deadline));
options.credentials = token.as_ref().map(credentials::git_credentials);
// The driver retries a clone the remote refused while the token may
// still be replicating, inside what is left of the clone budget.
let policy = git_policy::clone_policy(deadline.saturating_duration_since(time::Instant::now()));
let target = layout.primary_repo_path.clone();
git_retry::retry_git_operation(
kind.clone(),
"clone",
&retry_plan,
|_attempt| {
let options = options.clone();
let target = target.clone();
let origin_url = plan.origin_url.clone();
sandbox_driver::retry_git(
&policy,
options.credentials.as_ref(),
"git clone",
|_attempt, _timeout| {
let git = &git;
async move {
git.clone_repo(&origin_url, &target, &options)
.await
.map_err(|error| CloneFailure {
retry_reason: git_retry::classify_driver_failure(
&error,
credential_context,
),
error: clone_failure_error(
crate::Error::driver_error(error),
CloneStep::Network,
has_app,
),
})
}
let options = &options;
let target = &target;
let origin_url = &plan.origin_url;
async move { git.clone_repo(origin_url, target, options).await }
},
|failure: &CloneFailure| failure.retry_reason,
)
.await
.map_err(|failure| failure.error)?;
if let Some(pin) =
PinnedRevision::from_selectors(plan.tag.as_deref(), plan.commit_sha.as_deref())
{
let head = run_local_step(
exec,
&clone_source::exact_head_revision_command(&layout.primary_repo_path),
"git rev-parse HEAD (pinned checkout)",
deadline,
auth_url.as_ref(),
.map_err(|failure| {
clone_failure_error(
crate::Error::from(failure.error),
CloneStep::Network,
has_app,
)
.await?;
pin.verify_head(&head.stdout)?;
}
})?;
run_local_step(
exec,
&clone_source::repo_symlink_command(&layout),
"create workspace repo symlink",
deadline,
auth_url.as_ref(),
has_app,
)
.await?;
if let Some(token) = resolved_token {
embed_origin_credentials(exec, &layout, auth_url.as_ref(), token, credentials).await;
if let Some(token) = &token {
RepoCredentials::install(&git, &layout.primary_repo_path, token).await?;
}
Ok(CloneOutcome { layout })
}
async fn verify_git_available(exec: &SandboxExec<'_>) -> crate::Result<()> {
let result = exec
.run("git --version", Some(STEP_TIMEOUT), Some("/"), None, None)
.await?;
if !result.is_success() {
return Err(crate::Error::message(
"The sandbox image must include git for repository clone and git lifecycle \
operations. Use an image with bash and git, such as buildpack-deps:noble.",
));
}
Ok(())
}
/// Run a local (non-network) step under the shared clone deadline.
///
/// Materializing a large working tree takes far longer than the short fixed
@ -211,7 +149,6 @@ async fn run_local_step(
command: &str,
label: &'static str,
deadline: time::Instant,
auth_url: Option<&DisplaySafeUrl>,
has_app: bool,
) -> crate::Result<ExecResult> {
let remaining = deadline.saturating_duration_since(time::Instant::now());
@ -224,17 +161,20 @@ async fn run_local_step(
.run(command, Some(remaining), Some("/"), None, None)
.await
.map_err(|error| crate::Error::context(format!("{label} transport failed"), error))?;
if result.is_success() {
if result.success() {
return Ok(result);
}
Err(clone_failure_error(
result.into_exec_error_with_redactor(label, |output| redact_auth_url(output, auth_url)),
result.into_exec_error(label),
CloneStep::Local,
has_app,
))
}
fn clone_failure_error(error: crate::Error, step: CloneStep, has_app: bool) -> crate::Error {
if git_unavailable(&error) {
return crate::Error::context(GIT_UNAVAILABLE_MESSAGE, error);
}
let message = match step {
CloneStep::Network if !has_app => {
"Git clone failed. If this is a private repository, configure a GitHub App with \
@ -246,51 +186,195 @@ fn clone_failure_error(error: crate::Error, step: CloneStep, has_app: bool) -> c
crate::Error::context(message, error)
}
/// Point `origin` at the authenticated URL so pushes from the checkout
/// carry the clone token, and record that generation for refreshes. A
/// failure here is logged, not fatal: the checkout is complete, and the
/// first push will re-embed.
async fn embed_origin_credentials(
exec: &SandboxExec<'_>,
layout: &GitHubRepoLayout,
auth_url: Option<&DisplaySafeUrl>,
token: ResolvedToken,
credentials: &PushCredentialState,
) {
credentials.record_embedded(token).await;
let Some(auth_url) = auth_url else {
return;
};
let command = format!(
"git -c maintenance.auto=0 remote set-url origin {}",
shell_quote(auth_url.as_raw_url().as_str())
);
match exec
.run(
&command,
Some(STEP_TIMEOUT),
Some(&layout.execution_directory),
None,
None,
)
.await
{
Ok(result) if result.is_success() => {}
Ok(result) => {
let err = result
.into_exec_error_with_redactor("git remote set-url origin (post-clone)", |s| {
redact_auth_url(s, Some(auth_url))
});
tracing::warn!(
error = %err,
"Failed to set sandbox push credentials on origin; git push from this sandbox will fail"
);
}
Err(err) => {
tracing::warn!(
error = %redact_auth_url(&crate::display_for_log(&err), Some(auth_url)),
"Failed to set sandbox push credentials on origin; git push from this sandbox will fail"
);
/// Whether the driver found no usable `git` in the sandbox.
fn git_unavailable(error: &crate::Error) -> bool {
matches!(
error.driver(),
Some(sandbox_driver::Error::Git(failure))
if failure.kind() == GitFailureKind::GitUnavailable
)
}
#[cfg(test)]
mod tests {
use fabro_github::token_source::InstallationTokenSource;
use sandbox_driver::{ExecFailure, GitFailure, Termination};
use sandbox_driver_testing::ScriptedSandbox;
use super::*;
const ORIGIN: &str = "https://github.com/acme/widgets";
fn ok() -> ExecResult {
ExecResult::new(Termination::Exited, Some(0), Duration::from_millis(1))
}
/// A scripted sandbox whose `origin` answers with the fixture URL and
/// whose every other command succeeds.
fn scripted_handle() -> ScriptedSandbox {
let handle = ScriptedSandbox::with_id_and_working_dir("scripted", "/workspace")
.runtime_directory("/tmp/sandbox-driver/runtime");
handle.scripted_exec().respond_with(|spec| {
let script = spec.args.last().map(String::as_str).unwrap_or_default();
script.contains("'remote' 'get-url' 'origin'").then(|| {
let mut result = ok();
result.stdout = format!("{ORIGIN}\n").into_bytes();
result
})
});
handle.scripted_exec().set_default(ok());
handle
}
fn plan() -> GitHubClone {
GitHubClone {
origin_url: ORIGIN.to_owned(),
branch: Some("main".to_owned()),
tag: None,
commit_sha: None,
depth: Some(1),
}
}
async fn clone_with(handle: &ScriptedSandbox, credentials: &RepoCredentials) -> CloneOutcome {
let exec = SandboxExec::new(handle.exec());
clone_github_repo(
&SandboxProviderKind::DOCKER,
handle,
&exec,
&plan(),
"/workspace",
"/repos",
credentials,
)
.await
.expect("clone succeeds")
}
#[tokio::test]
async fn a_clone_carries_the_token_per_call_and_installs_it_for_the_checkout() {
let handle = scripted_handle();
let credentials =
RepoCredentials::new(Some(InstallationTokenSource::pat("ghp_test".to_owned())));
let outcome = clone_with(&handle, &credentials).await;
assert_eq!(outcome.layout.primary_repo_path, "/repos/acme/widgets");
let commands = handle.scripted_exec().commands();
assert!(
commands
.iter()
.all(|command| !command.contains("git --version")),
"no probe runs ahead of the clone: {commands:#?}"
);
assert!(
commands.iter().all(|command| !command.contains("set-url")),
"the remote URL is never rewritten: {commands:#?}"
);
let clone = commands
.iter()
.find(|command| command.contains("'clone'"))
.expect("the clone ran");
assert!(
clone.contains(
"x-access-token:ghp_test@github.com/acme/widgets.insteadOf=https://github.com/acme/widgets"
),
"the clone carries the token per call: {clone}"
);
assert!(
commands.iter().any(|command| command.starts_with("ln -s ")),
"{commands:#?}"
);
let install = commands
.iter()
.find(|command| command.contains("--add credential.helper"))
.expect("the checkout's credential store is installed");
assert!(
install.contains("/tmp/sandbox-driver/runtime/git-credentials/"),
"{install}"
);
assert!(
commands
.iter()
.all(|command| !command.contains("ghp_test") || command.contains("insteadOf")),
"the secret enters no command but the clone's own rewrite: {commands:#?}"
);
assert!(
handle.scripted_exec().recorded().iter().any(|spec| {
spec.env
.get("SANDBOX_DRIVER_GIT_CREDENTIAL")
.map(String::as_str)
== Some("https://x-access-token:ghp_test@github.com")
}),
"the store line travels in the environment"
);
}
#[tokio::test]
async fn a_clone_without_managed_credentials_installs_nothing() {
let handle = scripted_handle();
clone_with(&handle, &RepoCredentials::none()).await;
let commands = handle.scripted_exec().commands();
assert!(
commands.iter().any(|command| command.contains("'clone'")),
"{commands:#?}"
);
assert!(
commands
.iter()
.all(|command| !command.contains("insteadOf")
&& !command.contains("credential.helper")),
"{commands:#?}"
);
}
fn git_failure(exit_code: i32, stderr: &str) -> crate::Error {
crate::Error::from(sandbox_driver::Error::Git(GitFailure::from_command(
"git clone",
ExecFailure::new(
"git clone",
Termination::Exited,
Some(exit_code),
Vec::new(),
stderr.as_bytes().to_vec(),
),
)))
}
#[test]
fn a_missing_git_executable_names_the_image_requirement() {
let error = clone_failure_error(
git_failure(127, "bash: line 1: git: command not found"),
CloneStep::Network,
true,
);
assert!(
error.to_string().contains("image must include git"),
"{error}"
);
}
#[test]
fn other_network_failures_keep_the_credential_guidance() {
let without_app = clone_failure_error(
git_failure(128, "remote: Repository not found."),
CloneStep::Network,
false,
);
assert!(without_app.to_string().contains("fabro install"));
let with_app = clone_failure_error(
git_failure(128, "remote: Repository not found."),
CloneStep::Network,
true,
);
assert!(
with_app
.to_string()
.contains("Failed to clone repository into the sandbox")
);
let local = clone_failure_error(git_failure(1, "ln: failed"), CloneStep::Local, true);
assert!(local.to_string().contains("prepare the cloned repository"));
}
}

108
lib/components/fabro-sandbox/src/clone_source.rs
View file

@ -1,3 +1,5 @@
use fabro_util::shell;
use crate::sandbox;
#[derive(Clone, Debug, PartialEq, Eq)]
@ -68,24 +70,26 @@ fn validate_path_component(label: &str, component: &str) -> crate::Result<()> {
pub(crate) fn repo_symlink_command(layout: &GitHubRepoLayout) -> String {
format!(
"ln -s {} {}",
sandbox::shell_quote(&layout.primary_repo_path),
sandbox::shell_quote(&layout.primary_repo_link),
shell::shell_quote(&layout.primary_repo_path),
shell::shell_quote(&layout.primary_repo_link),
)
}
/// A revision the checkout is pinned to instead of the branch's current HEAD.
/// The kind of revision a checkout is pinned to instead of the branch's
/// current HEAD.
///
/// The working branch names the checkout the run works on; it never constrains
/// which revision is fetched. No layer proves branch/revision ancestry, and an
/// unavailable revision fails without falling back to branch HEAD.
#[derive(Debug, Clone, PartialEq, Eq)]
/// which revision is fetched. No layer proves branch/revision ancestry. The
/// driver fetches the pin directly and attaches the branch to it, so an
/// unavailable revision fails the clone without falling back to branch HEAD,
/// and a successful clone has the pin checked out.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum PinnedRevision {
/// An exact commit SHA, already normalized by
/// [`normalize_exact_commit_sha`].
Commit(String),
/// An exact commit SHA.
Commit,
/// A bare tag name; the driver fetches it as `refs/tags/<tag>` so a
/// same-named branch is never consulted.
Tag(String),
Tag,
}
impl PinnedRevision {
@ -93,60 +97,19 @@ impl PinnedRevision {
/// target as durable identity but does not drive the checkout.
pub(crate) fn from_selectors(tag: Option<&str>, commit_sha: Option<&str>) -> Option<Self> {
match (commit_sha, tag) {
(Some(sha), _) => Some(Self::Commit(sha.to_string())),
(None, Some(tag)) => Some(Self::Tag(tag.to_string())),
(Some(_), _) => Some(Self::Commit),
(None, Some(_)) => Some(Self::Tag),
(None, None) => None,
}
}
/// Human-readable prefix for error messages.
pub(crate) fn label(&self) -> &'static str {
pub(crate) fn label(self) -> &'static str {
match self {
Self::Commit(_) => "Exact commit checkout",
Self::Tag(_) => "Tag checkout",
Self::Commit => "Exact commit checkout",
Self::Tag => "Tag checkout",
}
}
/// The commit HEAD must resolve to after checkout, when one is known.
pub(crate) fn expected_sha(&self) -> Option<&str> {
match self {
Self::Commit(sha) => Some(sha),
Self::Tag(_) => None,
}
}
/// Validate the `rev-parse HEAD` output of a pinned checkout and return the
/// resolved commit ID.
pub(crate) fn verify_head(&self, output: &str) -> crate::Result<String> {
let actual_sha = verify_resolved_head(output)?;
if self
.expected_sha()
.is_some_and(|expected| expected != actual_sha)
{
return Err(crate::Error::message(
"Exact checkout HEAD did not match the requested commit",
));
}
Ok(actual_sha)
}
}
/// Print the current HEAD commit and nothing else, for
/// [`PinnedRevision::verify_head`].
pub(crate) fn exact_head_revision_command(checkout_path: &str) -> String {
format!(
"{git} -C {path} rev-parse HEAD",
path = sandbox::shell_quote(checkout_path),
git = sandbox::GIT,
)
}
/// Validate that a `rev-parse HEAD` output is a single commit ID and return it
/// normalized.
pub(crate) fn verify_resolved_head(output: &str) -> crate::Result<String> {
normalize_exact_commit_sha(output.trim()).map_err(|err| {
crate::Error::context("Pinned checkout produced an invalid HEAD commit ID", err)
})
}
fn trim_root(root: &str) -> &str {
@ -334,13 +297,17 @@ mod tests {
}
#[test]
fn pinned_revision_prefers_exact_commit_and_qualifies_tags() {
fn pinned_revision_prefers_exact_commit_over_a_tag() {
let sha = "0123456789abcdef0123456789abcdef01234567";
assert_eq!(PinnedRevision::from_selectors(None, None), None);
let tag = PinnedRevision::from_selectors(Some("release/v1"), None).unwrap();
assert_eq!(tag.expected_sha(), None);
let commit = PinnedRevision::from_selectors(Some("release/v1"), Some(sha)).unwrap();
assert_eq!(commit.expected_sha(), Some(sha));
assert_eq!(
PinnedRevision::from_selectors(Some("release/v1"), None),
Some(PinnedRevision::Tag)
);
assert_eq!(
PinnedRevision::from_selectors(Some("release/v1"), Some(sha)),
Some(PinnedRevision::Commit)
);
}
#[test]
@ -468,25 +435,6 @@ mod tests {
assert!(empty_tag.to_string().contains("non-empty tag"));
}
#[test]
fn exact_checkout_verification_rejects_invalid_or_mismatched_head() {
let expected = "0123456789abcdef0123456789abcdef01234567";
let pin = PinnedRevision::Commit(expected.to_string());
pin.verify_head("0123456789ABCDEF0123456789ABCDEF01234567\n")
.expect("uppercase command output should normalize");
let invalid = pin
.verify_head("fatal: not a revision")
.expect_err("non-SHA output should fail verification");
assert!(invalid.to_string().contains("invalid HEAD commit ID"));
assert!(!invalid.to_string().contains("fatal: not a revision"));
let mismatched = pin
.verify_head("1123456789abcdef0123456789abcdef01234567")
.expect_err("mismatched SHA should fail verification");
assert!(mismatched.to_string().contains("did not match"));
}
#[test]
fn github_layout_maps_ssh_origin_to_repos_checkout_and_workspace_link() {
let layout = github_repo_layout(

158
lib/components/fabro-sandbox/src/credentials.rs Normal file
View file

@ -0,0 +1,158 @@
//! GitHub credentials for a clone-based sandbox's repository.
//!
//! Fabro decides which credential a checkout works with and when it is
//! renewed; the sandbox driver applies it. The facet's own network
//! operations, fabro's clone and pushes, take the token per call and never
//! write it into the repository. The agent's own git commands read it from
//! the credential store the driver installs beside the checkout, which the
//! workflow's refresh tick rewrites as the token is renewed. The remote URL
//! is never touched, so no secret shows in `git remote -v` or in
//! `.git/config`. The token cache itself sits below, in
//! [`InstallationTokenSource`].
use std::sync::Arc;
use std::time::SystemTime;
use fabro_github::GitHubCredentials;
use fabro_github::token_source::{InstallationTokenSource, ResolvedToken};
use sandbox_driver::{Git as _, GitCredentials, GitFacet};
/// The username GitHub expects with an installation token or PAT.
pub(crate) const GITHUB_TOKEN_USERNAME: &str = "x-access-token";
/// Build the shared installation-token source for a clone-based sandbox.
///
/// Returns `None` when there are no managed credentials or no GitHub origin
/// to scope them to. Minted tokens carry the same `contents: write`
/// permission the clone token uses.
pub(crate) fn build_token_source(
github_app: Option<&GitHubCredentials>,
clone_origin_url: Option<&str>,
) -> crate::Result<Option<Arc<InstallationTokenSource>>> {
let Some(creds) = github_app else {
return Ok(None);
};
let Some(origin_url) = clone_origin_url.filter(|url| !url.trim().is_empty()) else {
return Ok(None);
};
let normalized = fabro_github::normalize_repo_origin_url(origin_url);
let Ok((owner, repo)) = fabro_github::parse_github_owner_repo(&normalized) else {
// Non-GitHub origins never clone in these providers, so there is no
// remote to keep credentials fresh for.
return Ok(None);
};
InstallationTokenSource::for_repository(
creds,
owner,
repo,
serde_json::json!({ "contents": "write" }),
)
.map(Some)
.map_err(|err| crate::Error::context_anyhow("Failed to build GitHub token source", err))
}
/// The GitHub credentials a run's checkout works with: a token source when
/// fabro manages them, nothing when the repository was cloned without a
/// GitHub App or the sandbox was reattached by a later process.
pub(crate) struct RepoCredentials {
source: Option<Arc<InstallationTokenSource>>,
}
impl RepoCredentials {
pub(crate) fn new(source: Option<Arc<InstallationTokenSource>>) -> Self {
Self { source }
}
/// No managed credentials: pushes and the agent's git commands use
/// whatever the checkout already has.
pub(crate) fn none() -> Self {
Self::new(None)
}
pub(crate) fn source(&self) -> Option<&Arc<InstallationTokenSource>> {
self.source.as_ref()
}
pub(crate) fn managed(&self) -> bool {
self.source.is_some()
}
/// Mint the clone token. Never a warm-cache reuse: a clone retried on
/// replication lag must hold the token minted for it. The mint seeds
/// the source, so later resolves reuse this token until it nears
/// expiry.
pub(crate) async fn mint_for_clone(&self) -> crate::Result<Option<ResolvedToken>> {
let Some(source) = &self.source else {
return Ok(None);
};
source.mint_for_clone().await.map(Some).map_err(|err| {
crate::Error::context_anyhow("Failed to get GitHub App credentials for clone", err)
})
}
/// The token one operation works with, reused from the cache until it
/// nears expiry. A refresh that fails while the cached token is still
/// valid returns that token.
pub(crate) async fn resolve(&self) -> crate::Result<Option<ResolvedToken>> {
let Some(source) = &self.source else {
return Ok(None);
};
source.resolve().await.map(Some).map_err(|err| {
crate::Error::context_anyhow("Failed to refresh GitHub App credentials", err)
})
}
/// Install `token` as the credentials every git command run inside the
/// sandbox picks up for the checkout at `repo_path`. The driver keeps
/// them in a credential store beside the checkout and points the
/// repository's helper configuration at it; calling again replaces
/// them in place.
pub(crate) async fn install(
git: &GitFacet<'_>,
repo_path: &str,
token: &ResolvedToken,
) -> crate::Result<()> {
git.set_ambient_credentials(repo_path, Some(&git_credentials(token)))
.await
.map_err(|error| {
crate::Error::context("Failed to install the checkout's GitHub credentials", error)
})
}
}
/// The per-call form of `token` for the driver's network operations. The
/// mint time travels with a minted token so the driver's retry knows a
/// rejection may be replication lag; a static credential carries none.
pub(crate) fn git_credentials(token: &ResolvedToken) -> GitCredentials {
let credentials = GitCredentials::new(GITHUB_TOKEN_USERNAME, token.token.expose());
match token.snapshot.minted_at() {
Some(minted_at) => credentials.minted_at(SystemTime::from(minted_at)),
None => credentials,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn unmanaged_credentials_resolve_to_nothing() {
let credentials = RepoCredentials::none();
assert!(!credentials.managed());
assert!(credentials.mint_for_clone().await.unwrap().is_none());
assert!(credentials.resolve().await.unwrap().is_none());
}
#[tokio::test]
async fn a_pat_becomes_per_call_credentials_under_the_github_username() {
let source = InstallationTokenSource::pat("ghp_static".to_owned());
let token = source.resolve().await.unwrap();
let credentials = git_credentials(&token);
assert_eq!(credentials.username, GITHUB_TOKEN_USERNAME);
assert_eq!(credentials.password, "ghp_static");
assert!(
credentials.minted_at.is_none(),
"a static credential has no mint time"
);
}
}

638
lib/components/fabro-sandbox/src/daytona.rs
View file

@ -1,30 +1,28 @@
//! The `daytona` provider kind: what fabro adds to a run's spec for the
//! sandbox-driver Daytona provider.
//!
//! The environment's options build the spec once; Daytona's overlay creates
//! sandboxes from a snapshot (built from the environment's image or
//! Dockerfile and named by an HMAC of its inputs, or Daytona's default when
//! the environment names neither), fixes the working directory, and sets
//! the lifecycle timers. The run works in `/home/daytona/workspace`, with a
//! cloned repository checked out under `/home/daytona/repos` and linked
//! into the workspace.
//! The environment's options build the spec once; Daytona's overlay fixes
//! the working directory, names the run, sets the lifecycle timers, and
//! falls back to Daytona's default snapshot when the environment names no
//! image or Dockerfile. An image or Dockerfile goes to the driver as is:
//! the Daytona provider builds it into a snapshot named by its inputs under
//! the API key and reuses that snapshot for the same inputs. The run works
//! in `/home/daytona/workspace`, with a cloned repository checked out under
//! `/home/daytona/repos` and linked into the workspace.
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use fabro_types::settings::server::ServerSandboxProviderSettings;
use fabro_types::{RunId, SandboxProviderKind};
use sandbox_driver::{
EventContext, HealthStatus, LifecycleTimers, Resources, SandboxProvider, SandboxSource,
SandboxSpec as DriverSpec, SnapshotId, SnapshotSource, SnapshotSpec,
HealthStatus, Resources, SandboxProvider, SandboxSource, SandboxSpec as DriverSpec, SnapshotId,
};
use tokio::time;
pub use crate::driver::DaytonaCredentials;
use crate::driver::{ProviderConnectOptions, connect_provider};
use crate::driver_sandbox::{CreatePlan, PreparedCreate, WorkspaceLayout};
use crate::options::SandboxOptions;
use crate::driver_sandbox::WorkspaceLayout;
pub(crate) const WORKING_DIRECTORY: &str = "/home/daytona/workspace";
pub(crate) const REPOS_ROOT: &str = "/home/daytona/repos";
@ -32,8 +30,6 @@ const DEFAULT_SNAPSHOT: &str = "daytona-medium";
pub const DEFAULT_DAYTONA_API_URL: &str = "https://app.daytona.io/api";
/// Budget for the credential probe `fabro doctor` and the install flow run.
pub const DAYTONA_CREDENTIAL_PROBE_TIMEOUT: Duration = Duration::from_secs(20);
/// Budget for a custom snapshot to reach Daytona's active state.
const DAYTONA_SNAPSHOT_ACTIVE_TIMEOUT: Duration = Duration::from_mins(30);
/// Auto-stop applied when `lifecycle.auto_stop` is unset. Omitting the timer
/// would inherit Daytona's server-side default of 15 idle minutes, which is
/// shorter than a single long inference call and stops the sandbox mid-run;
@ -41,137 +37,15 @@ const DAYTONA_SNAPSHOT_ACTIVE_TIMEOUT: Duration = Duration::from_mins(30);
/// leaked by a dead worker. An explicit zero disables auto-stop entirely.
const DEFAULT_AUTO_STOP: Duration = Duration::from_hours(2);
/// Scopes a Daytona API key needs for fabro's snapshot and sandbox flow, in
/// the order the remediation text lists them.
pub const REQUIRED_DAYTONA_SCOPES: &[&str] = &[
"write:snapshots",
"delete:snapshots",
"write:sandboxes",
"delete:sandboxes",
];
/// What a custom snapshot is built from: the environment's image or
/// Dockerfile and its resources in whole gigabytes, the units Daytona
/// sizes snapshots in and the values the snapshot's name is derived from.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct SnapshotInputs<'a> {
pub source: SnapshotInput<'a>,
pub cpu: Option<i32>,
pub memory_gb: Option<i32>,
pub disk_gb: Option<i32>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SnapshotInput<'a> {
/// A pullable image reference such as `ubuntu:24.04`.
Image(&'a str),
/// A Dockerfile Daytona builds into the snapshot.
Dockerfile(&'a str),
}
/// The snapshot `options` ask for, or `None` when the environment names no
/// image or Dockerfile and the sandbox comes from Daytona's default.
pub fn snapshot_inputs(options: &SandboxOptions) -> Option<SnapshotInputs<'_>> {
let source = match (&options.image, &options.dockerfile) {
(Some(image), _) => SnapshotInput::Image(image),
(None, Some(dockerfile)) => SnapshotInput::Dockerfile(dockerfile),
(None, None) => return None,
};
Some(SnapshotInputs {
source,
cpu: options.cpu.and_then(|cpu| i32::try_from(cpu).ok()),
memory_gb: options.memory_bytes.map(bytes_to_gb),
disk_gb: options.disk_bytes.map(bytes_to_gb),
})
}
/// Whole decimal gigabytes, the unit Daytona sizes snapshots in.
fn bytes_to_gb(bytes: u64) -> i32 {
i32::try_from(bytes / 1_000_000_000).unwrap_or(i32::MAX)
}
pub mod snapshot_identity {
use hmac::{Hmac, Mac};
use serde::Serialize;
use sha2::{Digest, Sha256};
use uuid::Uuid;
use super::{SnapshotInput, SnapshotInputs};
const IDENTITY_VERSION: u8 = 1;
const PROVIDER: &str = "daytona";
const TENANT: &str = "single-tenant";
type HmacSha256 = Hmac<Sha256>;
/// The snapshot source as it appears in the identity manifest. Each
/// variant flattens into a single `"<key>": "<value>"` entry.
#[derive(Serialize)]
#[serde(rename_all = "snake_case")]
enum SourceManifest<'a> {
DockerfileSha256(String),
Image(&'a str),
}
#[derive(Serialize)]
struct SnapshotManifest<'a> {
identity_version: u8,
provider: &'static str,
tenant: &'static str,
#[serde(flatten)]
source: SourceManifest<'a>,
cpu: Option<i32>,
memory_gb: Option<i32>,
disk_gb: Option<i32>,
/// Nothing sets an entrypoint yet. The field stays because removing
/// it would rename every existing snapshot under `IDENTITY_VERSION` 1.
entrypoint: Option<&'static str>,
}
/// The name of the snapshot built from `inputs`: a UUIDv8 derived from an
/// HMAC of the build inputs keyed by the API key, so the same inputs reuse
/// the same snapshot and a rotated key never collides with another
/// tenant's.
pub fn snapshot_name(api_key: &str, inputs: &SnapshotInputs<'_>) -> crate::Result<String> {
let manifest = canonical_manifest(inputs)?;
let mut mac = HmacSha256::new_from_slice(api_key.as_bytes())
.expect("HMAC-SHA256 accepts keys of any length");
mac.update(&manifest);
let digest = mac.finalize().into_bytes();
let mut bytes = [0_u8; 16];
bytes.copy_from_slice(&digest[..16]);
Ok(format!("fabro-{}", Uuid::new_v8(bytes)))
}
fn canonical_manifest(inputs: &SnapshotInputs<'_>) -> crate::Result<Vec<u8>> {
let source = match inputs.source {
SnapshotInput::Image(image) => SourceManifest::Image(image),
SnapshotInput::Dockerfile(text) => {
SourceManifest::DockerfileSha256(hex::encode(Sha256::digest(text.as_bytes())))
}
};
let manifest = SnapshotManifest {
identity_version: IDENTITY_VERSION,
provider: PROVIDER,
tenant: TENANT,
source,
cpu: inputs.cpu,
memory_gb: inputs.memory_gb,
disk_gb: inputs.disk_gb,
entrypoint: None,
};
serde_json::to_vec(&manifest).map_err(|err| {
crate::Error::context("Failed to serialize Daytona snapshot identity", err)
})
}
}
/// Outcome of probing a Daytona credential through the provider's health
/// check.
/// check. The provider owns the list of scopes it needs and the order it
/// reports them in; fabro only renders them.
#[derive(Debug)]
pub struct DaytonaKeyCheck {
/// Scopes the key lacks, in Daytona's wire names.
pub missing: Vec<String>,
pub missing: Vec<String>,
/// Every scope the provider requires, for the remediation text.
pub required: Vec<String>,
}
#[derive(Debug, thiserror::Error)]
@ -211,11 +85,12 @@ impl DaytonaKeyCheck {
self.missing_display()
)
}
}
#[must_use]
pub fn required_perms_display() -> String {
REQUIRED_DAYTONA_SCOPES.join(", ")
/// Every scope the provider requires, comma separated, for remediation.
#[must_use]
pub fn required_display(&self) -> String {
self.required.join(", ")
}
}
/// Whether `credentials` reach Daytona, are accepted, and carry the scopes
@ -233,11 +108,13 @@ pub async fn check_daytona_api_key(
.map_err(|error| anyhow::Error::new(error).context("Daytona health check failed"))?;
match health.status {
HealthStatus::Ok | HealthStatus::Unknown => Ok(DaytonaKeyCheck {
missing: Vec::new(),
missing: Vec::new(),
required: health.required_permissions,
}),
HealthStatus::Unauthorized if !health.missing_permissions.is_empty() => {
Ok(DaytonaKeyCheck {
missing: ordered_scopes(&health.missing_permissions),
missing: health.missing_permissions,
required: health.required_permissions,
})
}
HealthStatus::Unauthorized => Err(anyhow::anyhow!(
@ -262,22 +139,6 @@ pub async fn check_daytona_api_key(
}
}
/// The scopes fabro requires, in fabro's documented order, followed by any
/// other scope the provider reported missing.
fn ordered_scopes(missing: &[String]) -> Vec<String> {
let mut ordered: Vec<String> = REQUIRED_DAYTONA_SCOPES
.iter()
.filter(|scope| missing.iter().any(|reported| reported == *scope))
.map(|scope| (*scope).to_string())
.collect();
for scope in missing {
if !ordered.contains(scope) {
ordered.push(scope.clone());
}
}
ordered
}
async fn connect(credentials: &DaytonaCredentials) -> anyhow::Result<Arc<dyn SandboxProvider>> {
connect_provider(
&SandboxProviderKind::DAYTONA,
@ -300,203 +161,61 @@ pub(crate) fn layout() -> WorkspaceLayout {
}
}
/// Daytona's additions to the base spec: the snapshot the sandbox is created
/// from, the fixed working directory, the run's Daytona name, and the
/// lifecycle timers.
pub(crate) fn overlay(
spec: DriverSpec,
options: &SandboxOptions,
run_id: Option<&RunId>,
snapshot: &SnapshotId,
) -> DriverSpec {
/// Daytona's additions to the environment's spec: the fixed working
/// directory, the run's Daytona name, the lifecycle timers, and Daytona's
/// default snapshot when the environment names no image or Dockerfile. An
/// image or Dockerfile stays as it is: the driver builds it into a cached
/// snapshot sized by the spec's resources. A create from the default
/// snapshot carries no resources, which Daytona refuses on a sandbox
/// created from a snapshot.
pub(crate) fn overlay(spec: DriverSpec, run_id: Option<&RunId>) -> DriverSpec {
let mut spec = spec.working_directory(WORKING_DIRECTORY);
spec.source = SandboxSource::Snapshot {
id: snapshot.clone(),
};
if !matches!(
spec.source,
SandboxSource::Image { .. } | SandboxSource::Dockerfile { .. }
) {
spec.source = SandboxSource::Snapshot {
id: SnapshotId::try_new(DEFAULT_SNAPSHOT).expect("the default snapshot name is valid"),
};
spec.resources = Resources::default();
}
spec.name = run_id.map(|run_id| format!("fabro-{run_id}"));
let mut timers = LifecycleTimers::default();
let mut timers = spec.timers;
// An explicit zero disables auto-stop; the driver encodes
// `Duration::ZERO` as that wire value.
timers.auto_stop_after_idle = Some(options.auto_stop.unwrap_or(DEFAULT_AUTO_STOP));
timers.auto_stop_after_idle = Some(timers.auto_stop_after_idle.unwrap_or(DEFAULT_AUTO_STOP));
// Run sandboxes are never deleted on stop: the run record may need
// them again on resume, and `fabro system prune` reclaims them.
timers.auto_delete_after_stop = Some(Duration::ZERO);
spec.timers(timers)
}
/// Ensures the snapshot `inputs` describe exists and is active, building
/// it when Daytona does not have it. Returns the snapshot to create
/// sandboxes from.
async fn ensure_snapshot(
provider: &dyn SandboxProvider,
api_key: &str,
inputs: &SnapshotInputs<'_>,
events: Option<EventContext>,
) -> crate::Result<(SnapshotId, String)> {
let name = snapshot_identity::snapshot_name(api_key, inputs)?;
let snapshots = provider.snapshots().ok_or_else(|| {
crate::Error::message("The Daytona provider does not expose snapshot management")
})?;
let id = snapshots
.ensure(
&snapshot_spec(&name, inputs),
DAYTONA_SNAPSHOT_ACTIVE_TIMEOUT,
events,
)
.await
.map_err(|error| {
crate::Error::context(format!("Failed to ensure snapshot '{name}'"), error)
})?;
Ok((id, name))
}
fn snapshot_spec(name: &str, inputs: &SnapshotInputs<'_>) -> SnapshotSpec {
let source = match inputs.source {
SnapshotInput::Image(image) => SnapshotSource::Image {
reference: image.to_string(),
},
SnapshotInput::Dockerfile(content) => SnapshotSource::Dockerfile {
content: content.to_string(),
},
};
let mut resources = Resources::default();
resources.cpu_cores = inputs.cpu.and_then(|cpu| u32::try_from(cpu).ok());
resources.memory_mb = inputs
.memory_gb
.and_then(|gb| u64::try_from(gb).ok())
.map(|gb| gb * 1024);
resources.disk_mb = inputs
.disk_gb
.and_then(|gb| u64::try_from(gb).ok())
.map(|gb| gb * 1024);
SnapshotSpec::new(source).name(name).resources(resources)
}
/// Prepares a Daytona create: the snapshot first, then the spec naming it.
pub(crate) struct DaytonaCreatePlan {
provider: Arc<dyn SandboxProvider>,
api_key: String,
base: DriverSpec,
options: SandboxOptions,
run_id: Option<RunId>,
}
/// The create plan for a run on Daytona: `base` is the spec the
/// environment's options built, which the plan completes with the snapshot
/// once it exists.
pub(crate) fn create_plan(
provider: Arc<dyn SandboxProvider>,
api_key: String,
base: DriverSpec,
options: SandboxOptions,
run_id: Option<RunId>,
) -> DaytonaCreatePlan {
DaytonaCreatePlan {
provider,
api_key,
base,
options,
run_id,
}
}
#[async_trait]
impl CreatePlan for DaytonaCreatePlan {
async fn prepare(&self, events: Option<EventContext>) -> crate::Result<PreparedCreate> {
let (snapshot_id, snapshot_name) = match snapshot_inputs(&self.options) {
// The driver finds, activates, builds, or waits for the snapshot
// as needed, and reports that work through `events`.
Some(inputs) => {
ensure_snapshot(self.provider.as_ref(), &self.api_key, &inputs, events).await?
}
None => (
SnapshotId::try_new(DEFAULT_SNAPSHOT).expect("the default snapshot name is valid"),
DEFAULT_SNAPSHOT.to_string(),
),
};
Ok(PreparedCreate {
spec: overlay(
self.base.clone(),
&self.options,
self.run_id.as_ref(),
&snapshot_id,
),
snapshot: Some(snapshot_name),
})
}
}
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use sandbox_driver::NetworkPolicy;
use sandbox_driver::{LifecycleTimers, NetworkPolicy};
use super::*;
use crate::options::base_spec;
fn run_id() -> RunId {
"01HY0000000000000000000000".parse().unwrap()
}
fn dockerfile_inputs(dockerfile: &str) -> SnapshotInputs<'_> {
SnapshotInputs {
source: SnapshotInput::Dockerfile(dockerfile),
cpu: Some(2),
memory_gb: Some(4),
disk_gb: Some(10),
}
}
#[test]
fn snapshot_inputs_come_from_the_image_or_dockerfile_in_whole_gigabytes() {
assert!(snapshot_inputs(&SandboxOptions::default()).is_none());
let options = SandboxOptions {
image: Some("ubuntu:24.04".to_string()),
cpu: Some(2),
memory_bytes: Some(4_000_000_000),
disk_bytes: Some(10_500_000_000),
..SandboxOptions::default()
};
assert_eq!(
snapshot_inputs(&options),
Some(SnapshotInputs {
source: SnapshotInput::Image("ubuntu:24.04"),
cpu: Some(2),
memory_gb: Some(4),
disk_gb: Some(10),
})
);
let options = SandboxOptions {
dockerfile: Some("FROM ubuntu".to_string()),
..SandboxOptions::default()
};
assert_eq!(
snapshot_inputs(&options).map(|inputs| inputs.source),
Some(SnapshotInput::Dockerfile("FROM ubuntu"))
);
}
#[test]
fn overlay_names_the_run_and_carries_fabro_labels_and_timers() {
let options = SandboxOptions {
labels: BTreeMap::from([("team".to_string(), "platform".to_string())]),
network: NetworkPolicy::CidrAllowList {
let mut resources = Resources::default();
resources.cpu_cores = Some(2);
let base = DriverSpec::new(SandboxSource::HostDirectory)
.label("team", "platform")
.network(NetworkPolicy::CidrAllowList {
cidrs: vec!["10.0.0.0/8".to_string()],
},
..SandboxOptions::default()
};
let snapshot = SnapshotId::try_new("snap-1").unwrap();
let spec = overlay(
base_spec(&options, Some(&run_id())),
&options,
Some(&run_id()),
&snapshot,
);
})
.resources(resources);
let spec = overlay(base, Some(&run_id()));
assert!(matches!(&spec.source, SandboxSource::Snapshot { id } if id == &snapshot));
assert!(
matches!(&spec.source, SandboxSource::Snapshot { id } if id.as_str() == DEFAULT_SNAPSHOT),
"a spec without an image comes from Daytona's default snapshot"
);
assert_eq!(
spec.name.as_deref(),
Some("fabro-01HY0000000000000000000000")
@ -519,18 +238,53 @@ mod tests {
"an unset auto-stop gets fabro's explicit default, never Daytona's 15 minutes"
);
assert_eq!(spec.timers.auto_delete_after_stop, Some(Duration::ZERO));
assert_eq!(
spec.resources,
Resources::default(),
"the default snapshot carries the resources; Daytona refuses them on the sandbox"
);
assert!(!spec.ephemeral);
}
#[test]
fn overlay_leaves_an_image_and_its_resources_for_the_driver_to_cache() {
let mut resources = Resources::default();
resources.cpu_cores = Some(2);
resources.memory_mb = Some(4096);
let base = DriverSpec::new(SandboxSource::Image {
reference: "ubuntu:24.04".to_string(),
})
.resources(resources);
let spec = overlay(base, None);
assert!(
matches!(&spec.source, SandboxSource::Image { reference } if reference == "ubuntu:24.04")
);
assert_eq!(
spec.resources, resources,
"the resources size the cached snapshot"
);
assert_eq!(spec.working_directory.as_deref(), Some(WORKING_DIRECTORY));
let dockerfile = overlay(
DriverSpec::new(SandboxSource::Dockerfile {
content: "FROM ubuntu".to_string(),
}),
None,
);
assert!(matches!(
dockerfile.source,
SandboxSource::Dockerfile { .. }
));
}
#[test]
fn overlay_passes_explicit_auto_stop_through_and_zero_disables() {
let snapshot = SnapshotId::try_new(DEFAULT_SNAPSHOT).unwrap();
let options = SandboxOptions {
auto_stop: Some(Duration::from_mins(45)),
network: NetworkPolicy::Block,
..SandboxOptions::default()
};
let explicit = overlay(base_spec(&options, None), &options, None, &snapshot);
let mut timers = LifecycleTimers::default();
timers.auto_stop_after_idle = Some(Duration::from_mins(45));
let base = DriverSpec::new(SandboxSource::HostDirectory)
.network(NetworkPolicy::Block)
.timers(timers);
let explicit = overlay(base, None);
assert_eq!(
explicit.timers.auto_stop_after_idle,
Some(Duration::from_mins(45))
@ -538,164 +292,44 @@ mod tests {
assert!(matches!(explicit.network, NetworkPolicy::Block));
assert!(explicit.name.is_none());
let options = SandboxOptions {
auto_stop: Some(Duration::ZERO),
..SandboxOptions::default()
};
let disabled = overlay(base_spec(&options, None), &options, None, &snapshot);
let mut timers = LifecycleTimers::default();
timers.auto_stop_after_idle = Some(Duration::ZERO);
let disabled = overlay(
DriverSpec::new(SandboxSource::HostDirectory).timers(timers),
None,
);
assert_eq!(disabled.timers.auto_stop_after_idle, Some(Duration::ZERO));
}
#[test]
fn snapshot_spec_maps_sources_and_gigabyte_resources() {
let inputs = SnapshotInputs {
source: SnapshotInput::Image("ubuntu:24.04"),
cpu: Some(2),
memory_gb: Some(4),
disk_gb: Some(10),
};
let spec = snapshot_spec("fabro-x", &inputs);
assert_eq!(spec.name.as_deref(), Some("fabro-x"));
assert!(matches!(
&spec.source,
SnapshotSource::Image { reference } if reference == "ubuntu:24.04"
));
assert_eq!(spec.resources.cpu_cores, Some(2));
assert_eq!(spec.resources.memory_mb, Some(4096));
assert_eq!(spec.resources.disk_mb, Some(10_240));
let dockerfile = snapshot_spec("fabro-y", &SnapshotInputs {
source: SnapshotInput::Dockerfile("FROM ubuntu"),
..inputs
});
assert!(matches!(
&dockerfile.source,
SnapshotSource::Dockerfile { content } if content == "FROM ubuntu"
));
}
#[test]
fn computed_snapshot_identity_is_deterministic_and_keyed() {
let inputs = dockerfile_inputs("FROM ubuntu:24.04\nRUN apt-get update");
let first = snapshot_identity::snapshot_name("dtn_secret", &inputs).unwrap();
let second = snapshot_identity::snapshot_name("dtn_secret", &inputs).unwrap();
let rotated_key = snapshot_identity::snapshot_name("dtn_rotated", &inputs).unwrap();
assert_eq!(first, second);
assert_eq!(first, "fabro-e607185f-c7ab-88c9-bf9d-d70addba9298");
assert_ne!(first, rotated_key);
let uuid = first
.strip_prefix("fabro-")
.and_then(|raw| uuid::Uuid::parse_str(raw).ok())
.expect("snapshot name should be fabro-<uuid>");
assert_eq!(uuid.get_version_num(), 8);
assert_eq!(uuid.get_variant(), uuid::Variant::RFC4122);
}
#[test]
fn computed_snapshot_identity_changes_for_generation_inputs() {
let base = dockerfile_inputs("FROM ubuntu:24.04");
let base_name = snapshot_identity::snapshot_name("dtn_secret", &base).unwrap();
let cases = [
SnapshotInputs {
source: SnapshotInput::Dockerfile("FROM ubuntu:24.04\n# roll cache"),
..base.clone()
},
SnapshotInputs {
cpu: Some(4),
..base.clone()
},
SnapshotInputs {
memory_gb: Some(8),
..base.clone()
},
SnapshotInputs {
disk_gb: Some(20),
..base.clone()
},
];
for changed in cases {
let changed_name = snapshot_identity::snapshot_name("dtn_secret", &changed).unwrap();
assert_ne!(base_name, changed_name);
}
}
#[test]
fn computed_snapshot_identity_excludes_raw_dockerfile_and_key_material() {
let inputs = SnapshotInputs {
source: SnapshotInput::Dockerfile(
"FROM private.example.com/secret-image\nRUN echo raw-secret",
),
cpu: None,
memory_gb: None,
disk_gb: None,
};
let name = snapshot_identity::snapshot_name("dtn_super_secret_key", &inputs).unwrap();
assert!(name.starts_with("fabro-"));
assert!(!name.contains("private.example.com"));
assert!(!name.contains("raw-secret"));
assert!(!name.contains("dtn_super_secret_key"));
}
#[test]
fn computed_snapshot_identity_changes_for_image_reference() {
let inputs = SnapshotInputs {
source: SnapshotInput::Image("ubuntu:24.04"),
cpu: Some(2),
memory_gb: Some(4),
disk_gb: Some(10),
};
let first = snapshot_identity::snapshot_name("dtn_secret", &inputs).unwrap();
let changed = snapshot_identity::snapshot_name("dtn_secret", &SnapshotInputs {
source: SnapshotInput::Image("ubuntu:24.10"),
..inputs
})
.unwrap();
assert_eq!(first, "fabro-5d23a023-d7ff-8d68-b3ca-e6286f4211d9");
assert_ne!(first, changed);
}
#[test]
fn missing_scopes_render_in_documented_order() {
fn missing_scopes_render_as_the_provider_reports_them() {
let check = DaytonaKeyCheck {
missing: ordered_scopes(&[
"write:sandboxes".to_string(),
missing: vec!["write:snapshots".to_string(), "write:sandboxes".to_string()],
required: vec![
"write:snapshots".to_string(),
"manage:secrets".to_string(),
]),
"delete:snapshots".to_string(),
"write:sandboxes".to_string(),
"delete:sandboxes".to_string(),
],
};
assert!(!check.ok());
assert_eq!(
check.missing_display(),
"write:snapshots, write:sandboxes, manage:secrets"
);
assert_eq!(check.missing_display(), "write:snapshots, write:sandboxes");
assert_eq!(
check.missing_message(),
"Daytona API key is missing required scopes: write:snapshots, write:sandboxes, \
manage:secrets. Regenerate the key with all snapshot and sandbox scopes."
"Daytona API key is missing required scopes: write:snapshots, write:sandboxes. \
Regenerate the key with all snapshot and sandbox scopes."
);
assert_eq!(
required_perms_display(),
check.required_display(),
"write:snapshots, delete:snapshots, write:sandboxes, delete:sandboxes"
);
}
#[tokio::test]
async fn credential_probe_reports_configured_timeout() {
let credentials = DaytonaCredentials {
api_key: "dtn_test".to_string(),
// A non-routable address: the probe cannot finish within the budget.
api_url: Some("http://10.255.255.1:1/api".to_string()),
organization_id: None,
target: None,
http_client: None,
};
// A non-routable address: the probe cannot finish within the budget.
let credentials = DaytonaCredentials::new("dtn_test".to_string())
.with_api_url(Some("http://10.255.255.1:1/api".to_string()));
let err = check_daytona_api_key(&credentials, Duration::from_millis(1))
.await
.expect_err("probe should time out");
@ -728,7 +362,7 @@ mod wire_gate {
use super::*;
use crate::driver_sandbox::{LayoutSource, RepoWorkspace, RunSandbox};
use crate::options::base_spec;
use crate::environment::CloneRequest;
#[expect(
clippy::disallowed_methods,
@ -736,15 +370,9 @@ mod wire_gate {
)]
fn live_credentials() -> Option<DaytonaCredentials> {
let api_key = std::env::var(EnvVars::DAYTONA_API_KEY).ok()?;
Some(DaytonaCredentials {
api_key,
api_url: std::env::var(EnvVars::DAYTONA_API_URL)
.or_else(|_| std::env::var(EnvVars::DAYTONA_SERVER_URL))
.ok(),
organization_id: std::env::var(EnvVars::DAYTONA_ORGANIZATION_ID).ok(),
target: None,
http_client: None,
})
Some(DaytonaCredentials::from_api_key(api_key, |name| {
std::env::var(name).ok()
}))
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@ -765,18 +393,16 @@ mod wire_gate {
let workspace = RepoWorkspace::plan(
LayoutSource::Fixed(layout()),
false,
Some("https://github.com/brynary/rack-test"),
None,
None,
None,
Some(100),
&CloneRequest {
origin_url: Some("https://github.com/brynary/rack-test".to_string()),
depth: Some(100),
..CloneRequest::default()
},
None,
)
.expect("clone plan");
let snapshot = SnapshotId::try_new(DEFAULT_SNAPSHOT).expect("snapshot id");
let options = SandboxOptions::default();
let spec = overlay(base_spec(&options, None), &options, None, &snapshot);
// No image: the overlay creates from Daytona's default snapshot.
let spec = overlay(DriverSpec::new(SandboxSource::HostDirectory), None);
let sandbox = RunSandbox::pending(SandboxProviderKind::DAYTONA, remote, spec, workspace);
sandbox
.initialize()
@ -800,8 +426,8 @@ mod wire_gate {
)
.await
.expect("layout check");
assert!(result.is_success(), "{result:?}");
assert!(result.stdout.contains("true"));
assert!(result.success(), "{result:?}");
assert!(result.stdout_lossy().contains("true"));
let layout = sandbox.workspace_layout().expect("layout record");
assert_eq!(
layout.primary_repo_path.as_deref(),
@ -809,6 +435,6 @@ mod wire_gate {
);
};
checks.await;
sandbox.cleanup().await.expect("cleanup");
sandbox.delete().await.expect("cleanup");
}
}

256
lib/components/fabro-sandbox/src/details.rs
View file

@ -1,29 +1,17 @@
use std::collections::BTreeMap;
use anyhow::Result;
use chrono::{DateTime, Utc};
use fabro_types::{
BundledProvider, RunId, RunSandboxInstance, SandboxDetails, SandboxNetwork,
SandboxProviderKind, SandboxResources, SandboxState, SandboxTimestamps,
};
use fabro_types::{RunId, RunSandboxInstance, SandboxDetails};
use crate::driver::ProviderAccess;
use crate::reconnect;
/// Inspect the sandbox identified by `record` and return provider-neutral
/// details for control-plane display.
///
/// `local` always returns a minimal record describing the host; every other
/// provider is described through the sandbox driver.
/// The sandbox identified by `record`, as the run record fabro keeps and
/// the status the sandbox driver reports for it, on every provider.
pub async fn sandbox_details(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
) -> Result<SandboxDetails> {
if record.provider.bundled() == Some(BundledProvider::Local) {
return Ok(local_details(record));
}
let sandbox = reconnect::reconnect_driver_for_run(record, access, run_id, None).await?;
let sandbox = reconnect::reconnect_for_run(record, access, run_id, None).await?;
let status = sandbox.handle()?.describe().await.map_err(|err| {
anyhow::anyhow!(
"Failed to describe {} sandbox '{}': {err}",
@ -31,236 +19,8 @@ pub async fn sandbox_details(
record.runtime.id
)
})?;
Ok(details_from_status(record, &status))
}
fn local_details(record: &RunSandboxInstance) -> SandboxDetails {
SandboxDetails {
sandbox: record.clone(),
state: SandboxState::Running,
native_state: None,
region: None,
web_url: None,
resources: SandboxResources::default(),
network: SandboxNetwork::unknown(),
labels: BTreeMap::new(),
timestamps: SandboxTimestamps::default(),
}
}
/// Projection of a sandbox-driver [`sandbox_driver::SandboxStatus`] into
/// fabro's inventory shape. The driver reports what a provider exposes
/// through its public facets; fields no facet carries (network policy) stay
/// unknown rather than being read from provider SDK types.
pub(crate) fn info_from_status(
kind: &fabro_types::SandboxProviderKind,
status: &sandbox_driver::SandboxStatus,
) -> fabro_types::SandboxInfo {
let fields = fields_from_status(status);
fabro_types::SandboxInfo {
provider: kind.clone(),
id: status.id.to_string(),
display_name: status.name.clone().filter(|name| !name.is_empty()),
state: fields.state,
native_state: fields.native_state,
image: status.source.clone(),
snapshot: None,
region: status.region.clone(),
web_url: status.web_url.clone(),
working_directory: None,
resources: fields.resources,
network: SandboxNetwork::unknown(),
labels: status.labels.clone(),
timestamps: fields.timestamps,
}
}
pub(crate) fn details_from_status(
record: &RunSandboxInstance,
status: &sandbox_driver::SandboxStatus,
) -> SandboxDetails {
let fields = fields_from_status(status);
SandboxDetails {
sandbox: RunSandboxInstance {
image: (record.provider == SandboxProviderKind::DOCKER)
.then(|| status.source.clone())
.flatten()
.or_else(|| record.image.clone()),
snapshot: (record.provider == SandboxProviderKind::DAYTONA)
.then(|| status.source.clone())
.flatten()
.or_else(|| record.snapshot.clone()),
..record.clone()
},
state: fields.state,
native_state: fields.native_state,
region: status.region.clone(),
web_url: status.web_url.clone(),
resources: fields.resources,
network: SandboxNetwork::unknown(),
labels: status.labels.clone(),
timestamps: fields.timestamps,
}
}
struct StatusFields {
state: SandboxState,
native_state: Option<String>,
resources: SandboxResources,
timestamps: SandboxTimestamps,
}
fn fields_from_status(status: &sandbox_driver::SandboxStatus) -> StatusFields {
StatusFields {
state: normalize_driver_state(status.state),
native_state: Some(status.provider_state.clone()).filter(|value| !value.is_empty()),
resources: status
.resources
.as_ref()
.map(|resources| SandboxResources {
cpu_cores: resources.cpu_cores.map(f64::from),
memory_bytes: resources.memory_mb.map(|mb| mb * 1024 * 1024),
disk_bytes: resources.disk_mb.map(|mb| mb * 1024 * 1024),
})
.unwrap_or_default(),
timestamps: SandboxTimestamps {
created_at: status.created_at.map(DateTime::<Utc>::from),
last_activity_at: status.updated_at.map(DateTime::<Utc>::from),
},
}
}
pub(crate) fn normalize_driver_state(state: sandbox_driver::SandboxState) -> SandboxState {
use sandbox_driver::SandboxState as Driver;
match state {
Driver::Creating | Driver::Forking => SandboxState::Provisioning,
Driver::Starting | Driver::Resuming => SandboxState::Starting,
// A sandbox mid-snapshot keeps serving commands.
Driver::Running | Driver::Snapshotting => SandboxState::Running,
Driver::Stopping | Driver::Archiving => SandboxState::Stopping,
Driver::Stopped => SandboxState::Stopped,
Driver::Pausing | Driver::Paused => SandboxState::Paused,
Driver::Archived => SandboxState::Archived,
Driver::Restoring => SandboxState::Restoring,
Driver::Resizing => SandboxState::Resizing,
Driver::Deleting => SandboxState::Deleting,
Driver::Deleted => SandboxState::Deleted,
Driver::Error => SandboxState::Error,
_ => SandboxState::Unknown,
}
}
#[cfg(test)]
mod tests {
use sandbox_driver::SandboxId;
use super::*;
#[test]
fn driver_states_map_onto_fabro_states() {
use sandbox_driver::SandboxState as Driver;
for (driver, fabro) in [
(Driver::Creating, SandboxState::Provisioning),
(Driver::Starting, SandboxState::Starting),
(Driver::Running, SandboxState::Running),
(Driver::Snapshotting, SandboxState::Running),
(Driver::Stopping, SandboxState::Stopping),
(Driver::Stopped, SandboxState::Stopped),
(Driver::Paused, SandboxState::Paused),
(Driver::Archived, SandboxState::Archived),
(Driver::Deleting, SandboxState::Deleting),
(Driver::Deleted, SandboxState::Deleted),
(Driver::Error, SandboxState::Error),
(Driver::Unknown, SandboxState::Unknown),
] {
assert_eq!(normalize_driver_state(driver), fabro, "{driver:?}");
}
}
#[test]
fn status_projection_carries_identity_source_and_labels() {
let mut status = sandbox_driver::SandboxStatus::new(
SandboxId::try_new("container-abc123").unwrap(),
sandbox_driver::SandboxState::Running,
);
status.name = Some("fabro-run-abc".to_string());
status.provider_state = "running".to_string();
status.source = Some("buildpack-deps:noble".to_string());
status
.labels
.insert("sh.fabro.managed".to_string(), "true".to_string());
let mut resources = sandbox_driver::Resources::default();
resources.cpu_cores = Some(2);
resources.memory_mb = Some(2048);
status.resources = Some(resources);
let info = info_from_status(&SandboxProviderKind::DOCKER, &status);
assert_eq!(info.id, "container-abc123");
assert_eq!(info.display_name.as_deref(), Some("fabro-run-abc"));
assert_eq!(info.state, SandboxState::Running);
assert_eq!(info.native_state.as_deref(), Some("running"));
assert_eq!(info.image.as_deref(), Some("buildpack-deps:noble"));
assert_eq!(info.resources.cpu_cores, Some(2.0));
assert_eq!(info.resources.memory_bytes, Some(2_147_483_648));
assert_eq!(
info.labels.get("sh.fabro.managed").map(String::as_str),
Some("true")
);
let record = RunSandboxInstance {
provider: SandboxProviderKind::DOCKER,
image: None,
snapshot: None,
runtime: fabro_types::RunSandboxRuntime {
id: "container-abc123".to_string(),
working_directory: "/workspace".to_string(),
repo_cloned: Some(true),
clone_origin_url: None,
clone_branch: None,
workspace_root: None,
repos_root: None,
primary_repo_path: None,
primary_repo_link: None,
},
};
let details = details_from_status(&record, &status);
assert_eq!(
details.sandbox.image.as_deref(),
Some("buildpack-deps:noble")
);
assert_eq!(details.sandbox.runtime.id, "container-abc123");
assert_eq!(details.network, SandboxNetwork::unknown());
}
#[test]
fn local_details_returns_running_with_no_metadata() {
let record = RunSandboxInstance {
provider: SandboxProviderKind::LOCAL,
image: None,
snapshot: None,
runtime: fabro_types::RunSandboxRuntime {
id: "local:01JNQVR7M0EJ5GKAT2SC4ERS1Z".to_string(),
working_directory: "/Users/client/project".to_string(),
repo_cloned: None,
clone_origin_url: None,
clone_branch: None,
workspace_root: None,
repos_root: None,
primary_repo_path: None,
primary_repo_link: None,
},
};
let details = local_details(&record);
assert_eq!(details.sandbox.provider, SandboxProviderKind::LOCAL);
assert_eq!(details.state, SandboxState::Running);
let runtime = &details.sandbox.runtime;
assert_eq!(runtime.id, "local:01JNQVR7M0EJ5GKAT2SC4ERS1Z");
assert_eq!(runtime.working_directory, "/Users/client/project");
assert!(details.region.is_none());
assert!(details.sandbox.image.is_none());
assert!(details.labels.is_empty());
assert_eq!(details.resources, SandboxResources::default());
assert_eq!(details.network, SandboxNetwork::unknown());
assert_eq!(details.timestamps, SandboxTimestamps::default());
}
Ok(SandboxDetails {
sandbox: record.clone(),
status,
})
}

87
lib/components/fabro-sandbox/src/docker.rs
View file

@ -8,12 +8,11 @@
//! [`REPOS_ROOT`] and is linked into the workspace, so the run works in
//! `/workspace/<repo>`.
use sandbox_driver::{HealthStatus, SandboxSource, SandboxSpec as DriverSpec};
use sandbox_driver::{HealthStatus, LifecycleTimers, SandboxSource, SandboxSpec as DriverSpec};
use sandbox_driver_docker_config::DockerProviderConfig;
use crate::driver::ProviderAccess;
use crate::driver_sandbox::WorkspaceLayout;
use crate::options::SandboxOptions;
use crate::provider_sandbox;
pub const WORKING_DIRECTORY: &str = "/workspace";
@ -30,28 +29,28 @@ pub(crate) fn layout() -> WorkspaceLayout {
}
/// The image a Docker sandbox runs: the environment's, or the default.
pub(crate) fn effective_image(options: &SandboxOptions) -> String {
options
.image
.clone()
.unwrap_or_else(|| DEFAULT_IMAGE.to_string())
pub(crate) fn effective_image(spec: &DriverSpec) -> String {
match &spec.source {
SandboxSource::Image { reference } => reference.clone(),
_ => DEFAULT_IMAGE.to_string(),
}
}
/// Docker's additions to the base spec, and the image it will run.
pub(crate) fn overlay(spec: DriverSpec, options: &SandboxOptions) -> (DriverSpec, String) {
let image = effective_image(options);
/// Docker's additions to the environment's spec: the image it will run,
/// the fixed working directory, and a pull for a missing image. Docker has
/// no lifecycle timers, so the environment's auto-stop does not apply.
pub(crate) fn overlay(spec: DriverSpec) -> DriverSpec {
let image = effective_image(&spec);
let mut spec = spec;
spec.source = SandboxSource::Image {
reference: image.clone(),
};
let spec = spec.working_directory(WORKING_DIRECTORY).provider_config(
spec.source = SandboxSource::Image { reference: image };
spec.timers = LifecycleTimers::default();
spec.working_directory(WORKING_DIRECTORY).provider_config(
DockerProviderConfig {
auto_pull: true,
..DockerProviderConfig::default()
}
.into_value(),
);
(spec, image)
)
}
/// Whether the Docker daemon answers. Used by `fabro doctor`.
@ -76,57 +75,49 @@ pub async fn check_docker_daemon() -> crate::Result<()> {
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use std::time::Duration;
use fabro_types::RunId;
use sandbox_driver::NetworkPolicy;
use super::*;
use crate::options::base_spec;
#[test]
fn overlay_fixes_the_workspace_and_pulls_the_named_image() {
let run_id: RunId = "01HY0000000000000000000000".parse().unwrap();
let options = SandboxOptions {
image: Some("ghcr.io/acme/dev:1".to_string()),
env: BTreeMap::from([("FOO".to_string(), "bar".to_string())]),
memory_bytes: Some(4_000_000_000),
cpu: Some(2),
network: NetworkPolicy::Block,
..SandboxOptions::default()
};
let (spec, image) = overlay(base_spec(&options, Some(&run_id)), &options);
assert_eq!(image, "ghcr.io/acme/dev:1");
let mut requested = LifecycleTimers::default();
requested.auto_stop_after_idle = Some(Duration::from_mins(45));
let spec = overlay(
DriverSpec::new(SandboxSource::Image {
reference: "ubuntu:24.04".to_string(),
})
.network(NetworkPolicy::Block)
.timers(requested),
);
assert!(matches!(
&spec.source,
SandboxSource::Image { reference } if reference == "ghcr.io/acme/dev:1"
SandboxSource::Image { reference } if reference == "ubuntu:24.04"
));
assert_eq!(
spec.name.as_deref(),
Some("fabro-run-01HY0000000000000000000000")
);
assert_eq!(spec.working_directory.as_deref(), Some(WORKING_DIRECTORY));
assert!(
!spec.labels.contains_key("sh.fabro.managed"),
"ownership labels come from the scope the provider is connected through"
);
assert_eq!(spec.env.get("FOO").map(String::as_str), Some("bar"));
assert_eq!(spec.resources.cpu_cores, Some(2));
assert_eq!(spec.resources.memory_mb, Some(3815));
assert!(matches!(spec.network, NetworkPolicy::Block));
assert_eq!(spec.provider_config["auto_pull"], true);
assert_eq!(
spec.timers,
LifecycleTimers::default(),
"docker has no timers to honor the environment's auto-stop with"
);
let config: DockerProviderConfig =
serde_json::from_value(spec.provider_config).expect("docker provider config");
assert!(config.auto_pull);
}
#[test]
fn overlay_supplies_the_default_image_when_the_environment_names_none() {
let options = SandboxOptions::default();
let (spec, image) = overlay(base_spec(&options, None), &options);
assert_eq!(image, DEFAULT_IMAGE);
let spec = overlay(DriverSpec::new(SandboxSource::HostDirectory));
assert!(matches!(
&spec.source,
SandboxSource::Image { reference } if reference == DEFAULT_IMAGE
));
assert!(spec.name.is_none());
assert_eq!(
effective_image(&DriverSpec::new(SandboxSource::HostDirectory)),
DEFAULT_IMAGE
);
}
}

227
lib/components/fabro-sandbox/src/driver.rs
View file

@ -16,16 +16,12 @@ use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use fabro_static::EnvVars;
use fabro_types::settings::server::{
SandboxPluginSettings, ServerSandboxProviderSettings, ServerSandboxProvidersSettings,
};
use fabro_types::{BundledProvider, SandboxProviderKind};
use sandbox_driver::{
Capabilities, EventContext, ProviderHealth, ProviderKind, Sandbox, SandboxFilter, SandboxId,
SandboxProvider, SandboxSpec, SandboxStatus, SnapshotProvider, VolumeProvider,
};
use sandbox_driver::{ProviderKind, SandboxProvider};
use sandbox_driver_daytona::{DaytonaConfig, DaytonaProvider};
use sandbox_driver_docker::DockerProvider;
use sandbox_driver_host::HostProvider;
@ -38,39 +34,74 @@ pub const PLUGIN_BINARY_PREFIX: &str = "fabro-sandbox";
/// `User-Agent` fabro presents to remote sandbox control planes.
pub const USER_AGENT: &str = concat!("fabro-sandbox/", env!("CARGO_PKG_VERSION"));
/// Explicit Daytona credentials. The process environment is never consulted.
/// Explicit Daytona credentials: the SDK's configuration with the API key
/// always present and a `Debug` that never prints it. The process
/// environment is never consulted.
#[derive(Clone)]
pub struct DaytonaCredentials {
pub api_key: String,
pub api_url: Option<String>,
pub organization_id: Option<String>,
pub target: Option<String>,
/// Shared HTTP client; tests pass a no-proxy client here.
pub http_client: Option<reqwest::Client>,
}
pub struct DaytonaCredentials(DaytonaConfig);
impl DaytonaCredentials {
/// Credentials for `api_key` against Daytona's public control plane,
/// presenting fabro's `User-Agent`.
#[must_use]
pub fn new(api_key: String) -> Self {
Self(DaytonaConfig {
api_key: Some(api_key),
user_agent: Some(USER_AGENT.to_string()),
..DaytonaConfig::default()
})
}
/// Credentials for a vault API key, with the control-plane URL and
/// organization taken from `lookup` (server configuration, or the
/// process environment in a CLI worker). Nothing is read implicitly.
pub fn from_api_key(api_key: String, lookup: impl Fn(&str) -> Option<String>) -> Self {
Self {
api_key,
api_url: lookup(EnvVars::DAYTONA_API_URL)
.or_else(|| lookup(EnvVars::DAYTONA_SERVER_URL)),
organization_id: lookup(EnvVars::DAYTONA_ORGANIZATION_ID),
target: None,
http_client: None,
}
Self::new(api_key)
.with_api_url(
lookup(EnvVars::DAYTONA_API_URL).or_else(|| lookup(EnvVars::DAYTONA_SERVER_URL)),
)
.with_organization_id(lookup(EnvVars::DAYTONA_ORGANIZATION_ID))
}
/// The control-plane URL; Daytona's public API when `None`.
#[must_use]
pub fn with_api_url(mut self, api_url: Option<String>) -> Self {
self.0.api_url = api_url;
self
}
#[must_use]
pub fn with_organization_id(mut self, organization_id: Option<String>) -> Self {
self.0.organization_id = organization_id;
self
}
/// A shared HTTP client; tests pass a no-proxy client here.
#[must_use]
pub fn with_http_client(mut self, http_client: Option<reqwest::Client>) -> Self {
self.0.http_client = http_client;
self
}
/// The API key, which every constructor sets.
#[must_use]
pub fn api_key(&self) -> &str {
self.0.api_key.as_deref().unwrap_or_default()
}
/// The SDK configuration the driver's Daytona provider connects with.
#[must_use]
pub fn config(&self) -> &DaytonaConfig {
&self.0
}
}
impl std::fmt::Debug for DaytonaCredentials {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DaytonaCredentials")
.field("api_url", &self.api_url)
.field("organization_id", &self.organization_id)
.field("target", &self.target)
.field("api_url", &self.0.api_url)
.field("organization_id", &self.0.organization_id)
.field("target", &self.0.target)
.finish_non_exhaustive()
}
}
@ -155,8 +186,10 @@ pub enum ConnectError {
/// Connects the provider behind `kind`.
///
/// Bundled kinds return the in-process driver provider. Any other kind
/// launches the plugin named by `settings.plugin` and returns a supervised
/// handle that relaunches it after a crash for new work only. The
/// launches the plugin named by `settings.plugin` and returns the driver's
/// supervisor, which relaunches the executable after a crash for new work
/// only; handles from an earlier generation stay bound to it, and callers
/// rebuild them through `attach` with the persisted sandbox id. The
/// configured kind is fabro's name for whatever the executable serves; the
/// kind the plugin declares is not compared against it. Disabled entries
/// are refused here so no caller has to remember the policy check.
@ -187,17 +220,8 @@ pub async fn connect_provider(
.daytona
.as_ref()
.ok_or(ConnectError::MissingDaytonaCredentials)?;
let config = DaytonaConfig {
api_key: Some(credentials.api_key.clone()),
jwt_token: None,
organization_id: credentials.organization_id.clone(),
api_url: credentials.api_url.clone(),
target: credentials.target.clone(),
http_client: credentials.http_client.clone(),
user_agent: Some(USER_AGENT.to_string()),
};
Arc::new(
DaytonaProvider::connect_explicit(config)
DaytonaProvider::connect_explicit(credentials.config().clone())
.await
.map_err(driver)?,
)
@ -207,7 +231,20 @@ pub async fn connect_provider(
.plugin
.as_ref()
.ok_or_else(|| ConnectError::MissingPluginSettings { kind: kind.clone() })?;
Arc::new(PluginBackedProvider::launch(kind, plugin).await?)
let driver_kind = ProviderKind::try_new(kind.as_str()).map_err(|source| {
ConnectError::InvalidKind {
kind: kind.clone(),
source,
}
})?;
// The supervisor is the provider: it launches the executable now,
// so a misconfigured plugin fails at connect time, and relaunches
// it after a crash for new work only.
Arc::new(
PluginSupervisor::launch(PLUGIN_BINARY_PREFIX, plugin_config(driver_kind, plugin))
.await
.map_err(driver)?,
)
}
};
Ok(ConnectedProvider {
@ -216,57 +253,6 @@ pub async fn connect_provider(
})
}
/// A plugin provider that survives its executable crashing.
///
/// Wraps a [`PluginSupervisor`]: every call obtains the current plugin
/// generation, and a closed transport is replaced with a fresh launch before
/// the call. A failed call is never replayed, and handles obtained from an
/// earlier generation stay bound to it; callers rebuild them through
/// [`SandboxProvider::attach`] with the persisted sandbox id.
pub struct PluginBackedProvider {
kind: ProviderKind,
capabilities: Capabilities,
supervisor: PluginSupervisor,
}
impl PluginBackedProvider {
async fn launch(
kind: &SandboxProviderKind,
settings: &SandboxPluginSettings,
) -> Result<Self, ConnectError> {
let driver_kind =
ProviderKind::try_new(kind.as_str()).map_err(|source| ConnectError::InvalidKind {
kind: kind.clone(),
source,
})?;
let supervisor = PluginSupervisor::new(
PLUGIN_BINARY_PREFIX,
plugin_config(driver_kind.clone(), settings),
);
// Launch once now so a misconfigured plugin fails at connect time and
// the declared capabilities are known for preflight.
let capabilities = supervisor
.current()
.await
.map_err(|source| ConnectError::Driver {
kind: kind.clone(),
source,
})?
.capabilities()
.clone();
Ok(Self {
kind: driver_kind,
capabilities,
supervisor,
})
}
/// Asks the current plugin generation to exit and reaps it.
pub async fn shutdown(&self) -> sandbox_driver::Result<()> {
self.supervisor.shutdown().await
}
}
fn plugin_config(kind: ProviderKind, settings: &SandboxPluginSettings) -> PluginConfig {
PluginConfig {
kind,
@ -283,69 +269,6 @@ fn plugin_config(kind: ProviderKind, settings: &SandboxPluginSettings) -> Plugin
}
}
#[async_trait]
impl SandboxProvider for PluginBackedProvider {
fn kind(&self) -> &ProviderKind {
&self.kind
}
fn capabilities(&self) -> &Capabilities {
&self.capabilities
}
async fn create(
&self,
spec: &SandboxSpec,
events: Option<EventContext>,
) -> sandbox_driver::Result<Arc<dyn Sandbox>> {
self.supervisor.current().await?.create(spec, events).await
}
async fn attach(
&self,
id: &SandboxId,
events: Option<EventContext>,
) -> sandbox_driver::Result<Arc<dyn Sandbox>> {
self.supervisor.current().await?.attach(id, events).await
}
async fn undelete(
&self,
id: &SandboxId,
events: Option<EventContext>,
) -> sandbox_driver::Result<Arc<dyn Sandbox>> {
self.supervisor.current().await?.undelete(id, events).await
}
async fn delete(
&self,
id: &SandboxId,
events: Option<EventContext>,
) -> sandbox_driver::Result<()> {
self.supervisor.current().await?.delete(id, events).await
}
async fn list(&self, filter: &SandboxFilter) -> sandbox_driver::Result<Vec<SandboxStatus>> {
self.supervisor.current().await?.list(filter).await
}
async fn health(&self) -> sandbox_driver::Result<ProviderHealth> {
self.supervisor.current().await?.health().await
}
/// Snapshot and volume management cross the wire per plugin generation,
/// which these borrowing accessors cannot express. Fabro drives
/// snapshots on the bundled Daytona provider only, so a plugin reports
/// none until a generation-aware accessor exists.
fn snapshots(&self) -> Option<&dyn SnapshotProvider> {
None
}
fn volumes(&self) -> Option<&dyn VolumeProvider> {
None
}
}
#[cfg(test)]
mod tests {
use super::*;

544
lib/components/fabro-sandbox/src/driver_sandbox.rs
View file

@ -6,76 +6,36 @@
//! through the handle itself. Nothing here knows which provider is behind
//! the handle or whether it runs in-process or over the plugin wire.
//!
//! What stays fabro's: the exec ladder, the credential filter on explicit
//! environment variables, and the run-facing conventions (`platform` names,
//! grep line format, walk results relative to a caller-declared base). The
//! driver reports lifecycle events itself, through the [`EventContext`] a
//! sandbox is created or attached with.
//! What stays fabro's: the exec ladder and the run-facing conventions
//! (`platform` names, grep line format, walk results relative to a
//! caller-declared base). The driver reports lifecycle events itself,
//! through the [`EventContext`] a sandbox is created or attached with.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::path::Path;
use std::sync::{Arc, OnceLock};
use std::time::{Duration, Instant};
use async_trait::async_trait;
use fabro_github::GitHubCredentials;
use fabro_github::token_source::InstallationTokenSource;
use fabro_github::token_source::{InstallationTokenSource, TokenSnapshot};
use fabro_types::SandboxProviderKind;
use fabro_util::workspace_glob::WorkspaceGlob;
use sandbox_driver::{
Capability, DirEntry, EventContext, FileKind, GrepMatch, GrepOptions, LifecycleTimers,
PreviewUrl, PreviewUrls, PtyOptions, PtySize, Sandbox as DriverHandle,
SandboxProvider as DriverProvider, SandboxSource, SandboxSpec as DriverSpec, SandboxState,
Search as _, WaitOptions, WalkOptions,
Capability, DirEntry, EventContext, ExecControls, ExecResult, ExecSpec, ExecStreamingResult,
FileKind, GitRetryPolicy, GrepMatch, GrepOptions, PreviewUrl, PreviewUrls, PtyOptions,
PtySession, PtySize, Sandbox as DriverHandle, SandboxProvider as DriverProvider,
SandboxSpec as DriverSpec, SandboxState, Search as _, StdioProcess, WaitOptions, WalkOptions,
};
use sandbox_driver_host::HostProvider;
use tokio::fs;
use tokio::sync::OnceCell;
use tokio_util::sync::CancellationToken;
use crate::clone::{self, GitHubClone};
use crate::clone_source::{self, CloneDecision, EmptyWorkspaceReason};
use crate::push_credentials::{self, PushCredentialState};
use crate::terminal::{DriverTerminalSession, TerminalSize};
use crate::{GitRunInfo, GitSetupIntent, RefreshOutcome, RetryPlan};
/// A sandbox on the worker host at `working_directory`, the fabro `local`
/// kind, served by the driver's in-process Host provider.
///
/// The directory is designated: the sandbox uses it in place and never
/// removes it. It is created when missing so a run can point at a fresh
/// scratch path. The registry lives in a per-process temporary root, so a
/// later process rebuilds the handle by calling this again with the
/// persisted working directory rather than by id.
pub async fn local_sandbox(working_directory: impl Into<PathBuf>) -> crate::Result<RunSandbox> {
local_sandbox_with_events(working_directory, None).await
}
/// [`local_sandbox`] whose driver lifecycle events reach `events`.
pub async fn local_sandbox_with_events(
working_directory: impl Into<PathBuf>,
events: Option<EventContext>,
) -> crate::Result<RunSandbox> {
let working_directory: PathBuf = working_directory.into();
fs::create_dir_all(&working_directory)
.await
.map_err(|error| crate::Error::context("Failed to create working directory", error))?;
let provider = HostProvider::new();
let spec = DriverSpec::new(SandboxSource::HostDirectory)
.working_directory(working_directory.display().to_string());
let handle = provider
.create(&spec, events)
.await
.map_err(|error| crate::Error::context("Failed to create local sandbox", error))?;
let sandbox = RunSandbox::new(SandboxProviderKind::LOCAL, handle);
sandbox.learn_platform().await?;
Ok(sandbox)
}
use crate::exec::{ExplicitEnvPolicy, SandboxExec};
use crate::sandbox::{
self, ExecResult, ExecStreamingRequest, ExecStreamingResult, PushError, PushReport,
SandboxFile, SandboxWorkspaceLayout, StdioProcess,
};
use crate::credentials::{self, RepoCredentials};
use crate::environment::CloneRequest;
use crate::exec::SandboxExec;
use crate::sandbox::{self, PushError, PushReport, SandboxFile, SandboxWorkspaceLayout};
use crate::{GitRunInfo, GitSetupIntent};
/// Where a clone-based provider puts its files: the run works under
/// `workspace_root`, and repositories check out under `repos_root`.
@ -116,12 +76,14 @@ enum WorkspacePlan {
Attached,
}
/// Fabro's clone-based workspace on an isolated sandbox: the layout, the
/// clone it performs, and the GitHub credentials its checkout carries.
/// The run's workspace on a sandbox: the layout, the clone fabro performs
/// into it (if any), and the GitHub credentials its checkout carries. A
/// workspace fabro did not clone into is still a checkout the run may push
/// from, with whatever credentials the checkout carries itself.
pub(crate) struct RepoWorkspace {
layout: OnceLock<WorkspaceLayout>,
plan: WorkspacePlan,
credentials: PushCredentialState,
credentials: RepoCredentials,
repo_cloned: OnceLock<bool>,
origin_url: OnceLock<String>,
/// The directory the run works in once known: the repository link for a
@ -135,31 +97,22 @@ pub(crate) struct RepoWorkspace {
impl RepoWorkspace {
/// Decide the clone for a new sandbox. Fails before any provider call
/// when the selectors are inconsistent (a pin without a branch, a
/// non-GitHub origin without `skip_clone`).
#[expect(
clippy::too_many_arguments,
reason = "the clone selectors are validated together by decide_clone"
)]
/// non-GitHub origin without `skip`).
pub(crate) fn plan(
layout: LayoutSource,
skip_clone: bool,
clone_origin_url: Option<&str>,
clone_branch: Option<&str>,
clone_tag: Option<&str>,
clone_commit_sha: Option<&str>,
clone_depth: Option<u32>,
clone: &CloneRequest,
github_app: Option<&GitHubCredentials>,
) -> crate::Result<Self> {
let decision = clone_source::decide_clone(
skip_clone,
clone_origin_url,
clone_branch,
clone_tag,
clone_commit_sha,
clone.skip,
clone.origin_url.as_deref(),
clone.branch.as_deref(),
clone.tag.as_deref(),
clone.commit_sha.as_deref(),
)?;
let credentials = PushCredentialState::new(push_credentials::build_token_source(
let credentials = RepoCredentials::new(credentials::build_token_source(
github_app,
clone_origin_url,
clone.origin_url.as_deref(),
)?);
let plan = match decision {
CloneDecision::EmptyWorkspace { reason } => WorkspacePlan::Empty(reason),
@ -173,7 +126,7 @@ impl RepoWorkspace {
branch,
tag,
commit_sha,
depth: clone_depth,
depth: clone.depth,
}),
};
Ok(Self {
@ -189,7 +142,7 @@ impl RepoWorkspace {
/// A workspace prepared by an earlier process, described by the run
/// record. Pushes from a reattached sandbox use whatever credentials the
/// checkout's `origin` already carries.
/// checkout's credential store already carries.
pub(crate) fn attached(
layout: LayoutSource,
repo_cloned: bool,
@ -199,7 +152,7 @@ impl RepoWorkspace {
let workspace = Self {
layout: layout.into_cell(),
plan: WorkspacePlan::Attached,
credentials: PushCredentialState::new(None),
credentials: RepoCredentials::none(),
repo_cloned: OnceLock::new(),
origin_url: OnceLock::new(),
execution_directory: OnceLock::new(),
@ -216,6 +169,21 @@ impl RepoWorkspace {
workspace
}
/// The workspace an existing handle already works in, whatever it
/// holds: nothing fabro cloned, laid out from the handle's own working
/// directory.
pub(crate) fn existing() -> Self {
Self {
layout: LayoutSource::ProviderWorkingDirectory.into_cell(),
plan: WorkspacePlan::Attached,
credentials: RepoCredentials::none(),
repo_cloned: OnceLock::new(),
origin_url: OnceLock::new(),
execution_directory: OnceLock::new(),
checkout_path: OnceLock::new(),
}
}
/// Settle a provider-dependent layout from the sandbox's working
/// directory. A fixed layout is left alone.
fn resolve_layout(&self, provider_working_directory: &str) -> &WorkspaceLayout {
@ -285,69 +253,38 @@ impl LayoutSource {
}
}
/// What a create needs once its inputs are settled.
#[derive(Clone)]
pub(crate) struct PreparedCreate {
pub(crate) spec: DriverSpec,
/// The provider snapshot the sandbox is created from, when the provider
/// has that concept; recorded on the run.
pub(crate) snapshot: Option<String>,
}
/// Settles a create's inputs right before the provider call. A plan may
/// build provider resources first (a Daytona snapshot); the driver reports
/// that work through `events`.
#[async_trait]
pub(crate) trait CreatePlan: Send + Sync {
async fn prepare(&self, events: Option<EventContext>) -> crate::Result<PreparedCreate>;
}
/// A create whose spec is known up front.
struct SpecPlan(PreparedCreate);
#[async_trait]
impl CreatePlan for SpecPlan {
async fn prepare(&self, _events: Option<EventContext>) -> crate::Result<PreparedCreate> {
Ok(self.0.clone())
}
}
/// A sandbox that does not exist yet: `initialize` creates it on the
/// provider from the plan's spec.
/// provider from `spec`.
struct PendingCreate {
provider: Arc<dyn DriverProvider>,
plan: Box<dyn CreatePlan>,
spec: DriverSpec,
}
/// A fabro sandbox backed by a sandbox-driver handle.
pub struct RunSandbox {
kind: SandboxProviderKind,
kind: SandboxProviderKind,
/// Set at construction for an existing sandbox, at `initialize` for a
/// pending one.
handle: OnceCell<Arc<dyn DriverHandle>>,
pending: Option<PendingCreate>,
workspace: Option<RepoWorkspace>,
env_policy: ExplicitEnvPolicy,
handle: OnceCell<Arc<dyn DriverHandle>>,
pending: Option<PendingCreate>,
workspace: RepoWorkspace,
/// Where the driver reports the lifecycle of a sandbox this creates.
/// Set before `initialize` on a pending sandbox; an existing handle
/// already carries the context it was created or attached with.
events: Option<EventContext>,
events: Option<EventContext>,
/// `(platform, os_version)` learned from the sandbox at initialize or
/// start; unknown until then.
platform: OnceLock<(String, String)>,
platform: OnceLock<(String, String)>,
/// The provider snapshot the sandbox was created from, when known.
snapshot: OnceLock<String>,
snapshot: OnceLock<String>,
}
impl RunSandbox {
/// Wraps a driver handle. `local` runs on the worker host, so explicit
/// environment variables pass the credential filter; every other kind
/// is isolated and takes the caller's environment as composed.
/// Wraps an existing driver handle as a sandbox of `kind`, working in
/// whatever the handle's working directory holds.
#[must_use]
pub fn new(kind: SandboxProviderKind, handle: Arc<dyn DriverHandle>) -> Self {
let sandbox = Self::empty(kind);
let _ = sandbox.handle.set(handle);
sandbox
Self::attached(kind, handle, RepoWorkspace::existing())
}
/// A sandbox over an existing handle whose platform is already known,
@ -372,28 +309,8 @@ impl RunSandbox {
spec: DriverSpec,
workspace: RepoWorkspace,
) -> Self {
Self::pending_with_plan(
kind,
provider,
Box::new(SpecPlan(PreparedCreate {
spec,
snapshot: None,
})),
workspace,
)
}
/// A sandbox `initialize` will create on `provider` once `plan` has
/// settled its spec, then prepare per `workspace`.
pub(crate) fn pending_with_plan(
kind: SandboxProviderKind,
provider: Arc<dyn DriverProvider>,
plan: Box<dyn CreatePlan>,
workspace: RepoWorkspace,
) -> Self {
let mut sandbox = Self::empty(kind);
sandbox.pending = Some(PendingCreate { provider, plan });
sandbox.workspace = Some(workspace);
let mut sandbox = Self::empty(kind, workspace);
sandbox.pending = Some(PendingCreate { provider, spec });
sandbox
}
@ -410,23 +327,17 @@ impl RunSandbox {
workspace: RepoWorkspace,
) -> Self {
workspace.resolve_layout(handle.working_directory());
let mut sandbox = Self::new(kind, handle);
sandbox.workspace = Some(workspace);
let sandbox = Self::empty(kind, workspace);
let _ = sandbox.handle.set(handle);
sandbox
}
fn empty(kind: SandboxProviderKind) -> Self {
let env_policy = if kind.is_local() {
ExplicitEnvPolicy::FilterSensitive
} else {
ExplicitEnvPolicy::TrustCaller
};
fn empty(kind: SandboxProviderKind, workspace: RepoWorkspace) -> Self {
Self {
kind,
handle: OnceCell::new(),
pending: None,
workspace: None,
env_policy,
workspace,
events: None,
platform: OnceLock::new(),
snapshot: OnceLock::new(),
@ -461,11 +372,9 @@ impl RunSandbox {
/// Fabro's exec policy over the driver's exec facet, working in the
/// run's directory. Absent until a pending sandbox is initialized.
pub fn exec(&self) -> crate::Result<SandboxExec<'_>> {
let mut exec = SandboxExec::new(self.handle()?.exec(), self.env_policy);
if let Some(workspace) = &self.workspace {
if let Some(dir) = workspace.execution_directory.get() {
exec = exec.with_working_dir(dir.clone());
}
let mut exec = SandboxExec::new(self.handle()?.exec());
if let Some(dir) = self.workspace.execution_directory.get() {
exec = exec.with_working_dir(dir.clone());
}
Ok(exec)
}
@ -474,19 +383,17 @@ impl RunSandbox {
/// resolves relative paths against the sandbox's own working directory,
/// which sits above a cloned repository's link.
fn resolve(&self, path: &str) -> String {
match self
.workspace
.as_ref()
.and_then(|workspace| workspace.execution_directory.get())
{
match self.workspace.execution_directory.get() {
Some(working_directory) => sandbox::resolve_path(path, working_directory),
None => path.to_string(),
}
}
/// The driver's git facet for this sandbox's checkout. Absent until a
/// pending sandbox is initialized, or when the provider has no git.
pub(crate) fn git(&self) -> crate::Result<sandbox_driver::GitFacet<'_>> {
/// The driver's git facet for this sandbox's checkout, for fabro's own
/// git operations (checkpoints, diffs, the Run Files listing). Absent
/// until a pending sandbox is initialized, or when the provider has no
/// git. Pass [`Self::working_directory`] as the repository path.
pub fn git(&self) -> crate::Result<sandbox_driver::GitFacet<'_>> {
self.handle()?.git().ok_or_else(|| {
crate::Error::message(format!(
"sandbox provider `{}` does not support git",
@ -495,6 +402,20 @@ impl RunSandbox {
})
}
/// The driver's services facet for this sandbox: background processes
/// that outlive their exec (the agent's MCP servers, dev servers), the
/// wait for a port to answer, and the list of listeners. Absent until a
/// pending sandbox is initialized, or when the provider has no
/// services.
pub fn services(&self) -> crate::Result<sandbox_driver::ServicesFacet<'_>> {
self.handle()?.services().ok_or_else(|| {
crate::Error::message(format!(
"sandbox provider `{}` does not support background services",
self.kind
))
})
}
fn search(&self) -> crate::Result<sandbox_driver::SearchFacet<'_>> {
self.handle()?.search().ok_or_else(|| {
crate::Error::message(format!(
@ -512,23 +433,27 @@ impl RunSandbox {
let Some(pending) = &self.pending else {
return self.handle().map(|_| ());
};
let prepared = pending.plan.prepare(self.events.clone()).await?;
if let Some(snapshot) = prepared.snapshot {
let _ = self.snapshot.set(snapshot);
}
let handle = pending
.provider
.create(&prepared.spec, self.events.clone())
.create(&pending.spec, self.events.clone())
.await
.map_err(|error| {
crate::Error::context(format!("Failed to create {} sandbox", self.kind), error)
})?;
// The provider may have created the sandbox from a snapshot it
// built or chose (Daytona caches images as snapshots); the run
// record names it.
if let Ok(status) = handle.describe().await {
if let Some(snapshot) = status.snapshot {
let _ = self.snapshot.set(snapshot);
}
}
let _ = self.handle.set(handle);
Ok(())
}
/// Bring the sandbox to `Running` with a verified Bash, and learn its
/// platform. Shared by initialize and start.
/// platform. Shared by initialize and activate.
async fn make_ready(&self) -> crate::Result<()> {
sandbox_driver::activate(self.handle()?.as_ref(), &WaitOptions::default()).await?;
self.learn_platform().await
@ -537,9 +462,7 @@ impl RunSandbox {
/// Prepare the workspace after the sandbox runs for the first time:
/// an empty root, or fabro's clone.
async fn prepare_workspace(&self) -> crate::Result<()> {
let Some(workspace) = &self.workspace else {
return Ok(());
};
let workspace = &self.workspace;
let layout = workspace
.resolve_layout(self.handle()?.working_directory())
.clone();
@ -579,7 +502,7 @@ impl RunSandbox {
let handle = self.handle()?;
// The clone names every directory it touches, so it runs
// without fabro's working-directory override.
let exec = SandboxExec::new(handle.exec(), self.env_policy);
let exec = SandboxExec::new(handle.exec());
let outcome = clone::clone_github_repo(
&self.kind,
handle.as_ref(),
@ -623,7 +546,7 @@ impl RunSandbox {
/// Open an interactive shell in the sandbox's working directory over the
/// driver's Pty facet.
pub async fn open_terminal(&self, size: TerminalSize) -> crate::Result<DriverTerminalSession> {
pub async fn open_terminal(&self, size: PtySize) -> crate::Result<Box<dyn PtySession>> {
let handle = self.handle()?;
let pty = handle.pty().ok_or_else(|| {
crate::Error::message(format!(
@ -632,16 +555,11 @@ impl RunSandbox {
))
})?;
let mut options = PtyOptions::default();
options.size = PtySize {
rows: size.rows,
cols: size.cols,
};
options.size = size;
options.working_dir = Some(self.working_directory().to_string());
let session = pty
.open(&options)
pty.open(&options)
.await
.map_err(|error| crate::Error::context("Failed to open sandbox terminal", error))?;
Ok(DriverTerminalSession::new(session))
.map_err(|error| crate::Error::context("Failed to open sandbox terminal", error))
}
/// Ask the sandbox for its platform once; `platform` and `os_version`
@ -669,11 +587,7 @@ impl RunSandbox {
// A cloned repository is reached through a workspace link. The
// driver refuses a symlinked traversal root, so walk the real
// checkout; results are reported under the link.
if let Some(checkout) = self
.workspace
.as_ref()
.and_then(|workspace| workspace.checkout_path.get())
{
if let Some(checkout) = self.workspace.checkout_path.get() {
return sandbox::join_sandbox_path(checkout, relative_start);
}
if relative_start.is_empty() {
@ -713,23 +627,6 @@ impl RunSandbox {
.map_err(|err| crate::Error::context("File is not valid UTF-8", err))
}
/// A file's text with line numbers, from `offset` for `limit` lines.
///
/// Kept only for `fabro-agent`, which is being deleted; pebble's
/// `Environment::read_file` is the numbered read from then on.
pub async fn read_file(
&self,
path: &str,
offset: Option<usize>,
limit: Option<usize>,
) -> crate::Result<String> {
Ok(sandbox::format_lines_numbered(
&self.read_file_text(path).await?,
offset,
limit,
))
}
pub async fn write_file(&self, path: &str, content: &str) -> crate::Result<()> {
self.handle()?
.fs()
@ -803,22 +700,29 @@ impl RunSandbox {
.await
}
/// Runs `spec` under fabro's exec policy, delivering output through
/// `controls.sink` as it arrives. Build the spec with
/// [`ExecSpec::bash`]; the policy fills the stop grace, the run's
/// working directory, and the environment filter where the spec leaves
/// them open.
pub async fn exec_command_streaming(
&self,
request: ExecStreamingRequest<'_>,
spec: ExecSpec,
controls: ExecControls,
) -> crate::Result<ExecStreamingResult> {
self.exec()?.run_streaming(request).await
self.exec()?.run_streaming(spec, controls).await
}
/// Launches a long-lived process with bidirectional stdio. The returned
/// handle terminates the process; dropping it does not.
pub async fn spawn_stdio_process(
&self,
command: &str,
working_dir: Option<&str>,
env_vars: Option<&HashMap<String, String>>,
cancel_token: Option<CancellationToken>,
) -> crate::Result<StdioProcess> {
self.exec()?
.spawn_stdio(command, working_dir, env_vars, cancel_token)
.spawn_stdio(command, working_dir, env_vars)
.await
}
@ -936,12 +840,8 @@ impl RunSandbox {
}
/// The provider's console page for this sandbox, when it has one. Best
/// effort: a failed describe reports no page. The local sandbox is the
/// host and has none.
/// effort: a failed describe reports no page.
pub async fn console_url(&self) -> Option<String> {
if self.kind.is_local() {
return None;
}
self.handle()
.ok()?
.describe()
@ -950,48 +850,37 @@ impl RunSandbox {
.and_then(|status| status.web_url)
}
/// Idempotent access-time check: a running sandbox is left alone; a
/// stopped or paused one is brought back and its Bash verified.
/// Brings the sandbox back into use, idempotently: a running sandbox is
/// left alone and only its platform is learned when unknown; a stopped
/// or paused one is started and its Bash verified. Resume and every
/// access-time caller share this one entry point.
pub async fn activate(&self) -> crate::Result<()> {
let status = self.handle()?.describe().await?;
if status.state == SandboxState::Running {
return Ok(());
return self.learn_platform().await;
}
self.make_ready().await
}
pub async fn start(&self) -> crate::Result<()> {
self.make_ready().await
}
pub async fn stop(&self) -> crate::Result<()> {
self.handle()?.stop().await.map_err(crate::Error::from)
}
pub async fn delete(&self) -> crate::Result<()> {
self.release().await
}
/// Releases the sandbox. For a designated host directory this frees the
/// handle and leaves the directory in place; for an isolated provider it
/// removes the sandbox.
pub async fn cleanup(&self) -> crate::Result<()> {
/// removes the sandbox. A pending sandbox that was never created has
/// nothing to release.
pub async fn delete(&self) -> crate::Result<()> {
self.release().await
}
/// The directory the run works in: the cloned repository's link for a
/// clone-based workspace, the provider's working directory otherwise.
pub fn working_directory(&self) -> &str {
if let Some(directory) = self
.workspace
.as_ref()
.and_then(RepoWorkspace::working_directory)
{
return directory;
}
self.handle
.get()
.map_or("", |handle| handle.working_directory())
self.workspace
.working_directory()
.or_else(|| self.handle.get().map(|handle| handle.working_directory()))
.unwrap_or("")
}
pub fn runtime_directory(&self) -> Option<&str> {
@ -1013,14 +902,10 @@ impl RunSandbox {
)
}
/// The provider's id for this sandbox, or empty for `local`: a local
/// sandbox is its working directory, which the run record already
/// carries, and its Host registry id does not outlive the process.
/// Empty for a pending sandbox that has not been created.
/// The provider's id for this sandbox; for `local`, the id the Host
/// provider derives from the working directory. Empty for a pending
/// sandbox that has not been created.
pub fn sandbox_info(&self) -> String {
if self.kind.is_local() {
return String::new();
}
self.handle
.get()
.map(|handle| handle.id().to_string())
@ -1032,23 +917,7 @@ impl RunSandbox {
}
pub fn workspace_layout(&self) -> Option<SandboxWorkspaceLayout> {
self.workspace.as_ref().and_then(RepoWorkspace::record)
}
pub async fn set_autostop_interval(&self, minutes: i32) -> crate::Result<()> {
let mut timers = LifecycleTimers::default();
timers.auto_stop_after_idle = u64::try_from(minutes)
.ok()
.filter(|minutes| *minutes > 0)
.map(Duration::from_mins);
match self.handle()?.set_timers(&timers).await {
// A provider without timers has nothing to stop automatically.
Ok(()) | Err(sandbox_driver::Error::Unsupported { .. }) => Ok(()),
Err(error) => Err(crate::Error::context(
"Failed to set sandbox auto-stop",
error,
)),
}
self.workspace.record()
}
pub async fn setup_git(&self, intent: &GitSetupIntent) -> crate::Result<Option<GitRunInfo>> {
@ -1058,84 +927,64 @@ impl RunSandbox {
sandbox::setup_git(self, intent).await.map(Some)
}
pub fn resume_setup_commands(&self, run_branch: &str) -> Vec<String> {
if !self.repo_cloned() {
return Vec::new();
}
vec![format!(
"git fetch origin {} && git checkout {}",
sandbox::shell_quote(run_branch),
sandbox::shell_quote(run_branch)
)]
}
/// Push `refspec` from the run's checkout. A checkout fabro cloned
/// pushes with the credentials it was cloned with. Any other checkout
/// pushes only when it has an origin, with whatever credentials it
/// carries itself; a workspace without one has nothing to push.
pub async fn git_push_ref(
&self,
refspec: &str,
plan: &RetryPlan,
policy: &GitRetryPolicy,
) -> Result<PushReport, PushError> {
let Some(workspace) = &self.workspace else {
// A designated directory: push only when the checkout has an
// origin, with whatever credentials its URL already carries.
let has_origin = match self
.exec_command("git remote get-url origin", 10_000, None, None, None)
.await
{
Ok(result) if result.is_success() => true,
Ok(_) => false,
Err(err) => {
return Err(PushError {
report: PushReport::default(),
error: crate::Error::context("git remote get-url origin", err),
});
}
};
if !has_origin {
return Ok(PushReport::default());
let workspace = &self.workspace;
if workspace.repo_cloned() {
return sandbox::git_push(self, Some(&workspace.credentials), refspec, policy).await;
}
let has_origin = match self
.exec_command("git remote get-url origin", 10_000, None, None, None)
.await
{
Ok(result) => result.success(),
Err(err) => {
return Err(PushError {
report: PushReport::default(),
error: crate::Error::context("git remote get-url origin", err),
});
}
return sandbox::git_push(self, None, refspec, plan).await;
};
if !workspace.repo_cloned() {
if !has_origin {
return Ok(PushReport::default());
}
let credentials = workspace
.origin_url
.get()
.map(|origin_url| (&workspace.credentials, origin_url.as_str()));
sandbox::git_push(self, credentials, refspec, plan).await
sandbox::git_push(self, None, refspec, policy).await
}
pub fn origin_url(&self) -> Option<&str> {
let workspace = self.workspace.as_ref()?;
if !workspace.repo_cloned() {
if !self.workspace.repo_cloned() {
return None;
}
workspace.origin_url.get().map(String::as_str)
self.workspace.origin_url.get().map(String::as_str)
}
/// Renew the credentials the agent's own git commands read for the
/// checkout: resolve the current token and rewrite the checkout's
/// credential store with it. Returns the token's non-secret description,
/// or `None` when this sandbox has no managed credentials or no
/// checkout to install them in.
#[tracing::instrument(name = "git_op", skip_all, fields(op = "refresh-credentials"))]
pub async fn refresh_push_credentials(&self) -> crate::Result<RefreshOutcome> {
let Some(workspace) = &self.workspace else {
return Ok(RefreshOutcome::none());
pub async fn refresh_ambient_credentials(&self) -> crate::Result<Option<TokenSnapshot>> {
let workspace = &self.workspace;
let Some(checkout) = workspace.checkout_path.get() else {
return Ok(None);
};
if !workspace.repo_cloned() {
return Ok(RefreshOutcome::none());
}
let Some(origin_url) = workspace.origin_url.get() else {
return Ok(RefreshOutcome::none());
let Some(token) = workspace.credentials.resolve().await? else {
return Ok(None);
};
workspace
.credentials
.refresh(origin_url, |auth_url| {
push_credentials::set_auth_url_via_exec(self, auth_url)
})
.await
RepoCredentials::install(&self.git()?, checkout, &token).await?;
Ok(Some(token.snapshot))
}
pub fn push_token_source(&self) -> Option<Arc<InstallationTokenSource>> {
self.workspace
.as_ref()
.and_then(|workspace| workspace.credentials.source().cloned())
self.workspace.credentials.source().cloned()
}
/// The local command that opens a shell in the sandbox, from the
@ -1213,9 +1062,7 @@ impl PreviewUrls for SandboxPortRoutes {
impl RunSandbox {
fn repo_cloned(&self) -> bool {
self.workspace
.as_ref()
.is_some_and(RepoWorkspace::repo_cloned)
self.workspace.repo_cloned()
}
/// Delete the sandbox on the provider. A pending sandbox that was never
@ -1237,12 +1084,16 @@ fn elapsed_ms(started: Instant) -> u64 {
mod tests {
use std::sync::Mutex;
use fabro_types::CommandTermination;
use sandbox_driver::{SandboxProvider as _, SandboxSource, SandboxSpec};
use async_trait::async_trait;
use sandbox_driver::{SandboxProvider as _, SandboxSource, SandboxSpec, Termination};
use sandbox_driver_host::HostProvider;
use tokio::fs;
use super::*;
use crate::driver::ProviderAccess;
use crate::exec::ExecResultExt;
use crate::provider_sandbox::local_sandbox;
use crate::sandbox_spec::SandboxSpec as RunSandboxSpec;
struct Fixture {
dir: tempfile::TempDir,
@ -1290,7 +1141,10 @@ mod tests {
.read_file_text("nonexistent.txt")
.await
.unwrap_err();
assert!(read.is_not_found(), "{read}");
assert!(
matches!(read.driver(), Some(sandbox_driver::Error::NotFound { .. })),
"{read}"
);
}
#[tokio::test]
@ -1329,15 +1183,15 @@ mod tests {
)
.await
.unwrap();
assert_eq!(ok.stdout, "hello\nbash\n");
assert!(ok.is_success());
assert_eq!(ok.stdout_lossy(), "hello\nbash\n");
assert!(ok.success());
let timed_out = f
.sandbox
.exec_command("sleep 10", 200, None, None, None)
.await
.unwrap();
assert_eq!(timed_out.termination, CommandTermination::TimedOut);
assert_eq!(timed_out.exit_code, None);
assert_eq!(timed_out.termination, Termination::TimedOut);
assert_eq!(timed_out.program_exit_code(), None);
}
#[tokio::test]
@ -1471,14 +1325,13 @@ mod tests {
async fn lifecycle_reaches_the_driver_events_and_learns_the_platform() {
let dir = tempfile::tempdir().unwrap();
let recorded = Arc::new(Recorded(Mutex::new(Vec::new())));
let sandbox = local_sandbox_with_events(
dir.path(),
Some(EventContext::new(
let sandbox = RunSandboxSpec::local(dir.path(), ProviderAccess::default())
.build(Some(EventContext::new(
Arc::clone(&recorded) as Arc<dyn sandbox_driver::EventObserver>
)),
)
.await
.unwrap();
)))
.await
.unwrap();
sandbox.initialize().await.unwrap();
let expected = if cfg!(target_os = "macos") {
"darwin"
} else {
@ -1486,19 +1339,20 @@ mod tests {
};
assert_eq!(sandbox.platform(), expected);
assert!(sandbox.os_version().starts_with(expected));
assert_eq!(
sandbox.sandbox_info(),
"",
"local sandboxes are identified by directory"
);
let handle = Arc::clone(sandbox.handle().unwrap());
assert_eq!(sandbox.sandbox_info(), handle.id().to_string());
assert!(
sandbox.sandbox_info().starts_with("host-dir-"),
"a local sandbox is identified by its directory: {}",
sandbox.sandbox_info()
);
let isolated = RunSandbox::new(SandboxProviderKind::DOCKER, Arc::clone(&handle));
assert_eq!(isolated.sandbox_info(), handle.id().to_string());
assert_eq!(sandbox.console_url().await, None);
sandbox.stop().await.unwrap();
sandbox.activate().await.unwrap();
sandbox.cleanup().await.unwrap();
sandbox.delete().await.unwrap();
assert!(
dir.path().is_dir(),
"designated directories survive cleanup"
@ -1550,7 +1404,7 @@ mod tests {
Path::new(sandbox.working_directory()),
workspace.canonicalize().unwrap()
);
sandbox.cleanup().await.unwrap();
sandbox.delete().await.unwrap();
assert!(workspace.is_dir());
}

626
lib/components/fabro-sandbox/src/environment.rs
View file

@ -1,364 +1,314 @@
//! [`RunSandbox`] as the [`Environment`] pebble's coding agent runs in.
//! What an environment asks of a sandbox, mapped once onto the driver's spec.
//!
//! Pebble's tools speak the `Environment` contract; fabro's one sandbox type
//! speaks the sandbox driver's facets. This module is the mapping between the
//! two, and nothing else: every path resolves the way fabro resolves it, every
//! command runs through [`SandboxExec`](crate::SandboxExec) with fabro's
//! environment policy, and every failure keeps its driver cause. There is no
//! adapter struct; a run sandbox *is* an environment.
//!
//! Where the two contracts differ, pebble's wins here because the model reads
//! pebble's: a glob that pebble rejects is rejected before the driver sees it,
//! a directory listing is in tree order, and a command with no retention cap
//! still drains under the driver's default buffer rather than without bound.
//! The environment names an image or Dockerfile, resources, a network
//! policy, labels, variables, and a lifecycle. Every provider starts from
//! the same driver [`SandboxSpec`] built here; a bundled provider adds only
//! what its backend needs on top (the Docker working directory and default
//! image, the Daytona snapshot and timers) in its own overlay, and the
//! ownership scope adds fabro's labels. The clone policy travels beside the
//! spec as a [`CloneRequest`]: cloning is fabro's work once the sandbox
//! exists, not the provider's.
use std::sync::Arc;
use std::collections::BTreeMap;
use async_trait::async_trait;
use fabro_types::CommandOutputStream;
use pebble_coding_agent::environment::support::{capture_stats, tree_order, validate_glob};
use pebble_coding_agent::environment::{
DirEntry, EnvResult, Environment, EnvironmentError, EnvironmentErrorKind, ExecOutcome,
ExecOutputSink, ExecOutputStream, ExecRequest, ExecResult, GrepOptions,
use fabro_types::RunId;
use fabro_types::settings::run::{
DockerfileSource, EnvironmentNetworkMode, RunCloneSettings, RunEnvironmentSettings,
};
use sandbox_driver::{
Capabilities, LifecycleTimers, NetworkPolicy, Resources, SandboxSource, SandboxSpec,
};
use sandbox_driver::FileKind;
use crate::driver_sandbox::RunSandbox;
use crate::sandbox::{self, CommandOutputCallback, ExecStreamingRequest};
/// What to clone into a provider sandbox, if anything.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct CloneRequest {
pub origin_url: Option<String>,
/// The branch the checkout works on.
pub branch: Option<String>,
/// A tag to pin the checkout to; the branch still names the checkout.
pub tag: Option<String>,
/// An exact commit to pin the checkout to, authoritative over `tag`.
pub commit_sha: Option<String>,
/// Maximum Git history depth fetched; `None` fetches full history.
pub depth: Option<u32>,
/// Create an empty workspace instead of cloning, even when an origin
/// is present.
pub skip: bool,
}
#[async_trait]
impl Environment for RunSandbox {
fn working_directory(&self) -> &str {
Self::working_directory(self)
}
fn platform(&self) -> &str {
Self::platform(self)
}
fn os_version(&self) -> String {
Self::os_version(self)
}
async fn read_file_bytes(&self, path: &str) -> EnvResult<Vec<u8>> {
Self::read_file_bytes(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to read {path}"), error))
}
async fn write_file(&self, path: &str, content: &str) -> EnvResult<()> {
Self::write_file(self, path, content)
.await
.map_err(|error| environment_error(&format!("Failed to write {path}"), error))
}
async fn rename_file(&self, source: &str, destination: &str) -> EnvResult<()> {
let resolved_source = self.resolve_for_environment(source);
let resolved_destination = self.resolve_for_environment(destination);
if !Self::file_exists(self, source)
.await
.map_err(|error| environment_error(&format!("Failed to stat {source}"), error))?
{
return Err(EnvironmentError::new(
EnvironmentErrorKind::NotFound,
format!("Failed to move {source}: file does not exist"),
));
impl CloneRequest {
/// No clone: the run starts in an empty workspace.
#[must_use]
pub fn none() -> Self {
Self {
skip: true,
..Self::default()
}
// The same path spelled twice is a move to itself, which must leave
// the file where it is. Aliases the sandbox's own filesystem would
// resolve (a symlinked parent, a hard link) are not checked: fabro has
// no remote `realpath`, and a driver `mv a a` is a no-op anyway.
if normalize(&resolved_source) == normalize(&resolved_destination) {
return Ok(());
}
/// The environment's clone policy: whether to clone and how deep. The
/// origin and the selectors come from the run's target.
#[must_use]
pub fn from_settings(clone: &RunCloneSettings) -> Self {
Self {
depth: clone
.depth_limit()
.and_then(|depth| u32::try_from(depth).ok()),
skip: !clone.enabled,
..Self::default()
}
let handle = self
.handle()
.map_err(|error| environment_error("Sandbox is not initialized", error))?;
// The destination's parent is created first, and a parent that is a
// file fails here, before anything has moved, so the source stays
// intact as the contract requires.
if let Some(parent) = parent_directory(&resolved_destination) {
handle.fs().create_dir(parent).await.map_err(|error| {
environment_error(
&format!("Failed to create the parent directory of {destination}"),
crate::Error::from(error),
)
})?;
}
handle
.fs()
.rename(&resolved_source, &resolved_destination)
.await
.map_err(|error| {
environment_error(
&format!("Failed to move {source} to {destination}"),
crate::Error::from(error),
)
})
}
async fn delete_file(&self, path: &str) -> EnvResult<()> {
// The driver's delete is idempotent; pebble's is a `remove_file`, which
// reports a path that is not there.
if !Self::file_exists(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to stat {path}"), error))?
{
return Err(EnvironmentError::new(
EnvironmentErrorKind::NotFound,
format!("Failed to delete {path}: file does not exist"),
));
}
Self::delete_file(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to delete {path}"), error))
}
async fn file_exists(&self, path: &str) -> EnvResult<bool> {
Self::file_exists(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to stat {path}"), error))
}
async fn list_directory(&self, path: &str, depth: Option<usize>) -> EnvResult<Vec<DirEntry>> {
let mut entries: Vec<DirEntry> = Self::list_directory(self, path, depth)
.await
.map_err(|error| environment_error(&format!("Failed to list {path}"), error))?
.into_iter()
.map(|entry| DirEntry {
is_dir: entry.kind == FileKind::Directory,
size: (entry.kind == FileKind::File)
.then_some(entry.size)
.flatten(),
name: entry.path,
})
.collect();
// The driver lists in flat lexicographic order of the whole relative
// path, where `foo-bar` sorts between `foo` and `foo/x`. Pebble lists
// in tree order, and says how.
tree_order(&mut entries);
Ok(entries)
}
async fn grep(
&self,
pattern: &str,
path: &str,
options: &GrepOptions,
) -> EnvResult<Vec<String>> {
let mut driver_options = sandbox_driver::GrepOptions::default();
driver_options.case_insensitive = options.case_insensitive;
driver_options.max_matches = options.max_results;
driver_options.include = options.glob_filter.clone();
let matches = Self::grep(self, pattern, path, &driver_options)
.await
.map_err(|error| environment_error("Failed to search file contents", error))?;
Ok(matches
.into_iter()
.map(|found| format!("{}:{}:{}", found.path, found.line_number, found.line))
.collect())
}
async fn glob(&self, pattern: &str, path: Option<&str>) -> EnvResult<Vec<String>> {
// Validated by pebble's own grammar before the driver sees the
// pattern, so the reason reaches the model in pebble's words and the
// patterns pebble rejects are rejected even where fabro's glob would
// accept them.
validate_glob(pattern)?;
Self::glob(self, pattern, path)
.await
.map_err(|error| environment_error("Failed to match files", error))
}
async fn exec(&self, request: ExecRequest<'_>) -> EnvResult<ExecOutcome> {
let ExecRequest {
command,
timeout_ms,
working_dir,
env_vars,
cancel_token,
output_bytes_cap,
output_sink,
} = request;
let streaming = self
.exec_command_streaming(ExecStreamingRequest {
timeout_ms,
working_dir,
env_vars,
cancel_token,
stdin: None,
output_callback: output_sink.map(adapt_output_sink),
// `None` asks pebble for no cap at all. The driver always
// retains under a buffer, so a command with no cap drains under
// the driver's default rather than without bound; the capture
// counts still say what was dropped.
stream_output_bytes_cap: output_bytes_cap,
..ExecStreamingRequest::new(command)
})
.await
.map_err(|error| {
let kind = if error.is_transport() {
EnvironmentErrorKind::Io
} else {
EnvironmentErrorKind::Spawn
};
EnvironmentError::with_source(kind, "Failed to run the command", error)
})?;
Ok(ExecOutcome {
result: ExecResult {
stdout: streaming.result.stdout,
stderr: streaming.result.stderr,
exit_code: streaming.result.exit_code,
termination: streaming.result.termination,
duration_ms: streaming.result.duration_ms,
},
streams_separated: streaming.streams_separated,
stdout_capture: capture_stats(
streaming.stdout_capture.observed_bytes,
output_bytes_cap,
),
stderr_capture: capture_stats(
streaming.stderr_capture.observed_bytes,
output_bytes_cap,
),
})
}
}
impl RunSandbox {
/// A caller path as the driver will see it: fabro's working directory
/// applied where fabro applies it, and nothing more.
fn resolve_for_environment(&self, path: &str) -> String {
sandbox::resolve_path(path, Self::working_directory(self))
}
}
/// Pebble's glob grammar, beyond what fabro's glob already rejects.
/// The driver spec every provider starts from: the environment's source
/// (an image, a Dockerfile, or a managed directory when it names neither),
/// its labels, variables, resources, network policy, and auto-stop. `env`
/// is the environment's variables, resolved by the caller: the worker
/// resolves secrets through the vault, while preflight carries them in
/// source form.
///
/// A path with its redundant separators and `.` segments removed, for
/// deciding whether two spellings name the same file.
fn normalize(path: &str) -> String {
let absolute = path.starts_with('/');
let joined = path
.split('/')
.filter(|segment| !segment.is_empty() && *segment != ".")
.collect::<Vec<_>>()
.join("/");
if absolute {
format!("/{joined}")
} else {
joined
}
}
/// The directory a path is in, when the path names one.
fn parent_directory(path: &str) -> Option<&str> {
let trimmed = path.trim_end_matches('/');
let (parent, _) = trimmed.rsplit_once('/')?;
if parent.is_empty() {
return Some("/");
}
Some(parent)
}
/// Feeds the driver's asynchronous chunk callback into pebble's synchronous
/// sink.
fn adapt_output_sink(sink: ExecOutputSink) -> CommandOutputCallback {
Arc::new(move |stream, chunk: Vec<u8>| {
let stream = match stream {
CommandOutputStream::Stdout => ExecOutputStream::Stdout,
CommandOutputStream::Stderr => ExecOutputStream::Stderr,
};
sink(stream, &chunk);
Box::pin(async { Ok(()) })
})
}
/// A sandbox failure as pebble classifies it, keeping the driver cause.
fn environment_error(message: &str, error: crate::Error) -> EnvironmentError {
let kind = if error.is_not_found() {
EnvironmentErrorKind::NotFound
} else if error.is_unsupported() {
EnvironmentErrorKind::Unsupported
} else {
EnvironmentErrorKind::Io
/// A Dockerfile given as a path must have been resolved to inline content
/// earlier; none of the providers can read a path.
pub fn sandbox_spec_for_environment(
settings: &RunEnvironmentSettings,
env: BTreeMap<String, String>,
) -> crate::Result<SandboxSpec> {
// fabro-config rejects environments that set both image.docker and
// image.dockerfile. If both still arrive here, the image wins.
let source = match (&settings.image.docker, &settings.image.dockerfile) {
(Some(reference), _) => SandboxSource::Image {
reference: reference.clone(),
},
(None, Some(DockerfileSource::Inline(content))) => SandboxSource::Dockerfile {
content: content.clone(),
},
(None, Some(DockerfileSource::Path { path })) => {
return Err(crate::Error::message(format!(
"environment `{}` names a Dockerfile path ({path}) that should have been \
resolved to inline content before sandbox creation",
settings.id
)));
}
// A provider without images (a host-style plugin) manages a
// workspace directory of its own.
(None, None) => SandboxSource::HostDirectory,
};
EnvironmentError::with_source(kind, message, error)
let network = match settings.network.mode {
EnvironmentNetworkMode::Block => NetworkPolicy::Block,
EnvironmentNetworkMode::AllowAll => NetworkPolicy::AllowAll,
EnvironmentNetworkMode::CidrAllowList => NetworkPolicy::CidrAllowList {
cidrs: settings.network.allow.clone(),
},
};
let mut spec = SandboxSpec::new(source).network(network);
// The environment's labels; fabro's ownership labels are stamped by the
// ownership scope the provider is connected through.
for (key, value) in &settings.labels {
spec = spec.label(key, value);
}
for (key, value) in env {
spec = spec.env_var(key, value);
}
let mut resources = Resources::default();
resources.cpu_cores = settings
.resources
.cpu
.and_then(|cpu| u32::try_from(cpu).ok());
resources.memory_mb = settings
.resources
.memory
.map(|size| mebibytes(size.as_bytes()));
resources.disk_mb = settings
.resources
.disk
.map(|size| mebibytes(size.as_bytes()));
let mut timers = LifecycleTimers::default();
timers.auto_stop_after_idle = settings
.lifecycle
.auto_stop
.map(|duration| duration.as_std());
Ok(spec.resources(resources).timers(timers))
}
/// Whole mebibytes, rounded up: the unit the driver sizes resources in.
fn mebibytes(bytes: u64) -> u64 {
bytes.div_ceil(1024 * 1024)
}
/// The provider-side name of a run's sandbox.
pub(crate) fn run_name(run_id: &RunId) -> String {
format!("fabro-run-{run_id}")
}
/// The environment's default `allow_all` means "unrestricted", which a
/// provider without network controls already is; asking such a provider
/// for it explicitly would be rejected. An explicit restriction is still
/// requested, and refused by the provider when it cannot honor it.
pub(crate) fn supported_network(
requested: NetworkPolicy,
capabilities: &Capabilities,
) -> NetworkPolicy {
match requested {
NetworkPolicy::AllowAll if !capabilities.network.allow_all => {
NetworkPolicy::ProviderDefault
}
other => other,
}
}
/// The environment's auto-stop is a request a backend without timers
/// cannot take; such a provider gets no timers rather than a rejected spec.
pub(crate) fn supported_timers(
requested: LifecycleTimers,
capabilities: &Capabilities,
) -> LifecycleTimers {
if capabilities.lifecycle.timers {
requested
} else {
LifecycleTimers::default()
}
}
#[cfg(test)]
mod tests {
use pebble_coding_agent::test_support::EnvironmentContract;
use std::collections::HashMap;
use std::time::Duration;
use fabro_types::SandboxProviderKind;
use fabro_types::settings::run::{
EnvironmentImageSettings, EnvironmentLifecycleSettings, EnvironmentNetworkSettings,
EnvironmentResourcesSettings,
};
use fabro_types::settings::{Duration as SettingsDuration, Size};
use super::*;
use crate::local_sandbox;
/// The run sandbox over the driver's Host provider, in a directory that
/// goes away with the test.
async fn host_environment() -> (tempfile::TempDir, RunSandbox) {
let directory = tempfile::tempdir().expect("a temporary directory");
let sandbox = local_sandbox(directory.path().to_path_buf())
.await
.expect("a local sandbox");
(directory, sandbox)
}
#[tokio::test]
async fn host_files_satisfy_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_files()
.await
.expect("file contract");
}
#[tokio::test]
async fn host_search_satisfies_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_search()
.await
.expect("search contract");
}
#[tokio::test]
async fn host_commands_satisfy_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_commands()
.await
.expect("command contract");
}
#[tokio::test]
async fn a_directory_listing_is_in_tree_order() {
let (directory, sandbox) = host_environment().await;
for name in ["foo/x.txt", "foo-bar/y.txt", "foo.txt"] {
Environment::write_file(&sandbox, name, "content")
.await
.expect("fixture");
fn environment(kind: &str) -> RunEnvironmentSettings {
RunEnvironmentSettings {
id: kind.to_string(),
provider: SandboxProviderKind::try_new(kind).unwrap(),
cwd: None,
image: EnvironmentImageSettings::default(),
resources: EnvironmentResourcesSettings::default(),
network: EnvironmentNetworkSettings::default(),
lifecycle: EnvironmentLifecycleSettings::default(),
labels: HashMap::from([("team".to_string(), "platform".to_string())]),
env: HashMap::new(),
}
let names: Vec<String> = Environment::list_directory(&sandbox, ".", Some(2))
.await
.expect("listing")
.into_iter()
.map(|entry| entry.name)
.collect();
assert_eq!(names, [
"foo",
"foo/x.txt",
"foo-bar",
"foo-bar/y.txt",
"foo.txt"
]);
drop(directory);
}
#[test]
fn a_path_spelled_two_ways_is_one_path() {
assert_eq!(normalize("/work//a/./b.txt"), "/work/a/b.txt");
assert_eq!(parent_directory("/work/a/b.txt"), Some("/work/a"));
assert_eq!(parent_directory("/b.txt"), Some("/"));
assert_eq!(parent_directory("b.txt"), None);
fn an_environment_without_an_image_asks_for_a_managed_directory() {
let spec = sandbox_spec_for_environment(
&environment("host"),
BTreeMap::from([("FOO".to_string(), "bar".to_string())]),
)
.unwrap();
assert!(matches!(spec.source, SandboxSource::HostDirectory));
assert!(spec.working_directory.is_none());
assert!(
spec.name.is_none(),
"the run names the sandbox, not the environment"
);
assert_eq!(spec.env.get("FOO").map(String::as_str), Some("bar"));
assert_eq!(
spec.labels.get("team").map(String::as_str),
Some("platform")
);
assert!(
!spec.labels.contains_key("sh.fabro.managed"),
"ownership labels come from the scope, not the environment"
);
assert!(matches!(spec.network, NetworkPolicy::AllowAll));
assert_eq!(spec.resources, Resources::default());
assert_eq!(spec.timers, LifecycleTimers::default());
}
#[test]
fn an_environment_with_an_image_maps_resources_network_and_lifecycle() {
let mut settings = environment("e2b");
settings.image.docker = Some("ubuntu:24.04".to_string());
settings.resources.cpu = Some(2);
settings.resources.memory = Some(Size::from_bytes(4_000_000_000));
settings.network.mode = EnvironmentNetworkMode::Block;
settings.lifecycle.auto_stop = Some(SettingsDuration::from_std(Duration::from_mins(45)));
let spec = sandbox_spec_for_environment(&settings, BTreeMap::new()).unwrap();
assert!(matches!(
&spec.source,
SandboxSource::Image { reference } if reference == "ubuntu:24.04"
));
assert_eq!(spec.resources.cpu_cores, Some(2));
assert_eq!(spec.resources.memory_mb, Some(3815));
assert!(matches!(spec.network, NetworkPolicy::Block));
assert_eq!(
spec.timers.auto_stop_after_idle,
Some(Duration::from_mins(45))
);
}
#[test]
fn the_clone_request_carries_the_environments_policy() {
let clone = CloneRequest::from_settings(&RunCloneSettings::default());
assert_eq!(clone.depth, Some(100));
assert!(!clone.skip);
let clone = CloneRequest::from_settings(&RunCloneSettings {
enabled: false,
depth: 0,
});
assert_eq!(clone.depth, None);
assert!(clone.skip);
assert!(CloneRequest::none().skip);
}
#[test]
fn an_inline_dockerfile_becomes_the_source_and_a_path_is_rejected() {
let mut settings = environment("daytona");
settings.image.dockerfile = Some(DockerfileSource::Inline("FROM ubuntu".to_string()));
let spec = sandbox_spec_for_environment(&settings, BTreeMap::new()).unwrap();
assert!(matches!(
spec.source,
SandboxSource::Dockerfile { content } if content == "FROM ubuntu"
));
settings.image.dockerfile = Some(DockerfileSource::Path {
path: "Dockerfile".to_string(),
});
let error = sandbox_spec_for_environment(&settings, BTreeMap::new()).unwrap_err();
assert!(error.to_string().contains("Dockerfile path"), "{error}");
}
#[test]
fn allow_all_falls_back_to_the_provider_default_without_network_control() {
let none = Capabilities::minimal(sandbox_driver::Isolation::None);
assert!(matches!(
supported_network(NetworkPolicy::AllowAll, &none),
NetworkPolicy::ProviderDefault
));
assert!(matches!(
supported_network(NetworkPolicy::Block, &none),
NetworkPolicy::Block
));
let mut full = Capabilities::minimal(sandbox_driver::Isolation::Container);
full.network.allow_all = true;
assert!(matches!(
supported_network(NetworkPolicy::AllowAll, &full),
NetworkPolicy::AllowAll
));
}
#[test]
fn timers_are_dropped_for_a_provider_without_them() {
let mut requested = LifecycleTimers::default();
requested.auto_stop_after_idle = Some(Duration::from_mins(45));
let none = Capabilities::minimal(sandbox_driver::Isolation::None);
assert_eq!(
supported_timers(requested, &none),
LifecycleTimers::default()
);
let mut with_timers = Capabilities::minimal(sandbox_driver::Isolation::Container);
with_timers.lifecycle.timers = true;
assert_eq!(supported_timers(requested, &with_timers), requested);
}
}

264
lib/components/fabro-sandbox/src/error.rs
View file

@ -2,7 +2,6 @@ use std::fmt::Write as _;
use fabro_util::error::{collect_causes, render_with_causes};
use crate::ExecResult;
use crate::sandbox::{DEFAULT_EXEC_OUTPUT_TAIL_BYTES, redacted_output_tail};
#[derive(Debug, thiserror::Error)]
@ -26,21 +25,10 @@ pub enum Error {
/// A sandbox-driver failure: provider, transport, or an operation whose
/// outcome is unknown. The driver's own variants stay reachable through
/// [`Error::driver`] so callers can act on `NotFound`, `Unsupported`,
/// `Transport`, and `Incomplete` without string matching.
/// [`Error::driver`] so callers can act on `Exec`, `Git`, and `NotFound`
/// without string matching.
#[error(transparent)]
Driver(Box<sandbox_driver::Error>),
#[error(
"{label} failed (exit {exit}, termination={termination}, duration_ms={duration_ms}) - hint: {hint}",
exit = format_exit_code(result.exit_code),
termination = result.termination,
duration_ms = result.duration_ms,
hint = classify_exec_failure(&result.stderr)
.or_else(|| classify_exec_failure(&result.stdout))
.unwrap_or("unclassified")
)]
Exec { label: String, result: ExecResult },
}
impl Error {
@ -65,13 +53,6 @@ impl Error {
}
}
pub fn exec(label: impl Into<String>, result: ExecResult) -> Self {
Self::Exec {
label: label.into(),
result,
}
}
pub fn default_redacted_output_tail(&self) -> Option<fabro_types::ExecOutputTail> {
default_redacted_output_tail(self)
}
@ -80,10 +61,6 @@ impl Error {
collect_causes(self)
}
pub fn driver_error(source: sandbox_driver::Error) -> Self {
Self::Driver(Box::new(source))
}
/// The underlying sandbox-driver error, when this error carries one
/// anywhere in its chain.
pub fn driver(&self) -> Option<&sandbox_driver::Error> {
@ -100,36 +77,6 @@ impl Error {
None
}
/// The facts established when a driver operation ended without a
/// complete outcome. A caller that sees `Some` must not replay the
/// operation: its effects may already have happened.
pub fn incomplete_operation(&self) -> Option<&sandbox_driver::IncompleteOperation> {
match self.driver()? {
sandbox_driver::Error::Incomplete(incomplete) => Some(incomplete),
_ => None,
}
}
/// True when communication with an out-of-process provider failed. The
/// operation may or may not have run; fabro rebuilds handles through
/// `attach` rather than retrying blind.
pub fn is_transport(&self) -> bool {
matches!(self.driver(), Some(sandbox_driver::Error::Transport(_)))
}
/// True when the driver reported the resource missing.
pub fn is_not_found(&self) -> bool {
matches!(self.driver(), Some(sandbox_driver::Error::NotFound { .. }))
}
/// True when the provider does not support the requested capability.
pub fn is_unsupported(&self) -> bool {
matches!(
self.driver(),
Some(sandbox_driver::Error::Unsupported { .. })
)
}
pub fn display_with_causes(&self) -> String {
render_with_causes(&self.to_string(), &self.causes())
}
@ -137,60 +84,10 @@ impl Error {
impl From<sandbox_driver::Error> for Error {
fn from(value: sandbox_driver::Error) -> Self {
Self::driver_error(value)
Self::Driver(Box::new(value))
}
}
impl From<String> for Error {
fn from(value: String) -> Self {
Self::Message(value)
}
}
impl From<&str> for Error {
fn from(value: &str) -> Self {
Self::Message(value.to_string())
}
}
pub(crate) fn classify_exec_failure(stderr: &str) -> Option<&'static str> {
let lower = stderr.to_ascii_lowercase();
if lower.contains("could not read username") || lower.contains("terminal prompts disabled") {
Some(
"no credentials in origin URL - check that the sandbox forwarded \
GITHUB_APP_PRIVATE_KEY (or GITHUB_TOKEN) and that refresh_push_credentials succeeded",
)
} else if lower.contains("permission to") && lower.contains("denied") {
Some(
"github denied the push - installation token lacks contents:write \
on this repo, or a branch protection / push ruleset is rejecting the ref",
)
} else if lower.contains("protected branch")
|| lower.contains("ruleset")
|| lower.contains("rejected")
{
Some("github rejected the ref - likely a branch protection rule or push ruleset")
} else if lower.contains("authentication failed") || lower.contains("invalid username") {
Some("github authentication failed - installation token may be expired or wrong scope")
} else if lower.contains("could not resolve host") || lower.contains("network is unreachable") {
Some("network failure inside sandbox - check DNS / egress from the run container")
} else if lower.contains("repository not found") {
Some("github 404 - repository is unavailable to the current credentials")
} else if lower.contains("no such remote") && lower.contains("origin") {
Some("origin remote missing - push credentials could not be installed")
} else if lower.contains("not a git repository")
|| lower.contains("does not appear to be a git repository")
{
Some("git repository unavailable in sandbox working directory")
} else {
None
}
}
fn format_exit_code(exit_code: Option<i32>) -> String {
exit_code.map_or_else(|| "none".to_string(), |code| code.to_string())
}
pub type Result<T> = std::result::Result<T, Error>;
pub fn default_redacted_output_tail(
@ -198,14 +95,10 @@ pub fn default_redacted_output_tail(
) -> Option<fabro_types::ExecOutputTail> {
let mut current = Some(err);
while let Some(err) = current {
match err.downcast_ref::<Error>() {
Some(Error::Exec { result, .. }) => return result.default_redacted_output_tail(),
Some(Error::Driver(driver)) => {
if let Some(tail) = driver_output_tail(driver) {
return Some(tail);
}
if let Some(Error::Driver(driver)) = err.downcast_ref::<Error>() {
if let Some(tail) = driver_output_tail(driver) {
return Some(tail);
}
_ => {}
}
if let Some(driver) = err.downcast_ref::<sandbox_driver::Error>() {
if let Some(tail) = driver_output_tail(driver) {
@ -262,64 +155,69 @@ fn append_tail_for_log(rendered: &mut String, stream: &str, tail: Option<&str>,
#[cfg(test)]
mod tests {
use fabro_types::CommandTermination;
use std::time::Duration;
use sandbox_driver::{ExecResult, Termination};
use super::*;
use crate::exec::ExecResultExt;
const SECRET: &str = "ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA";
fn failed_push(stdout: &str, stderr: &str) -> Error {
let mut result =
ExecResult::new(Termination::Exited, Some(128), Duration::from_millis(210));
result.stdout = stdout.as_bytes().to_vec();
result.stderr = stderr.as_bytes().to_vec();
result.into_exec_error("git push origin refs/heads/run")
}
fn leaky_stderr() -> String {
format!(
"fatal: unable to access 'https://x-access-token:{SECRET}@github.com/owner/repo/':\n\
remote: Permission to owner/repo.git denied\n\
identity ~/.ssh/id_rsa_work"
)
}
#[test]
fn exec_display_is_log_safe() {
let stderr = "fatal: unable to access \
'https://x-access-token:ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA@github.com/owner/repo/':\n\
remote: Permission to owner/repo.git denied\n\
identity ~/.ssh/id_rsa_work";
let error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: String::new(),
stderr: stderr.to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let error = failed_push("", &leaky_stderr());
let rendered = error.to_string();
assert_exec_rendering_is_safe(&rendered);
assert!(rendered.contains("git push origin refs/heads/run"));
assert!(rendered.contains("exit 128"));
assert!(rendered.contains("termination=exited"));
assert!(rendered.contains("duration_ms=210"));
assert!(rendered.contains("hint:"));
assert!(rendered.contains("128"));
assert!(rendered.contains("210 ms"));
}
#[test]
fn display_with_causes_does_not_reintroduce_raw_exec_output() {
let stderr = "fatal: unable to access \
'https://x-access-token:ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA@github.com/owner/repo/':\n\
remote: Permission to owner/repo.git denied\n\
identity ~/.ssh/id_rsa_work";
let exec_error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: "stdout secret ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA".to_string(),
stderr: stderr.to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let exec_error = failed_push(&format!("stdout secret {SECRET}"), &leaky_stderr());
let error = Error::context("metadata push failed", exec_error);
let rendered = error.display_with_causes();
assert_exec_rendering_is_safe(&rendered);
assert!(rendered.contains("metadata push failed"));
assert!(rendered.contains("git push origin refs/heads/run"));
assert!(rendered.contains("hint:"));
}
#[test]
fn the_driver_error_is_reachable_through_the_context_chain() {
let error = Error::context("metadata push failed", failed_push("", "boom"));
let Some(sandbox_driver::Error::Exec(failure)) = error.driver() else {
panic!("expected an exec failure, got {error:?}");
};
assert_eq!(failure.label(), "git push origin refs/heads/run");
assert_eq!(failure.exit_code(), Some(128));
assert_eq!(failure.termination(), Termination::Exited);
assert!(Error::message("plain").driver().is_none());
}
#[test]
fn display_for_log_walks_context_chain_and_emits_tail() {
let exec_error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: "last stdout line".to_string(),
stderr: "last stderr line".to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let exec_error = failed_push("last stdout line", "last stderr line");
let error = Error::context("metadata push failed", exec_error);
let rendered = display_for_log(&error);
@ -334,18 +232,15 @@ mod tests {
#[test]
fn display_for_log_redacts_secrets() {
let error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: "stdout secret ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA".to_string(),
stderr: "stderr secret ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA".to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let error = failed_push(
&format!("stdout secret {SECRET}"),
&format!("stderr secret {SECRET}"),
);
let rendered = display_for_log(&error);
assert!(
!rendered.contains("ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA"),
!rendered.contains(SECRET),
"log rendering leaked raw secret: {rendered}"
);
assert!(rendered.contains("REDACTED"));
@ -367,7 +262,7 @@ mod tests {
"fatal:",
"remote:",
"x-access-token",
"ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA",
SECRET,
"~/.ssh",
"id_rsa_work",
] {
@ -380,14 +275,7 @@ mod tests {
#[test]
fn exec_error_exposes_default_redacted_output_tail() {
let stderr = "stderr secret ghs_xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA";
let error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: "last stdout line".to_string(),
stderr: stderr.to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let error = failed_push("last stdout line", &format!("stderr secret {SECRET}"));
let tail = error.default_redacted_output_tail().expect("tail present");
assert_eq!(tail.stdout.as_deref(), Some("last stdout line"));
@ -401,13 +289,7 @@ mod tests {
#[test]
fn free_tail_helper_walks_context_chain() {
let exec_error = Error::exec("git push origin refs/heads/run", crate::ExecResult {
stdout: "last stdout line".to_string(),
stderr: "last stderr line".to_string(),
exit_code: Some(128),
termination: CommandTermination::Exited,
duration_ms: 210,
});
let exec_error = failed_push("last stdout line", "last stderr line");
let error = Error::context("metadata push failed", exec_error);
let tail = default_redacted_output_tail(&error).expect("tail present");
@ -415,42 +297,4 @@ mod tests {
assert_eq!(tail.stdout.as_deref(), Some("last stdout line"));
assert_eq!(tail.stderr.as_deref(), Some("last stderr line"));
}
#[test]
fn classify_exec_failure_documents_known_branches() {
let cases = [
(
"fatal: could not read Username for 'https://github.com'",
"no credentials in origin URL",
),
(
"remote: Permission to owner/repo.git denied to fabro-app[bot].",
"github denied the push",
),
(
"remote: error: GH013: Repository rule violations found due to ruleset",
"github rejected the ref",
),
(
"fatal: Authentication failed for 'https://github.com/owner/repo'",
"github authentication failed",
),
(
"fatal: could not resolve host: github.com",
"network failure",
),
("remote: Repository not found.", "github 404"),
("error: No such remote 'origin'", "origin remote missing"),
("fatal: not a git repository", "git repository unavailable"),
];
for (stderr, expected) in cases {
let hint = classify_exec_failure(stderr).expect(stderr);
assert!(
hint.contains(expected),
"expected {hint:?} to contain {expected:?}"
);
}
assert_eq!(classify_exec_failure("weird new git error"), None);
}
}

710
lib/components/fabro-sandbox/src/exec.rs
View file

@ -1,41 +1,44 @@
//! Fabro's command execution policy over the sandbox-driver [`Exec`] facet.
//!
//! A command runs as Bash source under `bash -c` with `BASH_ENV` blanked,
//! and ends in one of three ways:
//! The vocabulary is the driver's own: an [`ExecSpec`] and [`ExecControls`]
//! go in, an [`ExecResult`] or [`ExecStreamingResult`] comes out. This
//! module adds fabro's policy on the way in and fabro's reading of a result
//! on the way out.
//!
//! A command runs as Bash source under `bash -c` with `BASH_ENV` blanked by
//! the driver whatever the caller passed, and ends in one of three ways:
//!
//! - **timeout**: the spec's timeout fires and the provider runs the stop
//! ladder fabro asks for — `TERM`, then `KILL` after
//! [`SandboxExec::stop_grace`]. The result reports
//! [`CommandTermination::TimedOut`].
//! [`SandboxExec::stop_grace`]. The result reports [`Termination::TimedOut`].
//! - **cancellation**: the caller's [`CancellationToken`] is the `term` stop;
//! the provider escalates to `KILL` after the same grace. The result reports
//! [`CommandTermination::Cancelled`].
//! [`Termination::Cancelled`].
//! - **exit**: the process ended on its own.
//!
//! Output is drained regardless of the retention cap, redacted only when a
//! tail is rendered for events or logs, and delivered live through the
//! caller's callback. Explicit environment variables pass through a
//! fail-closed secret filter under [`ExplicitEnvPolicy::FilterSensitive`],
//! matching what the Host provider already does for inherited variables.
//! Output is drained regardless of the retention cap and delivered live
//! through the caller's [`sandbox_driver::OutputSink`]. Fabro reads command
//! output as text, so the policy asks the driver for
//! [`OutputSanitization::StripAll`]: terminal escape sequences and stray
//! control characters never reach a result, a sink chunk, or a tail. Secret
//! redaction stays fabro's job and happens only when a tail is rendered for
//! events or logs ([`ExecResultExt`]). The explicit environment reaches the
//! provider as the caller composed it: the driver filters credential-shaped
//! names out of the *inherited* host environment itself and treats the
//! spec's own variables as the deliberate channel for secrets, so fabro adds
//! no filter of its own.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use fabro_static::EnvVars;
use fabro_types::{CommandOutputStream, CommandTermination};
use fabro_types::{CommandTermination, ExecOutputTail};
use sandbox_driver::{
BASH_ENV_VAR, CaptureStats, Exec, ExecControls, ExecSpec, OutputStream, SpawnSpec,
StdioProcessHandle as DriverStdioProcessHandle, Termination, TransportError,
Exec, ExecControls, ExecFailure, ExecResult, ExecSpec, ExecStreamingResult, OutputSanitization,
SpawnSpec, StdioProcess, Termination,
};
use tokio_util::sync::CancellationToken;
use crate::sandbox::{
CommandOutputCallback, ExecResult, ExecStreamingRequest, ExecStreamingResult,
OutputCaptureStats, StderrCollector, StdioProcess, StdioProcessControl, StdioProcessHandle,
StdioProcessTermination,
};
use crate::sandbox::{DEFAULT_EXEC_OUTPUT_TAIL_BYTES, redacted_output_tail};
/// Time between `TERM` and `KILL` when fabro stops a command.
pub const DEFAULT_STOP_GRACE: Duration = Duration::from_secs(2);
@ -44,50 +47,9 @@ pub const DEFAULT_STOP_GRACE: Duration = Duration::from_secs(2);
/// renders, bounded so a runaway command cannot exhaust memory.
pub const DEFAULT_RETAINED_OUTPUT_BYTES: usize = sandbox_driver::DEFAULT_BUFFER_BYTES;
/// How explicit per-command environment variables are treated.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ExplicitEnvPolicy {
/// Drop variables whose names look like credentials unless safelisted.
/// Used where the command runs on the worker host and the caller's env
/// may carry worker secrets.
FilterSensitive,
/// Pass every variable through. Used for isolated providers, where the
/// caller composed the environment deliberately.
TrustCaller,
}
/// Variables that look like credentials but are needed by ordinary tools.
const ENV_SAFELIST: &[&str] = &[
EnvVars::PATH,
EnvVars::HOME,
EnvVars::USER,
EnvVars::SHELL,
EnvVars::LANG,
EnvVars::TERM,
EnvVars::TMPDIR,
EnvVars::GOPATH,
EnvVars::CARGO_HOME,
EnvVars::NVM_DIR,
];
/// Whether an environment variable name looks like a credential.
#[must_use]
pub fn is_sensitive_env_var(key: &str) -> bool {
if ENV_SAFELIST.contains(&key) {
return false;
}
let lower = key.to_lowercase();
lower.ends_with("_api_key")
|| lower.ends_with("_secret")
|| lower.ends_with("_token")
|| lower.ends_with("_password")
|| lower.ends_with("_credential")
}
/// Fabro's exec policy bound to one driver [`Exec`] facet.
pub struct SandboxExec<'a> {
exec: &'a dyn Exec,
env_policy: ExplicitEnvPolicy,
stop_grace: Duration,
/// Where a command runs when the caller names no directory. `None`
/// leaves the choice to the provider's own working directory.
@ -96,10 +58,9 @@ pub struct SandboxExec<'a> {
impl<'a> SandboxExec<'a> {
#[must_use]
pub fn new(exec: &'a dyn Exec, env_policy: ExplicitEnvPolicy) -> Self {
pub fn new(exec: &'a dyn Exec) -> Self {
Self {
exec,
env_policy,
stop_grace: DEFAULT_STOP_GRACE,
working_dir: None,
}
@ -131,7 +92,8 @@ impl<'a> SandboxExec<'a> {
///
/// Equivalent to `bash -c <command>` with a clean, non-login shell: no
/// `errexit`, no `pipefail`, `BASH_ENV` blanked. A caller that wants
/// different semantics writes them into the command.
/// different semantics writes them into the command. `None` for
/// `timeout` runs without a deadline.
pub async fn run(
&self,
command: &str,
@ -140,217 +102,180 @@ impl<'a> SandboxExec<'a> {
env_vars: Option<&HashMap<String, String>>,
cancel_token: Option<CancellationToken>,
) -> crate::Result<ExecResult> {
let streaming = self
.run_streaming(ExecStreamingRequest {
timeout_ms: timeout
.map(|timeout| u64::try_from(timeout.as_millis()).unwrap_or(u64::MAX)),
working_dir,
env_vars,
cancel_token,
..ExecStreamingRequest::new(command)
})
.await?;
Ok(streaming.result)
}
/// Runs Bash source, delivering output through `request.output_callback`
/// as it arrives. Same interpreter contract as [`Self::run`].
pub async fn run_streaming(
&self,
request: ExecStreamingRequest<'_>,
) -> crate::Result<ExecStreamingResult> {
let ExecStreamingRequest {
command,
timeout_ms,
working_dir,
env_vars,
cancel_token,
stdin,
output_callback,
stream_output_bytes_cap,
} = request;
let mut spec = ExecSpec::bash(command)
.no_timeout()
.stop_grace(self.stop_grace);
if let Some(timeout_ms) = timeout_ms {
spec = spec.timeout(Duration::from_millis(timeout_ms));
let mut spec = ExecSpec::bash(command).no_timeout();
if let Some(timeout) = timeout {
spec = spec.timeout(timeout);
}
if let Some(dir) = working_dir.or(self.working_dir.as_deref()) {
if let Some(dir) = working_dir {
spec = spec.working_dir(dir);
}
for (key, value) in self.explicit_env(env_vars) {
for (key, value) in env_vars.into_iter().flatten() {
spec = spec.env_var(key, value);
}
if let Some(bytes) = stdin {
spec = spec.stdin(bytes);
}
// The caller's cancellation is the `term` stop; the provider runs
// the grace and the `kill` itself.
let controls = ExecControls {
term: cancel_token,
kill: None,
stdin: None,
sink: output_callback.map(adapt_output_callback),
retained_output_limit: Some(
stream_output_bytes_cap.unwrap_or(DEFAULT_RETAINED_OUTPUT_BYTES),
),
term: cancel_token,
..ExecControls::default()
};
Ok(self.run_streaming(spec, controls).await?.result)
}
let streaming = self.exec.run_streaming(&spec, controls).await?;
let termination = map_termination(streaming.result.termination);
let duration_ms = duration_ms(streaming.result.duration);
Ok(ExecStreamingResult {
result: ExecResult {
stdout: String::from_utf8_lossy(&streaming.result.stdout).into_owned(),
stderr: String::from_utf8_lossy(&streaming.result.stderr).into_owned(),
exit_code: exit_code_for(termination, streaming.result.exit_code),
termination,
duration_ms,
},
streams_separated: streaming.streams_separated,
live_streaming: streaming.live_streaming,
stdout_capture: capture_stats(streaming.stdout_capture),
stderr_capture: capture_stats(streaming.stderr_capture),
})
/// Runs `spec` under fabro's policy, delivering output through
/// `controls.sink` as it arrives.
///
/// The policy fills what the spec leaves open: the stop grace, the
/// working directory, and the text output policy. The spec's environment
/// goes to the provider as the caller composed it. The caller's
/// `controls.term` is the `term` stop; the provider runs the grace and
/// the `kill` itself. Output beyond `controls.retained_output_limit`
/// (fabro's default when unset) is drained and counted, not kept.
pub async fn run_streaming(
&self,
spec: ExecSpec,
mut controls: ExecControls,
) -> crate::Result<ExecStreamingResult> {
let spec = self.apply_policy(spec);
if controls.retained_output_limit.is_none() {
controls.retained_output_limit = Some(DEFAULT_RETAINED_OUTPUT_BYTES);
}
Ok(self.exec.run_streaming(&spec, controls).await?)
}
/// Launches a long-lived process with bidirectional stdio.
///
/// `command` is evaluated under the same non-login Bash contract before
/// the shell replaces itself with the requested process. Cancelling
/// `cancel_token` terminates the process.
/// the shell replaces itself with the requested process. The returned
/// handle terminates the process; dropping it does not.
pub async fn spawn_stdio(
&self,
command: &str,
working_dir: Option<&str>,
env_vars: Option<&HashMap<String, String>>,
cancel_token: Option<CancellationToken>,
) -> crate::Result<StdioProcess> {
let mut spec = SpawnSpec::bash(format!("exec {command}"));
if let Some(dir) = working_dir.or(self.working_dir.as_deref()) {
spec = spec.working_dir(dir);
}
for (key, value) in self.explicit_env(env_vars) {
for (key, value) in env_vars.into_iter().flatten() {
spec = spec.env_var(key, value);
}
let process = self.exec.spawn_stdio(&spec).await?;
let handle = StdioProcessHandle::new(DriverStdioControl {
handle: Arc::from(process.handle),
});
if let Some(token) = cancel_token {
let handle = handle.clone();
tokio::spawn(async move {
token.cancelled().await;
if let Err(error) = handle.terminate().await {
tracing::warn!(error = %error, "failed to terminate stdio process on cancel");
}
});
}
Ok(StdioProcess {
stdin: process.stdin,
stdout: process.stdout,
stderr: StderrCollector::from_driver_tail(process.stderr_tail),
handle,
})
Ok(self.exec.spawn_stdio(&spec).await?)
}
/// The explicit environment after policy: `BASH_ENV` never passes,
/// because the Bash helper blanks it and a caller value would override
/// that; credential-shaped names pass only under `TrustCaller`.
fn explicit_env(&self, env_vars: Option<&HashMap<String, String>>) -> Vec<(String, String)> {
let mut entries: Vec<(String, String)> = env_vars
.into_iter()
.flatten()
.filter(|(key, _)| key.as_str() != BASH_ENV_VAR)
.filter(|(key, _)| {
self.env_policy == ExplicitEnvPolicy::TrustCaller || !is_sensitive_env_var(key)
})
.map(|(key, value)| (key.clone(), value.clone()))
.collect();
entries.sort();
entries
/// Fills what a spec leaves open. The output policy has no "unset"
/// state: the driver's default is raw, and fabro reads command output
/// as text, so a spec still at that default gets
/// [`OutputSanitization::StripAll`]; a caller that chose another policy
/// keeps it. Long-lived stdio processes ([`Self::spawn_stdio`]) and PTY
/// sessions stay raw, as the driver requires.
fn apply_policy(&self, mut spec: ExecSpec) -> ExecSpec {
if spec.stop_grace.is_none() {
spec.stop_grace = Some(self.stop_grace);
}
if spec.working_dir.is_none() {
spec.working_dir.clone_from(&self.working_dir);
}
if spec.output_sanitization == OutputSanitization::default() {
spec.output_sanitization = OutputSanitization::StripAll;
}
spec
}
}
/// The driver says how the command ended; fabro's vocabulary has two stops.
/// A timeout is the provider's deadline (the ladder ran for it); a
/// The driver says how the command ended; fabro's event vocabulary has two
/// stops. A timeout is the provider's deadline (the ladder ran for it); a
/// cancelled or killed command was stopped by the caller's token, by a
/// foreign `kill`, or by a provider-side abort — it did not finish and no
/// deadline passed.
fn map_termination(termination: Termination) -> CommandTermination {
/// deadline passed. `Exited`, or a provider that could not tell, is a
/// completed process; nothing asserts success here.
#[must_use]
pub fn command_termination(termination: Termination) -> CommandTermination {
match termination {
Termination::TimedOut => CommandTermination::TimedOut,
Termination::Cancelled | Termination::Killed => CommandTermination::Cancelled,
// `Exited`, or a provider that could not tell how the command ended.
// Nothing asserts success here: `exit_code` is whatever was observed
// and `is_success` still requires `Some(0)`.
_ => CommandTermination::Exited,
}
}
/// An exit code is only the command's own when it exited on its own. A
/// stopped command may still report the shell's `128 + signal` (143 for a
/// trapped `TERM`), which callers must not mistake for a program result.
fn exit_code_for(termination: CommandTermination, exit_code: Option<i32>) -> Option<i32> {
// `CommandTermination` is non-exhaustive: only a command that exited on
// its own owns its exit code.
matches!(termination, CommandTermination::Exited)
.then_some(exit_code)
.flatten()
}
fn capture_stats(stats: CaptureStats) -> OutputCaptureStats {
OutputCaptureStats {
observed_bytes: stats.observed_bytes,
retained_bytes: stats.retained_bytes,
omitted_bytes: stats.omitted_bytes,
/// trapped `TERM`), which events must not present as a program result.
#[must_use]
pub fn program_exit_code(termination: Termination, exit_code: Option<i32>) -> Option<i32> {
// `CommandTermination` is pebble's and non-exhaustive: only a command
// that exited on its own owns its exit code.
match command_termination(termination) {
CommandTermination::Exited => exit_code,
_ => None,
}
}
fn adapt_output_callback(callback: CommandOutputCallback) -> sandbox_driver::OutputSink {
Arc::new(move |stream, chunk| {
let stream = match stream {
OutputStream::Stdout => CommandOutputStream::Stdout,
OutputStream::Stderr => CommandOutputStream::Stderr,
};
let callback = Arc::clone(&callback);
Box::pin(async move {
callback(stream, chunk).await.map_err(|error| {
sandbox_driver::Error::Transport(TransportError::with_source(
"command output callback failed",
error,
))
})
})
})
/// Fabro's reading of a driver [`ExecResult`]: the event-facing numbers,
/// the redacted output tail, and the failure a non-zero exit is.
pub trait ExecResultExt {
/// The provider's measured run time in whole milliseconds.
fn duration_ms(&self) -> u64;
/// The exit code when the command ended on its own; see
/// [`program_exit_code`].
fn program_exit_code(&self) -> Option<i32>;
/// Redacted tails of both streams, each bounded to
/// `max_bytes_per_stream`. `None` when both streams are empty. Terminal
/// control sequences were already stripped by the driver under
/// [`SandboxExec`]'s output policy.
fn redacted_output_tail(&self, max_bytes_per_stream: usize) -> Option<ExecOutputTail>;
/// [`Self::redacted_output_tail`] at fabro's event budget.
fn default_redacted_output_tail(&self) -> Option<ExecOutputTail>;
/// The failure this result is, reported under `label`. The raw output
/// stays behind the driver's [`ExecFailure`] accessors; `Display`
/// carries only the label and the classified metadata.
fn into_exec_error(self, label: impl Into<String>) -> crate::Error;
/// `Ok(self)` for a clean exit, the failure under `label` otherwise.
fn into_result(self, label: impl Into<String>) -> crate::Result<ExecResult>;
}
/// The provider's measured run time in whole milliseconds.
fn duration_ms(duration: Duration) -> u64 {
u64::try_from(duration.as_millis()).unwrap_or(u64::MAX)
}
struct DriverStdioControl {
handle: Arc<dyn DriverStdioProcessHandle>,
}
#[async_trait]
impl StdioProcessControl for DriverStdioControl {
async fn terminate(&self) -> crate::Result<()> {
self.handle.terminate().await;
Ok(())
impl ExecResultExt for ExecResult {
fn duration_ms(&self) -> u64 {
u64::try_from(self.duration.as_millis()).unwrap_or(u64::MAX)
}
async fn wait(&self) -> crate::Result<StdioProcessTermination> {
let (termination, exit_code) = self.handle.wait().await;
let termination = map_termination(termination);
Ok(StdioProcessTermination {
termination,
exit_code: exit_code_for(termination, exit_code),
})
fn program_exit_code(&self) -> Option<i32> {
program_exit_code(self.termination, self.exit_code)
}
fn redacted_output_tail(&self, max_bytes_per_stream: usize) -> Option<ExecOutputTail> {
redacted_output_tail(
&self.stdout_lossy(),
&self.stderr_lossy(),
max_bytes_per_stream,
)
}
fn default_redacted_output_tail(&self) -> Option<ExecOutputTail> {
self.redacted_output_tail(DEFAULT_EXEC_OUTPUT_TAIL_BYTES)
}
fn into_exec_error(self, label: impl Into<String>) -> crate::Error {
let failure = ExecFailure::new(
label,
self.termination,
self.exit_code,
self.stdout,
self.stderr,
)
.with_duration(self.duration);
crate::Error::from(sandbox_driver::Error::from(failure))
}
fn into_result(self, label: impl Into<String>) -> crate::Result<ExecResult> {
if self.success() {
Ok(self)
} else {
Err(self.into_exec_error(label))
}
}
}
@ -359,7 +284,10 @@ mod tests {
use std::sync::{Arc, Mutex};
use std::time::Instant;
use sandbox_driver::{SandboxProvider as _, SandboxSource, SandboxSpec};
use sandbox_driver::{
BASH_ENV_VAR, OutputSink, OutputStream, SandboxProvider as _, SandboxSource, SandboxSpec,
TransportError,
};
use sandbox_driver_host::HostProvider;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::{fs, time};
@ -391,30 +319,44 @@ mod tests {
}
}
fn exec(&self, policy: ExplicitEnvPolicy) -> SandboxExec<'_> {
fn exec(&self) -> SandboxExec<'_> {
let _ = &self.provider;
SandboxExec::new(self.sandbox.exec(), policy)
SandboxExec::new(self.sandbox.exec())
}
}
async fn run(fixture: &HostFixture, command: &str) -> ExecResult {
fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.exec()
.run(command, Some(Duration::from_secs(10)), None, None, None)
.await
.unwrap()
}
fn exec_result(
stdout: &str,
stderr: &str,
exit_code: Option<i32>,
termination: Termination,
duration_ms: u64,
) -> ExecResult {
let mut result =
ExecResult::new(termination, exit_code, Duration::from_millis(duration_ms));
result.stdout = stdout.as_bytes().to_vec();
result.stderr = stderr.as_bytes().to_vec();
result
}
#[tokio::test]
async fn runs_bash_source_and_reports_exit_code_and_streams() {
let fixture = HostFixture::new().await;
let result = run(&fixture, "echo out; echo err >&2; exit 3").await;
assert_eq!(result.stdout, "out\n");
assert_eq!(result.stderr, "err\n");
assert_eq!(result.stdout_lossy(), "out\n");
assert_eq!(result.stderr_lossy(), "err\n");
assert_eq!(result.exit_code, Some(3));
assert_eq!(result.termination, CommandTermination::Exited);
assert!(!result.is_success());
assert!(run(&fixture, "true").await.is_success());
assert_eq!(result.termination, Termination::Exited);
assert!(!result.success());
assert!(run(&fixture, "true").await.success());
}
#[tokio::test]
@ -426,7 +368,7 @@ mod tests {
set -o | grep -E '^(errexit|pipefail)' | awk '{print $2}' | sort -u",
)
.await;
assert_eq!(result.stdout, "nonlogin\noff\n", "{result:?}");
assert_eq!(result.stdout_lossy(), "nonlogin\noff\n", "{result:?}");
}
#[tokio::test]
@ -438,7 +380,7 @@ mod tests {
.unwrap();
let env = HashMap::from([(BASH_ENV_VAR.to_string(), startup.display().to_string())]);
let result = fixture
.exec(ExplicitEnvPolicy::TrustCaller)
.exec()
.run(
"echo body",
Some(Duration::from_secs(10)),
@ -448,47 +390,24 @@ mod tests {
)
.await
.unwrap();
assert_eq!(result.stdout, "body\n");
assert_eq!(result.stdout_lossy(), "body\n");
}
#[tokio::test]
async fn filter_sensitive_drops_credential_shaped_explicit_variables() {
async fn explicit_variables_reach_the_command_as_composed() {
let fixture = HostFixture::new().await;
let env = HashMap::from([
("FABRO_WORKER_TOKEN".to_string(), "leaked".to_string()),
("FABRO_WORKER_TOKEN".to_string(), "deliberate".to_string()),
("MY_VAR".to_string(), "ok".to_string()),
]);
let filtered = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
let stdout = fixture
.exec()
.run("env", Some(Duration::from_secs(10)), None, Some(&env), None)
.await
.unwrap();
assert!(!filtered.stdout.contains("FABRO_WORKER_TOKEN=leaked"));
assert!(filtered.stdout.contains("MY_VAR=ok"));
let trusted = fixture
.exec(ExplicitEnvPolicy::TrustCaller)
.run("env", Some(Duration::from_secs(10)), None, Some(&env), None)
.await
.unwrap();
assert!(trusted.stdout.contains("FABRO_WORKER_TOKEN=leaked"));
}
#[test]
fn sensitive_name_classification_matches_the_worker_policy() {
for key in [
"OPENAI_API_KEY",
"DB_PASSWORD",
"AWS_SECRET",
"AUTH_TOKEN",
"MY_CREDENTIAL",
"FABRO_WORKER_TOKEN",
] {
assert!(is_sensitive_env_var(key), "{key}");
}
for key in ["PATH", "HOME", "MY_VAR", "GITHUB_ACTOR"] {
assert!(!is_sensitive_env_var(key), "{key}");
}
.unwrap()
.stdout_lossy();
assert!(stdout.contains("FABRO_WORKER_TOKEN=deliberate"), "{stdout}");
assert!(stdout.contains("MY_VAR=ok"), "{stdout}");
}
#[tokio::test]
@ -496,7 +415,7 @@ mod tests {
let fixture = HostFixture::new().await;
let started = Instant::now();
let result = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.exec()
.run(
"sleep 10",
Some(Duration::from_millis(200)),
@ -506,8 +425,8 @@ mod tests {
)
.await
.unwrap();
assert_eq!(result.termination, CommandTermination::TimedOut);
assert_eq!(result.exit_code, None);
assert_eq!(result.termination, Termination::TimedOut);
assert_eq!(result.program_exit_code(), None);
assert!(
started.elapsed() < Duration::from_secs(5),
"sleep honours TERM, so KILL should not have been needed"
@ -519,7 +438,7 @@ mod tests {
let fixture = HostFixture::new().await;
let started = Instant::now();
let result = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.exec()
.with_stop_grace(Duration::from_millis(300))
.run(
"trap '' TERM; sleep 10",
@ -530,7 +449,7 @@ mod tests {
)
.await
.unwrap();
assert_eq!(result.termination, CommandTermination::TimedOut);
assert_eq!(result.termination, Termination::TimedOut);
let elapsed = started.elapsed();
assert!(elapsed >= Duration::from_millis(400), "{elapsed:?}");
assert!(elapsed < Duration::from_secs(5), "{elapsed:?}");
@ -546,7 +465,7 @@ mod tests {
cancel.cancel();
});
let result = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.exec()
.run(
"sleep 10",
Some(Duration::from_secs(30)),
@ -556,8 +475,8 @@ mod tests {
)
.await
.unwrap();
assert_eq!(result.termination, CommandTermination::Cancelled);
assert_eq!(result.exit_code, None);
assert_eq!(result.termination, Termination::Cancelled);
assert_eq!(result.program_exit_code(), None);
}
#[tokio::test]
@ -565,33 +484,36 @@ mod tests {
let fixture = HostFixture::new().await;
let seen = Arc::new(Mutex::new(Vec::<u8>::new()));
let sink_seen = Arc::clone(&seen);
let callback: CommandOutputCallback = Arc::new(move |stream, chunk| {
let sink: OutputSink = Arc::new(move |stream, chunk| {
let seen = Arc::clone(&sink_seen);
Box::pin(async move {
assert_eq!(stream, CommandOutputStream::Stdout);
assert_eq!(stream, OutputStream::Stdout);
seen.lock().unwrap().extend_from_slice(&chunk);
Ok(())
})
});
let streaming = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.run_streaming(ExecStreamingRequest {
timeout_ms: Some(10_000),
output_callback: Some(callback),
stream_output_bytes_cap: Some(64),
..ExecStreamingRequest::new("for i in $(seq 1 200); do echo line-$i; done")
})
.exec()
.run_streaming(
ExecSpec::bash("for i in $(seq 1 200); do echo line-$i; done")
.timeout(Duration::from_secs(10)),
ExecControls {
sink: Some(sink),
retained_output_limit: Some(64),
..ExecControls::default()
},
)
.await
.unwrap();
assert!(streaming.result.is_success());
assert!(streaming.result.success());
assert!(streaming.live_streaming);
assert!(streaming.streams_separated);
let delivered = seen.lock().unwrap().len();
assert_eq!(streaming.stdout_capture.observed_bytes, delivered);
assert!(streaming.stdout_capture.omitted_bytes > 0);
assert!(streaming.result.stdout.len() <= 64);
assert!(streaming.result.stdout.starts_with("line-1\n"));
assert!(streaming.result.stdout.ends_with("line-200\n"));
assert!(streaming.result.stdout.starts_with(b"line-1\n"));
assert!(streaming.result.stdout.ends_with(b"line-200\n"));
}
#[tokio::test]
@ -599,35 +521,43 @@ mod tests {
let fixture = HostFixture::new().await;
let stdin = b"first line\n$(touch must-not-run)\nlast line".to_vec();
let streaming = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.run_streaming(ExecStreamingRequest {
timeout_ms: Some(10_000),
stdin: Some(stdin.clone()),
..ExecStreamingRequest::new("cat; test -e must-not-run && echo RAN")
})
.exec()
.run_streaming(
ExecSpec::bash("cat; test -e must-not-run && echo RAN")
.timeout(Duration::from_secs(10))
.stdin(stdin.clone()),
ExecControls::default(),
)
.await
.unwrap();
assert_eq!(streaming.result.stdout.as_bytes(), stdin.as_slice());
assert_eq!(streaming.result.stdout, stdin);
}
#[tokio::test]
async fn a_failing_output_callback_stops_the_command_with_an_error() {
async fn a_failing_output_sink_stops_the_command_with_an_error() {
let fixture = HostFixture::new().await;
let callback: CommandOutputCallback =
Arc::new(|_, _| Box::pin(async { Err(crate::Error::message("consumer gave up")) }));
let error = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.run_streaming(ExecStreamingRequest {
timeout_ms: Some(10_000),
output_callback: Some(callback),
..ExecStreamingRequest::new("echo hello; sleep 5")
let sink: OutputSink = Arc::new(|_, _| {
Box::pin(async {
Err(sandbox_driver::Error::Transport(TransportError::new(
"consumer gave up",
)))
})
});
let error = fixture
.exec()
.run_streaming(
ExecSpec::bash("echo hello; sleep 5").timeout(Duration::from_secs(10)),
ExecControls {
sink: Some(sink),
..ExecControls::default()
},
)
.await
.map(|streaming| streaming.result.termination);
// The driver either surfaces the sink failure or reports the command
// cancelled by it; both keep the consumer's error visible.
match error {
Ok(termination) => assert_eq!(termination, CommandTermination::Cancelled),
Ok(termination) => assert_eq!(termination, Termination::Cancelled),
Err(error) => assert!(error.to_string().contains("consumer gave up"), "{error}"),
}
}
@ -635,11 +565,7 @@ mod tests {
#[tokio::test]
async fn stdio_process_round_trips_lines_and_reports_exit() {
let fixture = HostFixture::new().await;
let process = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.spawn_stdio("cat", None, None, None)
.await
.unwrap();
let process = fixture.exec().spawn_stdio("cat", None, None).await.unwrap();
let mut stdin = process.stdin;
let mut stdout = BufReader::new(process.stdout);
stdin.write_all(b"ping\n").await.unwrap();
@ -647,52 +573,166 @@ mod tests {
stdout.read_line(&mut line).await.unwrap();
assert_eq!(line, "ping\n");
drop(stdin);
let termination = process.handle.wait().await.unwrap();
assert_eq!(termination.termination, CommandTermination::Exited);
assert_eq!(termination.exit_code, Some(0));
let (termination, exit_code) = process.handle.wait().await;
assert_eq!(termination, Termination::Exited);
assert_eq!(exit_code, Some(0));
}
#[tokio::test]
async fn stdio_process_terminates_on_cancel_and_keeps_a_stderr_tail() {
async fn stdio_process_terminates_on_request_and_keeps_a_stderr_tail() {
let fixture = HostFixture::new().await;
let token = CancellationToken::new();
let process = fixture
.exec(ExplicitEnvPolicy::FilterSensitive)
.spawn_stdio(
"sh -c 'echo diag >&2; sleep 30'",
None,
None,
Some(token.clone()),
)
.exec()
.spawn_stdio("sh -c 'echo diag >&2; sleep 30'", None, None)
.await
.unwrap();
time::sleep(Duration::from_millis(200)).await;
token.cancel();
let termination = time::timeout(Duration::from_secs(5), process.handle.wait())
process.handle.terminate().await;
let (termination, _) = time::timeout(Duration::from_secs(5), process.handle.wait())
.await
.expect("cancel terminates the process")
.unwrap();
assert_ne!(termination.termination, CommandTermination::Exited);
assert_eq!(process.stderr.tail_string().await, "diag\n");
.expect("terminate ends the process");
assert_ne!(termination, Termination::Exited);
assert_eq!(process.stderr_tail.to_string_lossy(), "diag\n");
}
#[test]
fn termination_mapping_reads_the_drivers_verdict() {
assert_eq!(
map_termination(Termination::TimedOut),
command_termination(Termination::TimedOut),
CommandTermination::TimedOut
);
assert_eq!(
map_termination(Termination::Cancelled),
command_termination(Termination::Cancelled),
CommandTermination::Cancelled
);
assert_eq!(
map_termination(Termination::Killed),
command_termination(Termination::Killed),
CommandTermination::Cancelled
);
assert_eq!(
map_termination(Termination::Exited),
command_termination(Termination::Exited),
CommandTermination::Exited
);
}
#[test]
fn program_exit_code_is_the_commands_own_only_when_it_exited() {
assert_eq!(program_exit_code(Termination::Exited, Some(3)), Some(3));
assert_eq!(program_exit_code(Termination::TimedOut, Some(143)), None);
assert_eq!(program_exit_code(Termination::Cancelled, Some(143)), None);
assert_eq!(program_exit_code(Termination::Killed, Some(137)), None);
}
#[test]
fn into_result_reports_a_failure_under_its_label() {
let result = exec_result(
"out",
"fatal: could not read Username",
Some(128),
Termination::Exited,
42,
);
let error = result.into_result("git push").unwrap_err();
let Some(sandbox_driver::Error::Exec(failure)) = error.driver() else {
panic!("expected an exec failure, got {error:?}");
};
assert_eq!(failure.label(), "git push");
assert_eq!(failure.exit_code(), Some(128));
assert_eq!(failure.duration(), Some(Duration::from_millis(42)));
assert!(
!error.to_string().contains("could not read Username"),
"raw output leaked into Display: {error}"
);
let ok = exec_result("out", "", Some(0), Termination::Exited, 1);
assert!(ok.into_result("true").is_ok());
}
#[test]
fn output_tail_redacts_before_truncating() {
let secret = "sk-ant-api03-xK9mZ2vL8nQ5rT1wY4bC7dF0gH3jE6pA";
let result = exec_result(
&format!("{} {secret} done", "context ".repeat(20)),
"",
Some(1),
Termination::Exited,
1,
);
let tail = result
.redacted_output_tail(32)
.expect("redacted output tail");
let stdout = tail.stdout.expect("stdout tail");
assert!(stdout.contains("REDACTED"), "{stdout}");
assert!(!stdout.contains("F0gH3jE6pA"), "{stdout}");
assert!(tail.stdout_truncated);
}
#[tokio::test]
async fn command_output_arrives_stripped_of_terminal_control_sequences() {
let fixture = HostFixture::new().await;
let result = run(
&fixture,
"printf '\\033[31mred\\033[0m \\033]0;window-title\\007shown \\033(Bset \\033Mtwo-byte \
\\bbackspace'",
)
.await;
assert!(result.success(), "{result:?}");
assert_eq!(result.stdout_lossy(), "red shown set two-byte backspace");
let tail = result
.redacted_output_tail(1024)
.expect("redacted output tail");
assert_eq!(
tail.stdout.as_deref(),
Some("red shown set two-byte backspace")
);
}
#[tokio::test]
async fn policy_strips_output_unless_the_caller_chose_another_policy() {
let fixture = HostFixture::new().await;
let exec = fixture.exec();
assert_eq!(
exec.apply_policy(ExecSpec::bash("true"))
.output_sanitization,
OutputSanitization::StripAll
);
assert_eq!(
exec.apply_policy(
ExecSpec::bash("true").output_sanitization(OutputSanitization::StripAnsi)
)
.output_sanitization,
OutputSanitization::StripAnsi
);
}
#[test]
fn default_output_tail_serialized_budget_stays_below_40_kib() {
let result = exec_result(
&"o".repeat(DEFAULT_EXEC_OUTPUT_TAIL_BYTES + 128),
&"e".repeat(DEFAULT_EXEC_OUTPUT_TAIL_BYTES + 128),
Some(1),
Termination::Exited,
1,
);
let tail = result.default_redacted_output_tail().expect("tail present");
assert_eq!(
tail.stdout.as_deref().map(str::len),
Some(DEFAULT_EXEC_OUTPUT_TAIL_BYTES)
);
assert_eq!(
tail.stderr.as_deref().map(str::len),
Some(DEFAULT_EXEC_OUTPUT_TAIL_BYTES)
);
assert!(tail.stdout_truncated);
assert!(tail.stderr_truncated);
let serialized = serde_json::to_vec(&tail).expect("serialize tail");
assert!(
serialized.len() < 40 * 1024,
"tail JSON was {} bytes",
serialized.len()
);
}
}

278
lib/components/fabro-sandbox/src/git_policy.rs Normal file
View file

@ -0,0 +1,278 @@
//! Fabro's retry budgets for git operations against GitHub.
//!
//! The driver owns the retry loop and the decision
//! ([`sandbox_driver::retry_git`]): a remote that cannot be reached is retried,
//! a rejected credential is retried only while the token is fresh enough to
//! still be replicating to GitHub's git endpoints, a static credential fails
//! fast, and a command whose outcome is unknown is never replayed. Fabro keeps
//! what is policy: how many attempts each operation gets, how long the
//! operation may take, and when the credential it pushes with was minted.
//!
//! Retries reuse the same token on purpose. Replication of a given token
//! only makes progress, so each attempt strictly improves the odds, while
//! re-minting would restart the replication clock.
use std::future::Future;
use std::sync::{Mutex, PoisonError};
use std::time::{Duration, SystemTime};
use fabro_github::token_source::TokenSnapshot;
pub use fabro_types::run_event::GitPushRetryReason as GitRetryReason;
use sandbox_driver::{GitBackoff, GitCredentials, GitFailure, GitFailureKind, GitRetryPolicy};
use crate::credentials::GITHUB_TOKEN_USERNAME;
/// Backoff between attempts: 3s, then 9s.
///
/// GitHub's guidance for token replication is to wait a few seconds and
/// retry with the same token. Sub-second delays land inside the same
/// replication window and spend an attempt for nothing.
fn replication_backoff() -> GitBackoff {
GitBackoff::new(Duration::from_secs(3), 3.0, Duration::from_secs(10))
}
/// The clone policy: 3 attempts at replication pacing, inside whatever is
/// left of the whole-clone budget.
pub(crate) fn clone_policy(remaining: Duration) -> GitRetryPolicy {
GitRetryPolicy::new(3, replication_backoff()).max_elapsed(remaining)
}
/// Host-side repository probes use the clone's attempt count and pacing,
/// with no deadline of their own.
#[must_use]
pub fn repository_probe_policy() -> GitRetryPolicy {
GitRetryPolicy::new(3, replication_backoff())
}
/// Checkpoint pushes stay cheap: the next checkpoint re-pushes the same
/// branch anyway. Worst case about 90 seconds of wall clock.
#[must_use]
pub fn checkpoint_push_policy() -> GitRetryPolicy {
GitRetryPolicy::new(3, replication_backoff())
.max_elapsed(Duration::from_secs(90))
.per_attempt_timeout(Duration::from_mins(1))
}
/// The terminal publish push guards the whole run's value, so it gets a
/// real budget: 5 attempts with growing backoff (about 3s, 10s, 33s, 60s),
/// bounded at 4 minutes of wall clock. The bound must stay under the token
/// source's `REFRESH_MARGIN` (see the margin-invariant test) so a pinned
/// token always outlives the operation.
#[must_use]
pub fn publish_push_policy() -> GitRetryPolicy {
GitRetryPolicy::new(
5,
GitBackoff::new(Duration::from_secs(3), 10.0 / 3.0, Duration::from_mins(1)),
)
.max_elapsed(Duration::from_mins(4))
.per_attempt_timeout(Duration::from_mins(1))
}
/// The reason fabro records for a driver retry reason. A reason this build
/// does not know still retried the attempt, so it is recorded under the
/// broader class.
pub(crate) fn recorded_reason(reason: sandbox_driver::GitRetryReason) -> GitRetryReason {
match reason {
sandbox_driver::GitRetryReason::TokenReplication => GitRetryReason::TokenReplication,
_ => GitRetryReason::TransientInfra,
}
}
/// Credentials carrying only the token's mint time, which is all the
/// driver's decision reads for git that ran outside a sandbox. The token
/// itself never leaves its snapshot.
fn credential_age(snapshot: Option<&TokenSnapshot>) -> Option<GitCredentials> {
let snapshot = snapshot?;
let credentials = GitCredentials::new(GITHUB_TOKEN_USERNAME, "");
Some(match snapshot.minted_at() {
Some(minted_at) => credentials.minted_at(SystemTime::from(minted_at)),
None => credentials,
})
}
/// The driver's failure for a rendered git message, so git that ran
/// outside a sandbox (the host-side repository probe, the metadata push)
/// is classified the same way as git the driver ran.
fn classified_failure(operation: &str, message: &str) -> sandbox_driver::Error {
sandbox_driver::Error::Git(GitFailure::classified(
operation,
GitFailureKind::from_message(message),
None,
))
}
/// Whether a rendered git failure `message` is worth retrying with the
/// token behind `snapshot`: `None` means the failure is permanent for
/// these credentials or unrecognized.
#[must_use]
pub fn transient_git_failure(
message: &str,
snapshot: Option<&TokenSnapshot>,
) -> Option<GitRetryReason> {
let credentials = credential_age(snapshot);
sandbox_driver::retry_reason(&classified_failure("git", message), credentials.as_ref())
.map(recorded_reason)
}
/// Runs a host-side git operation that reports failures as rendered
/// messages under `policy`, retrying while the driver's decision says the
/// message is transient for the token behind `snapshot`. The final failure
/// comes back as the operation's own message.
pub async fn retry_git_messages<F, Fut>(
policy: &GitRetryPolicy,
snapshot: Option<&TokenSnapshot>,
operation: &str,
mut run: F,
) -> Result<(), String>
where
F: FnMut() -> Fut,
Fut: Future<Output = Result<(), String>>,
{
let credentials = credential_age(snapshot);
// The operation's own message is kept beside the classified failure the
// driver decides on, so the caller reads the message it knows.
let last_message = Mutex::new(None);
let result = sandbox_driver::retry_git(
policy,
credentials.as_ref(),
operation,
|_attempt, _timeout| {
let attempt = run();
let last_message = &last_message;
async move {
attempt.await.map_err(|message| {
let error = classified_failure(operation, &message);
*last_message.lock().unwrap_or_else(PoisonError::into_inner) = Some(message);
error
})
}
},
)
.await;
match result {
Ok(_) => Ok(()),
Err(failure) => Err(last_message
.into_inner()
.unwrap_or_else(PoisonError::into_inner)
.unwrap_or_else(|| failure.error.to_string())),
}
}
#[cfg(test)]
mod tests {
use chrono::Utc;
use fabro_github::token_source::{REFRESH_MARGIN, TokenProvenance};
use super::*;
fn snapshot(age: Duration) -> TokenSnapshot {
let now = Utc::now();
TokenSnapshot {
generation: 1,
provenance: TokenProvenance::Minted {
minted_at: now - chrono::Duration::from_std(age).unwrap(),
expires_at: now + chrono::Duration::hours(1),
},
}
}
fn static_snapshot() -> TokenSnapshot {
TokenSnapshot {
generation: 0,
provenance: TokenProvenance::Static,
}
}
#[test]
fn not_found_follows_the_credential_age() {
let message = "repository not found: Repository not found.";
assert_eq!(
transient_git_failure(message, Some(&snapshot(Duration::from_secs(5)))),
Some(GitRetryReason::TokenReplication)
);
assert_eq!(
transient_git_failure(message, Some(&snapshot(Duration::from_mins(2)))),
Some(GitRetryReason::TransientInfra)
);
assert_eq!(
transient_git_failure(message, Some(&static_snapshot())),
None
);
assert_eq!(transient_git_failure(message, None), None);
}
#[test]
fn infrastructure_failures_retry_without_credentials() {
assert_eq!(
transient_git_failure("fatal: unable to access: Could not resolve host", None),
Some(GitRetryReason::TransientInfra)
);
assert_eq!(
transient_git_failure("fatal: something else entirely", None),
None
);
}
/// `REFRESH_MARGIN` must exceed every push policy's `max_elapsed`: a
/// push resolves its token once, and the token the source returns has
/// at least the margin of validity left, so the pinned token outlives
/// the operation.
#[test]
fn refresh_margin_exceeds_every_push_policy_elapsed_bound() {
for policy in [checkpoint_push_policy(), publish_push_policy()] {
let max_elapsed = policy.max_elapsed.expect("push policies are bounded");
assert!(
REFRESH_MARGIN > max_elapsed,
"margin invariant violated: {max_elapsed:?}"
);
}
}
#[test]
fn publish_backoff_grows_toward_a_one_minute_cap() {
let backoff = publish_push_policy().backoff;
assert_eq!(backoff.delay_after(1), Duration::from_secs(3));
assert_eq!(backoff.delay_after(2), Duration::from_secs(10));
assert_eq!(backoff.delay_after(4), Duration::from_mins(1));
assert_eq!(
repository_probe_policy().backoff.delay_after(2),
Duration::from_secs(9)
);
}
#[tokio::test(start_paused = true)]
async fn host_side_retries_keep_the_operations_own_message() {
let calls = Mutex::new(0_u32);
let result = retry_git_messages(
&repository_probe_policy(),
Some(&snapshot(Duration::from_secs(1))),
"repository probe",
|| {
let attempt = {
let mut calls = calls.lock().unwrap();
*calls += 1;
*calls
};
async move {
if attempt < 3 {
Err(format!("remote: Repository not found. (attempt {attempt})"))
} else {
Ok(())
}
}
},
)
.await;
assert_eq!(result, Ok(()));
assert_eq!(*calls.lock().unwrap(), 3);
let permanent = retry_git_messages(
&repository_probe_policy(),
Some(&static_snapshot()),
"repository probe",
|| async { Err("remote: Repository not found.".to_owned()) },
)
.await;
assert_eq!(permanent, Err("remote: Repository not found.".to_owned()));
}
}

719
lib/components/fabro-sandbox/src/git_retry.rs
View file

@ -1,719 +0,0 @@
//! Retry for git operations against GitHub from clone-based sandboxes.
//!
//! Clone-based providers can mint a GitHub App installation token and use it
//! immediately. GitHub can reject that first operation before the token is
//! available to the git endpoint. On a private repository, the rejection can
//! arrive as `Repository not found.` or an authentication failure.
//!
//! Only a token minted recently makes those messages safe to retry. Static
//! PATs and pre-minted installation tokens fail fast; a mature App token can
//! still hit a service-side blip that presents the same surface, so it
//! retries as transient infrastructure.
//!
//! Retries reuse the same token on purpose. Replication of a given token only
//! makes progress, so each attempt strictly improves the odds, while
//! re-minting would restart the replication clock.
//!
//! The driver classifies what a failure was ([`GitFailureKind`]); this module
//! decides what the class means for the credentials in hand.
use std::future::Future;
use std::time::Duration;
use chrono::Utc;
use fabro_github::token_source::TokenSnapshot;
#[cfg(test)]
use fabro_github::token_source::{REFRESH_MARGIN, TokenProvenance};
use fabro_types::SandboxProviderKind;
pub use fabro_types::run_event::GitPushRetryReason as GitRetryReason;
use fabro_util::backoff::BackoffPolicy;
use sandbox_driver::GitFailureKind;
use tokio::time;
/// How long after its mint a token is presumed to still be replicating to
/// GitHub's git endpoints. Matches the observed scale of the lag (seconds,
/// occasionally tens of seconds).
pub(crate) const REPLICATION_HORIZON: Duration = Duration::from_mins(1);
/// What a git failure message tells us about retry safety.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum GitMessageClass {
Retry(GitRetryReason),
Permanent,
Unknown,
}
impl GitMessageClass {
pub(crate) fn retry_reason(self) -> Option<GitRetryReason> {
match self {
Self::Retry(reason) => Some(reason),
Self::Permanent | Self::Unknown => None,
}
}
}
/// What the operation's credentials say about retrying auth-shaped failures.
///
/// Derived from the [`TokenSnapshot`] of the token embedded for the attempt,
/// so classification reads provenance as data instead of threading booleans
/// through call stacks.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum CredentialContext {
/// An installation token younger than [`REPLICATION_HORIZON`] — a 404 or
/// auth failure is likely replication lag; retry with the same token.
FreshApp,
/// An installation token older than the horizon. A 404 with it is
/// indistinguishable from a service-side blip at this layer, so it stays
/// transient rather than proving access loss.
MatureApp,
/// A PAT or pre-minted token — it cannot become valid by waiting.
Static,
/// No credentials at all.
None,
}
impl CredentialContext {
#[must_use]
pub fn from_snapshot(snapshot: Option<&TokenSnapshot>) -> Self {
match snapshot {
None => Self::None,
Some(snapshot) => match snapshot.age_at(Utc::now()) {
None => Self::Static,
Some(age) if age < REPLICATION_HORIZON => Self::FreshApp,
Some(_) => Self::MatureApp,
},
}
}
}
/// What a classified git failure means for retrying with these credentials.
///
/// The driver reads the failure; fabro decides. A remote that could not
/// be reached is retried whatever the credential. A rejected credential
/// is retried only while a just-minted App token may still be replicating
/// (`FreshApp`), retried as a service blip for a mature App token, and
/// fails fast for a static credential or none, because waiting cannot make
/// those valid. Every other class is permanent.
pub(crate) fn decide(kind: GitFailureKind, cred: CredentialContext) -> GitMessageClass {
match kind {
GitFailureKind::RemoteUnavailable => GitMessageClass::Retry(GitRetryReason::TransientInfra),
GitFailureKind::AuthRejected => match cred {
CredentialContext::FreshApp => GitMessageClass::Retry(GitRetryReason::TokenReplication),
CredentialContext::MatureApp => GitMessageClass::Retry(GitRetryReason::TransientInfra),
CredentialContext::Static | CredentialContext::None => GitMessageClass::Permanent,
},
GitFailureKind::AccessDenied
| GitFailureKind::RefNotFound
| GitFailureKind::TargetExists => GitMessageClass::Permanent,
_ => GitMessageClass::Unknown,
}
}
/// Classify a failed git operation by its rendered message. For git that
/// ran outside a sandbox — the host-side repository probe and metadata
/// push — where the driver never saw the failure.
pub(crate) fn classify_message(message: &str, cred: CredentialContext) -> GitMessageClass {
decide(GitFailureKind::from_message(message), cred)
}
/// Classify a rendered git failure message, returning the retry reason when
/// the failure is transient for these credentials. `None` means the failure
/// is permanent or unrecognized.
#[must_use]
pub fn classify_failure(message: &str, cred: CredentialContext) -> Option<GitRetryReason> {
classify_message(message, cred).retry_reason()
}
/// Classify a sandbox-driver git failure.
///
/// The driver classifies every git failure it produces; fabro only decides
/// what the class means for these credentials. An operation whose outcome
/// is unknown (a transport break, a timeout, an incomplete operation) is
/// never retried: replaying it could overlap a clone that is still running.
#[must_use]
pub(crate) fn classify_driver_failure(
error: &sandbox_driver::Error,
cred: CredentialContext,
) -> Option<GitRetryReason> {
match error {
sandbox_driver::Error::Git(failure) => decide(failure.kind(), cred).retry_reason(),
sandbox_driver::Error::RateLimited { .. } | sandbox_driver::Error::Overloaded { .. } => {
Some(GitRetryReason::TransientInfra)
}
_ => None,
}
}
/// Backoff between attempts: 3s, then 9s.
///
/// GitHub's guidance for token replication is to wait a few seconds and retry
/// with the same token. Sub-second delays land inside the same replication
/// window and spend an attempt for nothing.
fn replication_backoff() -> BackoffPolicy {
BackoffPolicy {
initial_delay: Duration::from_secs(3),
factor: 3.0,
max_delay: Duration::from_secs(10),
jitter: false,
}
}
/// Attempt and time bounds for one retried git operation.
///
/// All bounds are optional so existing behaviors are expressible unchanged.
/// The effective deadline is the minimum of the bounds that are present
/// (`start + max_elapsed`, `outer_deadline`); each attempt runs with
/// `min(per_attempt_timeout, remaining)` over the caps that are present, and
/// no attempt or backoff starts past the effective deadline.
#[derive(Debug, Clone)]
pub struct RetryPlan {
/// Total attempts, including the first.
pub max_attempts: u32,
pub backoff: BackoffPolicy,
/// Wall clock for this whole operation.
pub max_elapsed: Option<Duration>,
/// Cap for any single attempt.
pub per_attempt_timeout: Option<Duration>,
/// Caller-supplied absolute bound.
pub outer_deadline: Option<time::Instant>,
}
impl RetryPlan {
/// Host-side repository probes use the same attempt count and pacing as
/// clone operations against a freshly minted token.
#[must_use]
pub fn repository_probe() -> Self {
Self::clone_default(None)
}
/// The clone policy both providers already trust: 3 attempts, 3s/9s
/// backoff, no plan-level bounds. Docker supplies its existing absolute
/// five-minute deadline through `outer_deadline`; Daytona supplies none.
#[must_use]
pub fn clone_default(outer_deadline: Option<time::Instant>) -> Self {
Self {
max_attempts: 3,
backoff: replication_backoff(),
max_elapsed: None,
per_attempt_timeout: None,
outer_deadline,
}
}
/// Checkpoint pushes stay cheap: the next checkpoint re-pushes the same
/// branch anyway. Worst case ~90 seconds of wall clock.
#[must_use]
pub fn checkpoint_push() -> Self {
Self {
max_attempts: 3,
backoff: replication_backoff(),
max_elapsed: Some(Duration::from_secs(90)),
per_attempt_timeout: Some(Duration::from_mins(1)),
outer_deadline: None,
}
}
/// The terminal publish push guards the whole run's value, so it gets a
/// real budget: 5 attempts with growing backoff (~3s/10s/33s/60s),
/// bounded at 4 minutes of wall clock. The 4-minute bound must stay
/// under the token source's `REFRESH_MARGIN` (see the margin-invariant
/// test) so a pinned token always outlives the operation.
#[must_use]
pub fn publish_push() -> Self {
Self {
max_attempts: 5,
backoff: BackoffPolicy {
initial_delay: Duration::from_secs(3),
factor: 10.0 / 3.0,
max_delay: Duration::from_mins(1),
jitter: false,
},
max_elapsed: Some(Duration::from_mins(4)),
per_attempt_timeout: Some(Duration::from_mins(1)),
outer_deadline: None,
}
}
/// The absolute deadline this operation must finish by, if any bound is
/// present.
pub(crate) fn effective_deadline(&self, start: time::Instant) -> Option<time::Instant> {
let elapsed_deadline = self.max_elapsed.map(|max| start + max);
match (elapsed_deadline, self.outer_deadline) {
(Some(a), Some(b)) => Some(a.min(b)),
(Some(a), None) => Some(a),
(None, Some(b)) => Some(b),
(None, None) => None,
}
}
/// Time cap for an attempt starting now: the per-attempt cap bounded by
/// the time remaining before the effective deadline.
pub(crate) fn attempt_timeout(&self, deadline: Option<time::Instant>) -> Option<Duration> {
let remaining = deadline.map(|d| d.saturating_duration_since(time::Instant::now()));
match (self.per_attempt_timeout, remaining) {
(Some(cap), Some(remaining)) => Some(cap.min(remaining)),
(Some(cap), None) => Some(cap),
(None, remaining) => remaining,
}
}
pub(crate) fn retry_delay(
&self,
attempt_number: u32,
deadline: Option<time::Instant>,
) -> Option<Duration> {
let delay = self.backoff.delay_for_attempt(attempt_number);
if deadline.is_some_and(|deadline| {
delay >= deadline.saturating_duration_since(time::Instant::now())
}) {
None
} else {
Some(delay)
}
}
}
/// Run a git operation, repeating it while the failure looks transient.
///
/// `attempt` receives the 1-based attempt number. `classify` decides whether
/// an error is worth repeating; `None` returns it to the caller untouched.
/// A retry starts only when its backoff fits before the plan's effective
/// deadline. The final error is returned as-is.
pub async fn retry_git_operation<T, E, Attempt, Fut, Classify>(
provider: SandboxProviderKind,
op: &str,
plan: &RetryPlan,
mut attempt: Attempt,
classify: Classify,
) -> Result<T, E>
where
Attempt: FnMut(u32) -> Fut,
Fut: Future<Output = Result<T, E>>,
Classify: Fn(&E) -> Option<GitRetryReason>,
{
let deadline = plan.effective_deadline(time::Instant::now());
for attempt_number in 1..plan.max_attempts.max(1) {
match attempt(attempt_number).await {
Ok(value) => return Ok(value),
Err(err) => {
let Some(reason) = classify(&err) else {
return Err(err);
};
let Some(delay) = plan.retry_delay(attempt_number, deadline) else {
return Err(err);
};
// The failure text can carry git stderr, so log the category
// rather than the message. The caller still reports the full
// error if the attempts run out.
tracing::warn!(
provider = %provider,
op,
attempt = attempt_number,
max_attempts = plan.max_attempts,
reason = %reason,
delay_ms = u64::try_from(delay.as_millis()).unwrap_or(u64::MAX),
"Git operation failed, retrying"
);
time::sleep(delay).await;
}
}
}
attempt(plan.max_attempts.max(1)).await
}
#[cfg(test)]
mod tests {
use std::sync::Mutex;
use super::*;
/// Records the attempt numbers a closure was called with.
#[derive(Default)]
struct Attempts(Mutex<Vec<u32>>);
impl Attempts {
fn record(&self, attempt: u32) {
self.0.lock().expect("attempt log mutex").push(attempt);
}
fn recorded(&self) -> Vec<u32> {
self.0.lock().expect("attempt log mutex").clone()
}
}
/// A classifier that treats every failure as worth repeating.
const ALWAYS_RETRY: fn(&String) -> Option<GitRetryReason> =
|_| Some(GitRetryReason::TokenReplication);
fn fresh_snapshot(age: Duration, ttl: Duration) -> TokenSnapshot {
let now = Utc::now();
TokenSnapshot {
generation: 1,
provenance: TokenProvenance::Minted {
minted_at: now - chrono::Duration::from_std(age).unwrap(),
expires_at: now + chrono::Duration::from_std(ttl).unwrap(),
},
}
}
#[test]
fn credential_context_reads_token_age_from_provenance() {
assert_eq!(
CredentialContext::from_snapshot(None),
CredentialContext::None
);
assert_eq!(
CredentialContext::from_snapshot(Some(&TokenSnapshot {
generation: 0,
provenance: TokenProvenance::Static,
})),
CredentialContext::Static
);
assert_eq!(
CredentialContext::from_snapshot(Some(&fresh_snapshot(
Duration::from_secs(5),
Duration::from_hours(1)
))),
CredentialContext::FreshApp
);
assert_eq!(
CredentialContext::from_snapshot(Some(&fresh_snapshot(
Duration::from_mins(2),
Duration::from_hours(1)
))),
CredentialContext::MatureApp
);
}
#[test]
fn private_repo_not_found_with_a_fresh_token_is_a_replication_lag() {
assert_eq!(
classify_message(
"repository not found: Repository not found.",
CredentialContext::FreshApp
),
GitMessageClass::Retry(GitRetryReason::TokenReplication)
);
}
#[test]
fn not_found_with_a_mature_token_is_transient_not_permanent() {
// A service-side blip is indistinguishable from access loss at this
// layer, so a mature-App 404 stays retryable.
assert_eq!(
classify_message(
"repository not found: Repository not found.",
CredentialContext::MatureApp
),
GitMessageClass::Retry(GitRetryReason::TransientInfra)
);
}
#[test]
fn not_found_with_static_or_no_credentials_is_permanent() {
for cred in [CredentialContext::Static, CredentialContext::None] {
assert_eq!(
classify_message("repository not found: Repository not found.", cred),
GitMessageClass::Permanent,
"{cred:?} cannot become valid by waiting"
);
}
}
#[test]
fn auth_failure_classification_follows_the_credential_context() {
let message = "fatal: Authentication failed for 'https://github.com/owner/repo'";
assert_eq!(
classify_message(message, CredentialContext::FreshApp),
GitMessageClass::Retry(GitRetryReason::TokenReplication)
);
assert_eq!(
classify_message(message, CredentialContext::MatureApp),
GitMessageClass::Retry(GitRetryReason::TransientInfra)
);
assert_eq!(
classify_message(message, CredentialContext::Static),
GitMessageClass::Permanent
);
}
#[test]
fn infra_failures_retry_without_credentials() {
for message in [
"fatal: unable to access: Could not resolve host: github.com",
"error: RPC failed; curl 56 recv failure",
"fatal: early EOF",
"Operation timed out",
] {
assert_eq!(
classify_message(message, CredentialContext::None),
GitMessageClass::Retry(GitRetryReason::TransientInfra),
"expected {message:?} to be transient"
);
}
}
#[test]
fn genuine_failures_are_not_retried() {
for message in [
"fatal: could not read Username for 'https://github.com'",
"remote: Permission to owner/repo.git denied",
"fatal: destination path 'repo' already exists",
] {
assert_eq!(
classify_message(message, CredentialContext::FreshApp),
GitMessageClass::Permanent,
"expected {message:?} to fail fast"
);
}
}
#[test]
fn unrecognized_failures_remain_unknown() {
assert_eq!(
classify_message(
"git operation stopped for an unexpected reason",
CredentialContext::FreshApp
),
GitMessageClass::Unknown
);
}
#[test]
fn backoff_waits_seconds_not_milliseconds() {
let plan = RetryPlan::clone_default(None);
assert_eq!(plan.backoff.delay_for_attempt(1), Duration::from_secs(3));
assert_eq!(plan.backoff.delay_for_attempt(2), Duration::from_secs(9));
}
#[test]
fn publish_backoff_grows_toward_a_one_minute_cap() {
let plan = RetryPlan::publish_push();
assert_eq!(plan.backoff.delay_for_attempt(1), Duration::from_secs(3));
assert_eq!(plan.backoff.delay_for_attempt(2), Duration::from_secs(10));
assert!(plan.backoff.delay_for_attempt(3) < Duration::from_secs(35));
assert_eq!(plan.backoff.delay_for_attempt(4), Duration::from_mins(1));
}
/// `REFRESH_MARGIN` must exceed every push plan's `max_elapsed`: a push
/// pins the token of its single successful resolve, and any token the
/// source returns has at least the margin of validity left, so the pinned
/// token must outlive the whole operation.
#[test]
fn refresh_margin_exceeds_every_push_plan_elapsed_bound() {
for plan in [RetryPlan::checkpoint_push(), RetryPlan::publish_push()] {
let max_elapsed = plan.max_elapsed.expect("push plans bound elapsed time");
assert!(
REFRESH_MARGIN > max_elapsed,
"margin invariant violated: {max_elapsed:?}"
);
}
}
#[test]
fn effective_deadline_takes_the_minimum_of_present_bounds() {
let start = time::Instant::now();
let outer = start + Duration::from_secs(30);
let unbounded = RetryPlan::clone_default(None);
assert_eq!(unbounded.effective_deadline(start), None);
let outer_only = RetryPlan::clone_default(Some(outer));
assert_eq!(outer_only.effective_deadline(start), Some(outer));
let mut both = RetryPlan::checkpoint_push();
both.outer_deadline = Some(outer);
assert_eq!(both.effective_deadline(start), Some(outer));
both.outer_deadline = Some(start + Duration::from_mins(10));
assert_eq!(
both.effective_deadline(start),
Some(start + Duration::from_secs(90))
);
}
#[tokio::test(start_paused = true)]
async fn attempt_timeout_is_capped_by_the_remaining_deadline() {
let plan = RetryPlan::checkpoint_push();
let deadline = Some(time::Instant::now() + Duration::from_secs(20));
assert_eq!(
plan.attempt_timeout(deadline),
Some(Duration::from_secs(20))
);
assert_eq!(plan.attempt_timeout(None), Some(Duration::from_mins(1)));
let unbounded = RetryPlan::clone_default(None);
assert_eq!(unbounded.attempt_timeout(None), None);
}
#[tokio::test(start_paused = true)]
async fn first_success_runs_one_attempt() {
let attempts = Attempts::default();
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"clone",
&RetryPlan::clone_default(None),
|attempt| {
attempts.record(attempt);
async move { Ok::<_, String>(attempt) }
},
ALWAYS_RETRY,
)
.await;
assert_eq!(result, Ok(1));
assert_eq!(attempts.recorded(), vec![1]);
}
#[tokio::test(start_paused = true)]
async fn retries_until_a_later_attempt_succeeds() {
let attempts = Attempts::default();
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"clone",
&RetryPlan::clone_default(None),
|attempt| {
attempts.record(attempt);
async move {
if attempt < 3 {
Err("Repository not found.".to_string())
} else {
Ok(attempt)
}
}
},
ALWAYS_RETRY,
)
.await;
assert_eq!(result, Ok(3));
assert_eq!(attempts.recorded(), vec![1, 2, 3]);
}
#[tokio::test(start_paused = true)]
async fn exhausted_attempts_return_the_final_error() {
let attempts = Attempts::default();
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"clone",
&RetryPlan::clone_default(None),
|attempt| {
attempts.record(attempt);
async move { Err::<(), _>(format!("Repository not found. (attempt {attempt})")) }
},
ALWAYS_RETRY,
)
.await;
assert_eq!(
result,
Err("Repository not found. (attempt 3)".to_string()),
"the caller should see the last failure, not the first"
);
assert_eq!(attempts.recorded(), vec![1, 2, 3]);
}
#[tokio::test(start_paused = true)]
async fn unretryable_failure_stops_immediately() {
let attempts = Attempts::default();
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"clone",
&RetryPlan::clone_default(None),
|attempt| {
attempts.record(attempt);
async move { Err::<(), _>("permission denied".to_string()) }
},
|_: &String| None,
)
.await;
assert_eq!(result, Err("permission denied".to_string()));
assert_eq!(
attempts.recorded(),
vec![1],
"a deterministic failure should not wait out the backoff"
);
}
/// Docker clone parity: the caller's absolute deadline stops retries when
/// the backoff no longer fits before it.
#[tokio::test(start_paused = true)]
async fn outer_deadline_stops_retry_when_backoff_does_not_fit() {
let attempts = Attempts::default();
let deadline = time::Instant::now() + Duration::from_secs(2);
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"clone",
&RetryPlan::clone_default(Some(deadline)),
|attempt| {
attempts.record(attempt);
async move { Err::<(), _>("temporary failure".to_string()) }
},
ALWAYS_RETRY,
)
.await;
assert_eq!(result, Err("temporary failure".to_string()));
assert_eq!(attempts.recorded(), vec![1]);
assert_eq!(time::Instant::now() + Duration::from_secs(2), deadline);
}
/// Daytona clone parity: with no bounds at all, attempts are limited only
/// by `max_attempts` and backoff.
#[tokio::test(start_paused = true)]
async fn unbounded_plan_runs_all_attempts() {
let attempts = Attempts::default();
let result = retry_git_operation(
SandboxProviderKind::DAYTONA,
"clone",
&RetryPlan::clone_default(None),
|attempt| {
attempts.record(attempt);
async move { Err::<(), _>("temporary failure".to_string()) }
},
|_: &String| Some(GitRetryReason::TransientInfra),
)
.await;
assert!(result.is_err());
assert_eq!(attempts.recorded(), vec![1, 2, 3]);
}
#[tokio::test(start_paused = true)]
async fn max_elapsed_stops_retry_when_backoff_does_not_fit() {
let attempts = Attempts::default();
let plan = RetryPlan {
max_attempts: 5,
backoff: replication_backoff(),
max_elapsed: Some(Duration::from_secs(4)),
per_attempt_timeout: None,
outer_deadline: None,
};
let result = retry_git_operation(
SandboxProviderKind::DOCKER,
"push",
&plan,
|attempt| {
attempts.record(attempt);
async move { Err::<(), _>("temporary failure".to_string()) }
},
ALWAYS_RETRY,
)
.await;
assert!(result.is_err());
// Attempt 1 fails instantly, 3s backoff fits inside 4s, attempt 2
// fails, and the 9s backoff no longer fits.
assert_eq!(attempts.recorded(), vec![1, 2]);
}
}

66
lib/components/fabro-sandbox/src/lib.rs
View file

@ -1,30 +1,27 @@
pub mod environment;
pub mod error;
pub mod options;
pub mod provider;
pub mod sandbox;
pub mod sandbox_spec;
mod clone_source;
mod git_retry;
mod git_policy;
mod managed_labels;
mod push_credentials;
pub mod redact;
mod credentials;
pub mod details;
pub mod driver;
pub mod driver_sandbox;
pub mod environment;
pub mod exec;
mod pebble_environment;
pub mod reconnect;
pub mod terminal;
mod redact;
mod clone;
pub mod docker;
@ -38,40 +35,37 @@ pub mod test_support;
pub use details::sandbox_details;
pub use docker::check_docker_daemon;
pub use driver::{DaytonaCredentials, ProviderAccess};
pub use driver_sandbox::{RunSandbox, local_sandbox};
pub use driver_sandbox::RunSandbox;
pub use environment::{CloneRequest, sandbox_spec_for_environment};
pub use error::{Error, Result, default_redacted_output_tail, display_for_log};
pub use exec::{ExplicitEnvPolicy, SandboxExec, is_sensitive_env_var};
pub use exec::{
DEFAULT_RETAINED_OUTPUT_BYTES, DEFAULT_STOP_GRACE, ExecResultExt, SandboxExec,
command_termination, program_exit_code,
};
pub use fabro_github::token_source::{
InstallationTokenSource, ResolvedToken, TokenProvenance, TokenSnapshot,
};
pub use fabro_types::{RunSandboxInstance, SandboxProviderKind};
pub use git_retry::{
CredentialContext, GitRetryReason, RetryPlan, classify_failure, retry_git_operation,
};
pub use options::{
SandboxOptions, local_working_directory_from_environment, options_from_environment,
unresolved_env,
};
pub use provider::driver::DriverInventoryProvider;
pub use provider::{
LocalSandboxProvider, SandboxLookupError, SandboxProvider, SandboxProviderRegistry,
};
pub use provider_sandbox::{attach_provider_sandbox, provider_sandbox};
pub use push_credentials::RefreshErrorKind;
pub use reconnect::{
reconnect, reconnect_driver_for_run, reconnect_for_run, reconnect_for_run_with_events,
pub use git_policy::{
GitRetryReason, checkpoint_push_policy, publish_push_policy, repository_probe_policy,
retry_git_messages, transient_git_failure,
};
pub use provider::{SandboxInventory, SandboxLookupError};
pub use provider_sandbox::{attach_provider_sandbox, local_sandbox, provider_sandbox};
pub use reconnect::{open_terminal_for_run, reconnect_for_run};
pub use redact::SecretRedactor;
pub use sandbox::{
CommandOutputCallback, DEFAULT_EXEC_OUTPUT_TAIL_BYTES, ExecResult, ExecStreamingRequest,
ExecStreamingResult, GitRunInfo, GitSetupIntent, OutputCaptureStats, PushAttempt, PushError,
PushReport, RefreshOutcome, RemoteCredentialAction, SandboxFile, SandboxWorkspaceLayout,
StderrCollector, StdioProcess, StdioProcessHandle, StdioProcessTermination,
format_lines_numbered, redacted_output_tail, setup_git, shell_quote,
DEFAULT_EXEC_OUTPUT_TAIL_BYTES, GitRunInfo, GitSetupIntent, PushAttempt, PushError, PushReport,
SandboxFile, SandboxWorkspaceLayout, redacted_output_tail, setup_git,
};
/// Driver types a run sandbox's file and search operations speak, and the
/// network policy a [`SandboxOptions`] asks for, re-exported so consumers
/// need no direct driver dependency.
pub use sandbox_driver::{DirEntry, FileKind, GrepMatch, GrepOptions, NetworkPolicy, WalkOptions};
pub use sandbox_spec::{ProviderSandboxSpec, SandboxSpec};
pub use terminal::{DriverTerminalSession, TerminalSession, TerminalSize, open_terminal_for_run};
/// Driver types a run sandbox speaks: what a command is and how it ended,
/// what the file and search operations return, and what an environment
/// asks of a sandbox. Re-exported so consumers need no direct driver
/// dependency.
pub use sandbox_driver::{
CaptureStats, DirEntry, ExecControls, ExecFailure, ExecResult, ExecSpec, ExecStreamingResult,
FileKind, GitRetryPolicy, GrepMatch, GrepOptions, LifecycleTimers, NetworkPolicy, OutputSink,
OutputStream, PtySession, PtySize, Resources, SandboxSource, SandboxSpec as DriverSpec,
StderrTail, StdioProcess, StdioProcessHandle, Termination, TransportError, WalkOptions,
};
pub use sandbox_spec::SandboxSpec;

391
lib/components/fabro-sandbox/src/options.rs
View file

@ -1,391 +0,0 @@
//! What an environment asks of a sandbox, mapped once for every provider.
//!
//! The environment names an image or Dockerfile, resources, a network
//! policy, labels, variables, a lifecycle, and a clone policy. Every
//! provider consumes the same [`SandboxOptions`]: the driver spec is built
//! from them in one place, and a bundled provider adds only what its
//! backend needs on top (the Docker working directory, the Daytona
//! snapshot and timers) in its own overlay.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::Duration;
use fabro_types::RunId;
use fabro_types::settings::run::{
DockerfileSource, EnvironmentNetworkMode, RunCloneSettings, RunEnvironmentSettings,
};
use sandbox_driver::{
Capabilities, NetworkPolicy, Resources, SandboxSource, SandboxSpec as DriverSpec,
};
/// What an environment asks of a sandbox, provider-neutral.
#[derive(Clone, Debug, Default)]
pub struct SandboxOptions {
/// Image reference, when the environment names one.
pub image: Option<String>,
/// Inline Dockerfile, when the environment names one instead of an
/// image.
pub dockerfile: Option<String>,
/// Environment variables for the sandbox, resolved.
pub env: BTreeMap<String, String>,
pub network: NetworkPolicy,
pub cpu: Option<u32>,
pub memory_bytes: Option<u64>,
pub disk_bytes: Option<u64>,
/// Labels from the environment; fabro's managed labels are added.
pub labels: BTreeMap<String, String>,
/// Idle time before the provider stops the sandbox, when the
/// environment sets one.
pub auto_stop: Option<Duration>,
/// Maximum Git history depth fetched during clone; `None` fetches full
/// history.
pub clone_depth: Option<u32>,
/// Create an empty workspace instead of cloning even when an origin
/// exists.
pub skip_clone: bool,
}
impl SandboxOptions {
/// Memory in whole mebibytes, rounded up.
pub fn memory_mb(&self) -> Option<u64> {
self.memory_bytes.map(|bytes| bytes.div_ceil(1024 * 1024))
}
/// Disk in whole mebibytes, rounded up.
pub fn disk_mb(&self) -> Option<u64> {
self.disk_bytes.map(|bytes| bytes.div_ceil(1024 * 1024))
}
}
/// Maps resolved environment settings onto sandbox options. `env` is the
/// environment's variables, resolved by the caller: the worker resolves
/// secrets through the vault, while preflight carries them in source form.
///
/// A Dockerfile given as a path must have been resolved to inline content
/// earlier; none of the providers can read a path.
pub fn options_from_environment(
settings: &RunEnvironmentSettings,
clone: &RunCloneSettings,
env: BTreeMap<String, String>,
) -> crate::Result<SandboxOptions> {
// fabro-config rejects environments that set both image.docker and
// image.dockerfile. If both still arrive here, the image wins.
let dockerfile = match (&settings.image.docker, &settings.image.dockerfile) {
(Some(_), _) | (None, None) => None,
(None, Some(DockerfileSource::Inline(content))) => Some(content.clone()),
(None, Some(DockerfileSource::Path { path })) => {
return Err(crate::Error::message(format!(
"environment `{}` names a Dockerfile path ({path}) that should have been \
resolved to inline content before sandbox creation",
settings.id
)));
}
};
Ok(SandboxOptions {
image: settings.image.docker.clone(),
dockerfile,
env,
network: match settings.network.mode {
EnvironmentNetworkMode::Block => NetworkPolicy::Block,
EnvironmentNetworkMode::AllowAll => NetworkPolicy::AllowAll,
EnvironmentNetworkMode::CidrAllowList => NetworkPolicy::CidrAllowList {
cidrs: settings.network.allow.clone(),
},
},
cpu: settings
.resources
.cpu
.and_then(|cpu| u32::try_from(cpu).ok()),
memory_bytes: settings.resources.memory.map(|size| size.as_bytes()),
disk_bytes: settings.resources.disk.map(|size| size.as_bytes()),
labels: settings
.labels
.iter()
.map(|(key, value)| (key.clone(), value.clone()))
.collect(),
auto_stop: settings
.lifecycle
.auto_stop
.map(|duration| duration.as_std()),
clone_depth: clone
.depth_limit()
.and_then(|depth| u32::try_from(depth).ok()),
skip_clone: !clone.enabled,
})
}
/// The environment's variables in source form, for a path with no vault
/// (server preflight): a `{{ secrets.* }}` value keeps its token, and
/// nothing else is left to resolve because `{{ vars.* }}` is substituted at
/// run creation.
pub fn unresolved_env(settings: &RunEnvironmentSettings) -> BTreeMap<String, String> {
#[expect(
clippy::disallowed_methods,
reason = "preflight has no vault, so an unresolved secret token is carried in source form"
)]
settings
.env
.iter()
.map(|(key, value)| (key.clone(), value.as_source()))
.collect()
}
pub fn local_working_directory_from_environment(
settings: &RunEnvironmentSettings,
source_directory: Option<&Path>,
) -> crate::Result<PathBuf> {
if let Some(cwd) = settings.cwd.as_deref() {
return Ok(PathBuf::from(cwd));
}
let Some(source_directory) = source_directory else {
return Err(crate::Error::message(
"local environment requires a server-side working directory; configure `environment.cwd = \"/absolute/path\"` on the selected local environment",
));
};
if source_directory.is_dir() {
return Ok(source_directory.to_path_buf());
}
Err(crate::Error::message(format!(
"local environment source_directory does not exist or is not a directory on this server: {}. Configure `environment.cwd = \"/absolute/path\"` on the selected local environment for remote client/server deployments.",
source_directory.display()
)))
}
/// The driver spec every provider starts from: the environment's source
/// (an image, a Dockerfile, or a managed directory when it names
/// neither), the run's name, the environment's labels, variables,
/// resources, and network policy. A bundled provider's overlay adjusts
/// what its backend needs, and the ownership scope adds fabro's labels.
pub(crate) fn base_spec(options: &SandboxOptions, run_id: Option<&RunId>) -> DriverSpec {
let source = match (&options.image, &options.dockerfile) {
(Some(reference), _) => SandboxSource::Image {
reference: reference.clone(),
},
(None, Some(content)) => SandboxSource::Dockerfile {
content: content.clone(),
},
// A provider without images (a host-style plugin) manages a
// workspace directory of its own.
(None, None) => SandboxSource::HostDirectory,
};
let mut spec = DriverSpec::new(source).network(options.network.clone());
if let Some(run_id) = run_id {
spec = spec.name(run_name(run_id));
}
// The environment's labels; fabro's ownership labels are stamped by the
// ownership scope the provider is connected through.
for (key, value) in &options.labels {
spec = spec.label(key, value);
}
for (key, value) in &options.env {
spec = spec.env_var(key, value);
}
let mut resources = Resources::default();
resources.cpu_cores = options.cpu;
resources.memory_mb = options.memory_mb();
resources.disk_mb = options.disk_mb();
spec.resources(resources)
}
/// The provider-side name of a run's sandbox.
pub(crate) fn run_name(run_id: &RunId) -> String {
format!("fabro-run-{run_id}")
}
/// The environment's default `allow_all` means "unrestricted", which a
/// provider without network controls already is; asking such a provider
/// for it explicitly would be rejected. An explicit restriction is still
/// requested, and refused by the provider when it cannot honor it.
pub(crate) fn supported_network(
requested: NetworkPolicy,
capabilities: &Capabilities,
) -> NetworkPolicy {
match requested {
NetworkPolicy::AllowAll if !capabilities.network.allow_all => {
NetworkPolicy::ProviderDefault
}
other => other,
}
}
#[cfg(test)]
mod tests {
use std::collections::HashMap;
use fabro_types::SandboxProviderKind;
use fabro_types::settings::run::{
EnvironmentImageSettings, EnvironmentLifecycleSettings, EnvironmentNetworkSettings,
EnvironmentResourcesSettings,
};
use fabro_types::settings::{Duration as SettingsDuration, Size};
use super::*;
fn environment(kind: &str) -> RunEnvironmentSettings {
RunEnvironmentSettings {
id: kind.to_string(),
provider: SandboxProviderKind::try_new(kind).unwrap(),
cwd: None,
image: EnvironmentImageSettings::default(),
resources: EnvironmentResourcesSettings::default(),
network: EnvironmentNetworkSettings::default(),
lifecycle: EnvironmentLifecycleSettings::default(),
labels: HashMap::from([("team".to_string(), "platform".to_string())]),
env: HashMap::new(),
}
}
fn run_id() -> RunId {
"01HY0000000000000000000000".parse().unwrap()
}
#[test]
fn options_without_an_image_ask_for_a_managed_directory() {
let options = options_from_environment(
&environment("host"),
&RunCloneSettings::default(),
BTreeMap::from([("FOO".to_string(), "bar".to_string())]),
)
.unwrap();
assert!(options.image.is_none());
assert!(options.dockerfile.is_none());
assert_eq!(options.clone_depth, Some(100));
assert!(!options.skip_clone);
assert!(options.auto_stop.is_none());
let spec = base_spec(&options, Some(&run_id()));
assert!(matches!(spec.source, SandboxSource::HostDirectory));
assert!(spec.working_directory.is_none());
assert_eq!(
spec.name.as_deref(),
Some("fabro-run-01HY0000000000000000000000")
);
assert_eq!(spec.env.get("FOO").map(String::as_str), Some("bar"));
assert_eq!(
spec.labels.get("team").map(String::as_str),
Some("platform")
);
assert!(
!spec.labels.contains_key("sh.fabro.managed"),
"ownership labels come from the scope, not the environment"
);
assert!(matches!(spec.network, NetworkPolicy::AllowAll));
}
#[test]
fn options_with_an_image_map_resources_network_and_lifecycle() {
let mut settings = environment("e2b");
settings.image.docker = Some("ubuntu:24.04".to_string());
settings.resources.cpu = Some(2);
settings.resources.memory = Some(Size::from_bytes(4_000_000_000));
settings.network.mode = EnvironmentNetworkMode::Block;
settings.lifecycle.auto_stop = Some(SettingsDuration::from_std(Duration::from_mins(45)));
let clone = RunCloneSettings {
enabled: false,
depth: 0,
};
let options = options_from_environment(&settings, &clone, BTreeMap::new()).unwrap();
assert_eq!(options.image.as_deref(), Some("ubuntu:24.04"));
assert!(options.skip_clone);
assert_eq!(options.clone_depth, None);
assert_eq!(options.memory_bytes, Some(4_000_000_000));
assert_eq!(options.memory_mb(), Some(3815));
assert_eq!(options.auto_stop, Some(Duration::from_mins(45)));
let spec = base_spec(&options, None);
assert!(matches!(
&spec.source,
SandboxSource::Image { reference } if reference == "ubuntu:24.04"
));
assert_eq!(spec.resources.cpu_cores, Some(2));
assert_eq!(spec.resources.memory_mb, Some(3815));
assert!(matches!(spec.network, NetworkPolicy::Block));
assert!(spec.name.is_none());
}
#[test]
fn an_inline_dockerfile_becomes_the_source_and_a_path_is_rejected() {
let mut settings = environment("daytona");
settings.image.dockerfile = Some(DockerfileSource::Inline("FROM ubuntu".to_string()));
let options =
options_from_environment(&settings, &RunCloneSettings::default(), BTreeMap::new())
.unwrap();
assert_eq!(options.dockerfile.as_deref(), Some("FROM ubuntu"));
assert!(matches!(
base_spec(&options, None).source,
SandboxSource::Dockerfile { content } if content == "FROM ubuntu"
));
settings.image.dockerfile = Some(DockerfileSource::Path {
path: "Dockerfile".to_string(),
});
let error =
options_from_environment(&settings, &RunCloneSettings::default(), BTreeMap::new())
.unwrap_err();
assert!(error.to_string().contains("Dockerfile path"), "{error}");
}
#[test]
fn allow_all_falls_back_to_the_provider_default_without_network_control() {
let none = Capabilities::minimal(sandbox_driver::Isolation::None);
assert!(matches!(
supported_network(NetworkPolicy::AllowAll, &none),
NetworkPolicy::ProviderDefault
));
assert!(matches!(
supported_network(NetworkPolicy::Block, &none),
NetworkPolicy::Block
));
let mut full = Capabilities::minimal(sandbox_driver::Isolation::Container);
full.network.allow_all = true;
assert!(matches!(
supported_network(NetworkPolicy::AllowAll, &full),
NetworkPolicy::AllowAll
));
}
#[test]
fn local_working_directory_prefers_environment_cwd() {
let mut settings = environment("local");
settings.cwd = Some("/srv/fabro/workspaces/team-a".to_string());
let missing_source = Path::new("/path/that/should/not/exist");
let resolved = local_working_directory_from_environment(&settings, Some(missing_source))
.expect("configured cwd should be accepted");
assert_eq!(resolved, PathBuf::from("/srv/fabro/workspaces/team-a"));
assert!(!missing_source.exists());
}
#[test]
fn local_working_directory_uses_existing_source_directory_without_cwd() {
let settings = environment("local");
let dir = tempfile::tempdir().unwrap();
let resolved = local_working_directory_from_environment(&settings, Some(dir.path()))
.expect("existing source directory should be accepted");
assert_eq!(resolved, dir.path());
}
#[test]
fn local_working_directory_rejects_missing_source_directory_without_cwd() {
let settings = environment("local");
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("client-only");
let err = local_working_directory_from_environment(&settings, Some(&missing))
.expect_err("missing source directory without cwd should fail");
let message = err.to_string();
assert!(
message.contains("environment.cwd") && message.contains("does not exist"),
"unexpected error: {message}"
);
assert!(!missing.exists());
}
}

373
lib/components/fabro-sandbox/src/pebble_environment.rs Normal file
View file

@ -0,0 +1,373 @@
//! [`RunSandbox`] as the [`Environment`] pebble's coding agent runs in.
//!
//! Pebble's tools speak the `Environment` contract; fabro's one sandbox type
//! speaks the sandbox driver's facets. This module is the mapping between the
//! two, and nothing else: every path resolves the way fabro resolves it, every
//! command runs through [`SandboxExec`](crate::SandboxExec) with fabro's
//! exec policy, and every failure keeps its driver cause. There is no adapter
//! struct; a run sandbox *is* an environment.
//!
//! Where the two contracts differ, pebble's wins here because the model reads
//! pebble's: a glob that pebble rejects is rejected before the driver sees it,
//! a directory listing is in tree order, and a command with no retention cap
//! still drains under the driver's default buffer rather than without bound.
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use pebble_coding_agent::environment::support::{capture_stats, tree_order, validate_glob};
use pebble_coding_agent::environment::{
DirEntry, EnvResult, Environment, EnvironmentError, EnvironmentErrorKind, ExecOutcome,
ExecOutputSink, ExecOutputStream, ExecRequest, ExecResult, GrepOptions,
};
use sandbox_driver::{ExecControls, ExecSpec, FileKind, OutputSink, OutputStream};
use crate::driver_sandbox::RunSandbox;
use crate::exec::{ExecResultExt as _, command_termination, program_exit_code};
use crate::sandbox;
#[async_trait]
impl Environment for RunSandbox {
fn working_directory(&self) -> &str {
Self::working_directory(self)
}
fn platform(&self) -> &str {
Self::platform(self)
}
fn os_version(&self) -> String {
Self::os_version(self)
}
async fn read_file_bytes(&self, path: &str) -> EnvResult<Vec<u8>> {
Self::read_file_bytes(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to read {path}"), error))
}
async fn write_file(&self, path: &str, content: &str) -> EnvResult<()> {
Self::write_file(self, path, content)
.await
.map_err(|error| environment_error(&format!("Failed to write {path}"), error))
}
async fn rename_file(&self, source: &str, destination: &str) -> EnvResult<()> {
let resolved_source = self.resolve_for_environment(source);
let resolved_destination = self.resolve_for_environment(destination);
if !Self::file_exists(self, source)
.await
.map_err(|error| environment_error(&format!("Failed to stat {source}"), error))?
{
return Err(EnvironmentError::new(
EnvironmentErrorKind::NotFound,
format!("Failed to move {source}: file does not exist"),
));
}
// The same path spelled twice is a move to itself, which must leave
// the file where it is. Aliases the sandbox's own filesystem would
// resolve (a symlinked parent, a hard link) are not checked: fabro has
// no remote `realpath`, and a driver `mv a a` is a no-op anyway.
if normalize(&resolved_source) == normalize(&resolved_destination) {
return Ok(());
}
let handle = self
.handle()
.map_err(|error| environment_error("Sandbox is not initialized", error))?;
// The destination's parent is created first, and a parent that is a
// file fails here, before anything has moved, so the source stays
// intact as the contract requires.
if let Some(parent) = parent_directory(&resolved_destination) {
handle.fs().create_dir(parent).await.map_err(|error| {
environment_error(
&format!("Failed to create the parent directory of {destination}"),
crate::Error::from(error),
)
})?;
}
handle
.fs()
.rename(&resolved_source, &resolved_destination)
.await
.map_err(|error| {
environment_error(
&format!("Failed to move {source} to {destination}"),
crate::Error::from(error),
)
})
}
async fn delete_file(&self, path: &str) -> EnvResult<()> {
// The driver's delete is idempotent; pebble's is a `remove_file`, which
// reports a path that is not there.
if !Self::file_exists(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to stat {path}"), error))?
{
return Err(EnvironmentError::new(
EnvironmentErrorKind::NotFound,
format!("Failed to delete {path}: file does not exist"),
));
}
Self::delete_file(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to delete {path}"), error))
}
async fn file_exists(&self, path: &str) -> EnvResult<bool> {
Self::file_exists(self, path)
.await
.map_err(|error| environment_error(&format!("Failed to stat {path}"), error))
}
async fn list_directory(&self, path: &str, depth: Option<usize>) -> EnvResult<Vec<DirEntry>> {
let mut entries: Vec<DirEntry> = Self::list_directory(self, path, depth)
.await
.map_err(|error| environment_error(&format!("Failed to list {path}"), error))?
.into_iter()
.map(|entry| DirEntry {
is_dir: entry.kind == FileKind::Directory,
size: (entry.kind == FileKind::File)
.then_some(entry.size)
.flatten(),
name: entry.path,
})
.collect();
// The driver lists in flat lexicographic order of the whole relative
// path, where `foo-bar` sorts between `foo` and `foo/x`. Pebble lists
// in tree order, and says how.
tree_order(&mut entries);
Ok(entries)
}
async fn grep(
&self,
pattern: &str,
path: &str,
options: &GrepOptions,
) -> EnvResult<Vec<String>> {
let mut driver_options = sandbox_driver::GrepOptions::default();
driver_options.case_insensitive = options.case_insensitive;
driver_options.max_matches = options.max_results;
driver_options.include = options.glob_filter.clone();
let matches = Self::grep(self, pattern, path, &driver_options)
.await
.map_err(|error| environment_error("Failed to search file contents", error))?;
Ok(matches
.into_iter()
.map(|found| format!("{}:{}:{}", found.path, found.line_number, found.line))
.collect())
}
async fn glob(&self, pattern: &str, path: Option<&str>) -> EnvResult<Vec<String>> {
// Validated by pebble's own grammar before the driver sees the
// pattern, so the reason reaches the model in pebble's words and the
// patterns pebble rejects are rejected even where fabro's glob would
// accept them.
validate_glob(pattern)?;
Self::glob(self, pattern, path)
.await
.map_err(|error| environment_error("Failed to match files", error))
}
async fn exec(&self, request: ExecRequest<'_>) -> EnvResult<ExecOutcome> {
let ExecRequest {
command,
timeout_ms,
working_dir,
env_vars,
cancel_token,
output_bytes_cap,
output_sink,
} = request;
let mut spec = ExecSpec::bash(command).no_timeout();
if let Some(timeout_ms) = timeout_ms {
spec = spec.timeout(Duration::from_millis(timeout_ms));
}
if let Some(dir) = working_dir {
spec = spec.working_dir(dir);
}
for (key, value) in env_vars.into_iter().flatten() {
spec = spec.env_var(key, value);
}
let controls = ExecControls {
term: cancel_token,
sink: output_sink.map(adapt_output_sink),
// `None` asks pebble for no cap at all. Fabro's exec policy fills
// its default buffer when the cap is unset, so a command with no
// cap drains under that default rather than without bound; the
// capture counts still say what was dropped.
retained_output_limit: output_bytes_cap,
..ExecControls::default()
};
let streaming = self
.exec_command_streaming(spec, controls)
.await
.map_err(|error| {
let kind = match error.driver() {
Some(sandbox_driver::Error::Transport(_)) => EnvironmentErrorKind::Io,
Some(sandbox_driver::Error::Unsupported { .. }) => {
EnvironmentErrorKind::Unsupported
}
_ => EnvironmentErrorKind::Spawn,
};
EnvironmentError::with_source(kind, "Failed to run the command", error)
})?;
let result = streaming.result;
Ok(ExecOutcome {
result: ExecResult {
stdout: result.stdout_lossy(),
stderr: result.stderr_lossy(),
exit_code: program_exit_code(result.termination, result.exit_code),
termination: command_termination(result.termination),
duration_ms: result.duration_ms(),
},
streams_separated: streaming.streams_separated,
stdout_capture: capture_stats(
streaming.stdout_capture.observed_bytes,
output_bytes_cap,
),
stderr_capture: capture_stats(
streaming.stderr_capture.observed_bytes,
output_bytes_cap,
),
})
}
}
impl RunSandbox {
/// A caller path as the driver will see it: fabro's working directory
/// applied where fabro applies it, and nothing more.
fn resolve_for_environment(&self, path: &str) -> String {
sandbox::resolve_path(path, Self::working_directory(self))
}
}
/// Pebble's glob grammar, beyond what fabro's glob already rejects.
///
/// A path with its redundant separators and `.` segments removed, for
/// deciding whether two spellings name the same file.
fn normalize(path: &str) -> String {
let absolute = path.starts_with('/');
let joined = path
.split('/')
.filter(|segment| !segment.is_empty() && *segment != ".")
.collect::<Vec<_>>()
.join("/");
if absolute {
format!("/{joined}")
} else {
joined
}
}
/// The directory a path is in, when the path names one.
fn parent_directory(path: &str) -> Option<&str> {
let trimmed = path.trim_end_matches('/');
let (parent, _) = trimmed.rsplit_once('/')?;
if parent.is_empty() {
return Some("/");
}
Some(parent)
}
/// Feeds the driver's asynchronous chunk callback into pebble's synchronous
/// sink.
fn adapt_output_sink(sink: ExecOutputSink) -> OutputSink {
Arc::new(move |stream, chunk: Vec<u8>| {
let stream = match stream {
OutputStream::Stdout => ExecOutputStream::Stdout,
OutputStream::Stderr => ExecOutputStream::Stderr,
};
sink(stream, &chunk);
Box::pin(async { Ok(()) })
})
}
/// A sandbox failure as pebble classifies it, keeping the driver cause.
fn environment_error(message: &str, error: crate::Error) -> EnvironmentError {
let kind = match error.driver() {
Some(sandbox_driver::Error::NotFound { .. }) => EnvironmentErrorKind::NotFound,
Some(sandbox_driver::Error::Unsupported { .. }) => EnvironmentErrorKind::Unsupported,
_ => EnvironmentErrorKind::Io,
};
EnvironmentError::with_source(kind, message, error)
}
#[cfg(test)]
mod tests {
use pebble_coding_agent::test_support::EnvironmentContract;
use super::*;
use crate::local_sandbox;
/// The run sandbox over the driver's Host provider, in a directory that
/// goes away with the test.
async fn host_environment() -> (tempfile::TempDir, RunSandbox) {
let directory = tempfile::tempdir().expect("a temporary directory");
let sandbox = local_sandbox(directory.path().to_path_buf())
.await
.expect("a local sandbox");
(directory, sandbox)
}
#[tokio::test]
async fn host_files_satisfy_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_files()
.await
.expect("file contract");
}
#[tokio::test]
async fn host_search_satisfies_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_search()
.await
.expect("search contract");
}
#[tokio::test]
async fn host_commands_satisfy_pebbles_environment_contract() {
let (_directory, sandbox) = host_environment().await;
EnvironmentContract::new(&sandbox, "contract")
.verify_commands()
.await
.expect("command contract");
}
#[tokio::test]
async fn a_directory_listing_is_in_tree_order() {
let (directory, sandbox) = host_environment().await;
for name in ["foo/x.txt", "foo-bar/y.txt", "foo.txt"] {
Environment::write_file(&sandbox, name, "content")
.await
.expect("fixture");
}
let names: Vec<String> = Environment::list_directory(&sandbox, ".", Some(2))
.await
.expect("listing")
.into_iter()
.map(|entry| entry.name)
.collect();
assert_eq!(names, [
"foo",
"foo/x.txt",
"foo-bar",
"foo-bar/y.txt",
"foo.txt"
]);
drop(directory);
}
#[test]
fn a_path_spelled_two_ways_is_one_path() {
assert_eq!(normalize("/work//a/./b.txt"), "/work/a/b.txt");
assert_eq!(parent_directory("/work/a/b.txt"), Some("/work/a"));
assert_eq!(parent_directory("/b.txt"), Some("/"));
assert_eq!(parent_directory("b.txt"), None);
}
}

490
lib/components/fabro-sandbox/src/provider.rs
View file

@ -1,47 +1,116 @@
pub mod driver;
//! Fabro's inventory of the sandboxes it manages, across the providers a
//! server has configured.
//!
//! Every entry is a sandbox-driver provider narrowed by fabro's ownership
//! labels, so a listing shows only the sandboxes fabro created and an
//! attach to anything else is refused. A provider connects on first use:
//! the inventory is assembled synchronously at startup, and a provider that
//! is down surfaces as a lookup error rather than a startup failure. The
//! `local` kind has an entry too, so a caller can ask whether the kind is
//! ready, but its sandboxes are directories the run record names and there
//! is nothing to list.
use std::sync::Arc;
use async_trait::async_trait;
use fabro_types::settings::server::ServerSandboxProviderSettings;
use fabro_types::{
SandboxInfo, SandboxListMeta, SandboxListResponse, SandboxProviderKind,
SandboxProviderLookupError,
};
use fabro_util::error::collect_chain;
use futures::future::join_all;
use sandbox_driver::{
Error as DriverError, OwnedProvider, SandboxFilter, SandboxId,
SandboxProvider as DriverProvider, SandboxState,
};
use tokio::sync::OnceCell;
#[async_trait]
pub trait SandboxProvider: Send + Sync {
fn kind(&self) -> SandboxProviderKind;
async fn list(&self) -> crate::Result<Vec<SandboxInfo>>;
async fn get(&self, id: &str) -> crate::Result<Option<SandboxInfo>>;
async fn delete(&self, id: &str) -> crate::Result<()>;
}
use crate::driver::{ConnectedProvider, ProviderConnectOptions, connect_provider};
use crate::managed_labels;
/// The sandboxes fabro manages, by provider.
#[derive(Clone, Default)]
pub struct SandboxProviderRegistry {
providers: Vec<Arc<dyn SandboxProvider>>,
pub struct SandboxInventory {
entries: Vec<Arc<InventoryEntry>>,
}
impl SandboxProviderRegistry {
pub fn new(providers: Vec<Arc<dyn SandboxProvider>>) -> Self {
Self { providers }
}
struct InventoryEntry {
kind: SandboxProviderKind,
connection: Connection,
}
enum Connection {
/// Sandboxes on this host are directories the run record names;
/// there is nothing to list.
HostDirectories,
Connected(Arc<dyn DriverProvider>),
/// Connected through [`connect_provider`] on first use.
Lazy(Box<LazyConnection>),
}
struct LazyConnection {
settings: ServerSandboxProviderSettings,
options: ProviderConnectOptions,
provider: OnceCell<Arc<dyn DriverProvider>>,
}
impl SandboxInventory {
#[must_use]
pub fn empty() -> Self {
Self::default()
}
pub fn providers(&self) -> &[Arc<dyn SandboxProvider>] {
&self.providers
/// A kind whose sandboxes are directories on this host: ready to run,
/// nothing to list.
#[must_use]
pub fn with_host_directories(self, kind: SandboxProviderKind) -> Self {
self.with_entry(kind, Connection::HostDirectories)
}
/// A provider already connected, tagged with the kind fabro persists
/// for it.
#[must_use]
pub fn with_connected(self, connected: ConnectedProvider) -> Self {
self.with_entry(
connected.kind,
Connection::Connected(owned(connected.provider)),
)
}
/// A provider connected through [`connect_provider`] on first use.
#[must_use]
pub fn with_lazy(
self,
kind: SandboxProviderKind,
settings: ServerSandboxProviderSettings,
options: ProviderConnectOptions,
) -> Self {
self.with_entry(
kind,
Connection::Lazy(Box::new(LazyConnection {
settings,
options,
provider: OnceCell::new(),
})),
)
}
fn with_entry(mut self, kind: SandboxProviderKind, connection: Connection) -> Self {
self.entries
.push(Arc::new(InventoryEntry { kind, connection }));
self
}
/// The provider kinds this inventory covers.
pub fn kinds(&self) -> impl Iterator<Item = &SandboxProviderKind> {
self.entries.iter().map(|entry| &entry.kind)
}
pub async fn list_managed(&self) -> SandboxListResponse {
let results = join_all(
self.providers
self.entries
.iter()
.map(|provider| async move { (provider.kind(), provider.list().await) }),
.map(|entry| async move { (&entry.kind, entry.list().await) }),
)
.await;
@ -50,7 +119,7 @@ impl SandboxProviderRegistry {
for (kind, result) in results {
match result {
Ok(mut sandboxes) => data.append(&mut sandboxes),
Err(err) => provider_errors.push(provider_error(kind, &err)),
Err(err) => provider_errors.push(provider_error(kind.clone(), &err)),
}
}
@ -65,9 +134,9 @@ impl SandboxProviderRegistry {
id: &str,
) -> Result<SandboxInfo, SandboxLookupError> {
let results = join_all(
self.providers
self.entries
.iter()
.map(|provider| async move { (provider.kind(), provider.get(id).await) }),
.map(|entry| async move { (&entry.kind, entry.get(id).await) }),
)
.await;
@ -77,7 +146,7 @@ impl SandboxProviderRegistry {
match result {
Ok(Some(sandbox)) => matches.push(sandbox),
Ok(None) => {}
Err(err) => provider_errors.push(provider_error(kind, &err)),
Err(err) => provider_errors.push(provider_error(kind.clone(), &err)),
}
}
@ -101,6 +170,94 @@ impl SandboxProviderRegistry {
}
}
impl InventoryEntry {
/// The provider narrowed to fabro's sandboxes, connected on first use;
/// `None` when the kind has nothing to list.
async fn provider(&self) -> crate::Result<Option<&Arc<dyn DriverProvider>>> {
match &self.connection {
Connection::HostDirectories => Ok(None),
Connection::Connected(provider) => Ok(Some(provider)),
Connection::Lazy(lazy) => lazy
.provider
.get_or_try_init(|| async {
connect_provider(&self.kind, &lazy.settings, &lazy.options)
.await
.map(|connected| owned(connected.provider))
.map_err(|error| {
crate::Error::context(
format!("Failed to connect to the {} provider", self.kind),
error,
)
})
})
.await
.map(Some),
}
}
async fn list(&self) -> crate::Result<Vec<SandboxInfo>> {
let Some(provider) = self.provider().await? else {
return Ok(Vec::new());
};
let statuses = provider
.list(&SandboxFilter::default())
.await
.map_err(|error| {
crate::Error::context(format!("Failed to list {} sandboxes", self.kind), error)
})?;
Ok(statuses
.into_iter()
.map(|status| SandboxInfo {
provider: self.kind.clone(),
status,
})
.collect())
}
async fn get(&self, id: &str) -> crate::Result<Option<SandboxInfo>> {
let Some(provider) = self.provider().await? else {
return Ok(None);
};
// An id the driver cannot even name is not one of ours.
let Ok(sandbox_id) = SandboxId::try_new(id) else {
return Ok(None);
};
let handle = match provider.attach(&sandbox_id, None).await {
Ok(handle) => handle,
// Unknown to the provider, or not fabro's: neither is in the
// inventory.
Err(DriverError::NotFound { .. } | DriverError::NotOwned { .. }) => return Ok(None),
Err(error) => {
return Err(crate::Error::context(
format!("Failed to look up {} sandbox '{id}'", self.kind),
error,
));
}
};
let status = handle.describe().await.map_err(|error| {
crate::Error::context(
format!("Failed to describe {} sandbox '{id}'", self.kind),
error,
)
})?;
if status.state == SandboxState::Deleted {
return Ok(None);
}
Ok(Some(SandboxInfo {
provider: self.kind.clone(),
status,
}))
}
}
/// The provider narrowed to fabro's sandboxes.
fn owned(provider: Arc<dyn DriverProvider>) -> Arc<dyn DriverProvider> {
Arc::new(OwnedProvider::new(
provider,
managed_labels::ownership(None),
))
}
#[derive(Debug, thiserror::Error)]
pub enum SandboxLookupError {
#[error("sandbox '{id}' was not found by any configured provider")]
@ -117,28 +274,6 @@ pub enum SandboxLookupError {
},
}
#[derive(Debug, Clone, Copy, Default)]
pub struct LocalSandboxProvider;
#[async_trait]
impl SandboxProvider for LocalSandboxProvider {
fn kind(&self) -> SandboxProviderKind {
SandboxProviderKind::LOCAL
}
async fn list(&self) -> crate::Result<Vec<SandboxInfo>> {
Ok(Vec::new())
}
async fn get(&self, _id: &str) -> crate::Result<Option<SandboxInfo>> {
Ok(None)
}
async fn delete(&self, _id: &str) -> crate::Result<()> {
Ok(())
}
}
fn provider_error(
provider: SandboxProviderKind,
err: &(dyn std::error::Error + 'static),
@ -151,170 +286,177 @@ fn provider_error(
#[cfg(test)]
mod tests {
use fabro_types::settings::server::SandboxPluginSettings;
use sandbox_driver::SandboxState;
use super::*;
use crate::test_support::{
FakeGet, FakeList, FakeSandboxProvider, fake_registry, fake_sandbox_info,
ScriptedSandbox, managed_scripted_sandbox, scripted_inventory_provider,
};
fn kind(name: &str) -> SandboxProviderKind {
SandboxProviderKind::try_new(name).expect("valid kind")
}
fn provider(kind: SandboxProviderKind, ids: &[&str]) -> ConnectedProvider {
scripted_inventory_provider(
kind,
ids.iter().map(|id| managed_scripted_sandbox(id)).collect(),
)
}
/// A plugin kind whose executable does not exist, so every lookup fails
/// to connect.
fn unreachable_plugin(inventory: SandboxInventory, name: &str) -> SandboxInventory {
let settings = ServerSandboxProviderSettings {
enabled: true,
plugin: Some(SandboxPluginSettings {
path: Some(format!("/nonexistent/fabro-sandbox-{name}")),
dev: true,
..SandboxPluginSettings::default()
}),
};
inventory.with_lazy(kind(name), settings, ProviderConnectOptions::default())
}
#[tokio::test]
async fn list_returns_aggregate_data_from_successful_providers() {
let docker = fake_sandbox_info(SandboxProviderKind::DOCKER, "docker-1");
let daytona = fake_sandbox_info(SandboxProviderKind::DAYTONA, "daytona-1");
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(vec![docker.clone()]),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(vec![daytona.clone()]),
FakeGet::Missing,
),
async fn list_aggregates_fabro_owned_sandboxes_across_providers() {
let foreign = Arc::new(
ScriptedSandbox::with_id_and_working_dir("someone-elses", "/work")
.state(SandboxState::Running),
);
let docker = scripted_inventory_provider(SandboxProviderKind::DOCKER, vec![
managed_scripted_sandbox("docker-1"),
foreign,
]);
let inventory = SandboxInventory::empty()
.with_host_directories(SandboxProviderKind::LOCAL)
.with_connected(docker)
.with_connected(provider(SandboxProviderKind::DAYTONA, &["daytona-1"]));
let response = registry.list_managed().await;
let response = inventory.list_managed().await;
assert_eq!(response.data, vec![docker, daytona]);
let mut ids: Vec<_> = response.data.iter().map(|s| s.status.id.as_str()).collect();
ids.sort_unstable();
assert_eq!(ids, ["daytona-1", "docker-1"]);
assert!(response.meta.provider_errors.is_empty());
}
#[tokio::test]
async fn list_includes_provider_error_metadata_when_one_provider_fails() {
let docker = fake_sandbox_info(SandboxProviderKind::DOCKER, "docker-1");
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(vec![docker.clone()]),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Err("daytona unavailable"),
FakeGet::Missing,
),
]);
let response = registry.list_managed().await;
assert_eq!(response.data, vec![docker]);
assert_eq!(response.meta.provider_errors, vec![
SandboxProviderLookupError {
provider: SandboxProviderKind::DAYTONA,
message: "daytona unavailable".to_string(),
}
let kinds: Vec<_> = inventory.kinds().cloned().collect();
assert_eq!(kinds, [
SandboxProviderKind::LOCAL,
SandboxProviderKind::DOCKER,
SandboxProviderKind::DAYTONA
]);
}
#[tokio::test]
async fn get_returns_one_matching_sandbox() {
let docker = fake_sandbox_info(SandboxProviderKind::DOCKER, "same-id");
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(docker.clone())),
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
]);
async fn list_reports_a_provider_that_cannot_connect_beside_the_others() {
let inventory = unreachable_plugin(
SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &["docker-1"])),
"e2b",
);
assert_eq!(
registry.get_managed_by_native_id("same-id").await.unwrap(),
docker
let response = inventory.list_managed().await;
assert_eq!(response.data.len(), 1);
assert_eq!(response.meta.provider_errors.len(), 1);
assert_eq!(response.meta.provider_errors[0].provider, kind("e2b"));
assert!(
response.meta.provider_errors[0]
.message
.contains("Failed to connect to the e2b provider"),
"{}",
response.meta.provider_errors[0].message
);
}
#[tokio::test]
async fn get_returns_not_found_when_all_providers_miss() {
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
]);
async fn get_finds_one_sandbox_by_native_id() {
let inventory = SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &[]))
.with_connected(provider(SandboxProviderKind::DAYTONA, &["native-id"]));
let err = registry
let sandbox = inventory
.get_managed_by_native_id("native-id")
.await
.expect("one provider matches");
assert_eq!(sandbox.status.id.as_str(), "native-id");
assert_eq!(sandbox.provider, SandboxProviderKind::DAYTONA);
}
#[tokio::test]
async fn get_reports_not_found_when_every_provider_misses() {
let inventory = SandboxInventory::empty()
.with_host_directories(SandboxProviderKind::LOCAL)
.with_connected(provider(SandboxProviderKind::DOCKER, &[]));
let error = inventory
.get_managed_by_native_id("missing")
.await
.unwrap_err();
.expect_err("nothing matches");
assert!(matches!(err, SandboxLookupError::NotFound { id } if id == "missing"));
assert!(matches!(error, SandboxLookupError::NotFound { id } if id == "missing"));
}
#[tokio::test]
async fn get_returns_conflict_when_two_providers_match() {
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(fake_sandbox_info(
SandboxProviderKind::DOCKER,
"same-id",
))),
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Found(Box::new(fake_sandbox_info(
SandboxProviderKind::DAYTONA,
"same-id",
))),
),
]);
async fn get_reports_a_conflict_when_two_providers_match() {
let inventory = SandboxInventory::empty()
.with_connected(provider(SandboxProviderKind::DOCKER, &["same-id"]))
.with_connected(provider(SandboxProviderKind::DAYTONA, &["same-id"]));
let err = registry
let error = inventory
.get_managed_by_native_id("same-id")
.await
.unwrap_err();
.expect_err("two providers match");
assert!(matches!(
err,
SandboxLookupError::Conflict { id, providers }
if id == "same-id"
&& providers == vec![SandboxProviderKind::DOCKER, SandboxProviderKind::DAYTONA]
));
let SandboxLookupError::Conflict { providers, .. } = error else {
panic!("expected a conflict, got {error:?}");
};
assert_eq!(providers, [
SandboxProviderKind::DOCKER,
SandboxProviderKind::DAYTONA
]);
}
#[tokio::test]
async fn get_returns_provider_unavailable_when_no_match_and_one_provider_fails() {
let registry = fake_registry(vec![
FakeSandboxProvider::new(
SandboxProviderKind::DOCKER,
FakeList::Ok(Vec::new()),
FakeGet::Missing,
),
FakeSandboxProvider::new(
SandboxProviderKind::DAYTONA,
FakeList::Ok(Vec::new()),
FakeGet::Err("daytona unavailable"),
),
]);
async fn get_is_unavailable_when_no_match_and_a_provider_failed() {
let inventory = unreachable_plugin(
SandboxInventory::empty().with_connected(provider(SandboxProviderKind::DOCKER, &[])),
"e2b",
);
let err = registry
let error = inventory
.get_managed_by_native_id("maybe-missing")
.await
.unwrap_err();
.expect_err("the failed provider may have held it");
assert!(matches!(
err,
SandboxLookupError::ProviderUnavailable {
id,
provider_errors
} if id == "maybe-missing"
&& provider_errors == vec![SandboxProviderLookupError {
provider: SandboxProviderKind::DAYTONA,
message: "daytona unavailable".to_string(),
}]
let SandboxLookupError::ProviderUnavailable {
provider_errors, ..
} = error
else {
panic!("expected provider unavailable, got {error:?}");
};
assert_eq!(provider_errors.len(), 1);
assert_eq!(provider_errors[0].provider, kind("e2b"));
}
#[tokio::test]
async fn get_ignores_a_sandbox_without_the_managed_label() {
let foreign = Arc::new(
ScriptedSandbox::with_id_and_working_dir("foreign", "/work")
.state(SandboxState::Running),
);
let inventory = SandboxInventory::empty().with_connected(scripted_inventory_provider(
SandboxProviderKind::DOCKER,
vec![foreign],
));
let error = inventory
.get_managed_by_native_id("foreign")
.await
.expect_err("a foreign sandbox is not in the inventory");
assert!(matches!(error, SandboxLookupError::NotFound { .. }));
}
}

234
lib/components/fabro-sandbox/src/provider/driver.rs
View file

@ -1,234 +0,0 @@
//! Fabro-managed inventory over a sandbox-driver provider.
//!
//! Lists and looks up the sandboxes fabro created, identified by fabro's
//! own `sh.fabro.managed` label. The driver marks every sandbox it creates
//! with its own label too, but that covers every application on the same
//! daemon or account; the provider is connected through the driver's
//! ownership scope, which lists only fabro's sandboxes and refuses to
//! attach to or delete any other.
use std::sync::Arc;
use async_trait::async_trait;
use fabro_types::settings::server::ServerSandboxProviderSettings;
use fabro_types::{SandboxInfo, SandboxProviderKind};
use sandbox_driver::{
Error as DriverError, OwnedProvider, SandboxFilter, SandboxId,
SandboxProvider as DriverProvider,
};
use tokio::sync::OnceCell;
use super::SandboxProvider;
use crate::driver::{ConnectedProvider, ProviderConnectOptions, connect_provider};
use crate::{details, managed_labels};
/// How the driver provider behind the inventory is obtained.
enum Connection {
Connected(Arc<dyn DriverProvider>),
/// Connected on first use, so a registry can be assembled synchronously
/// and a provider that is down surfaces as a lookup error rather than a
/// startup failure.
Lazy(Box<LazyConnection>),
}
struct LazyConnection {
settings: ServerSandboxProviderSettings,
options: ProviderConnectOptions,
provider: OnceCell<Arc<dyn DriverProvider>>,
}
pub struct DriverInventoryProvider {
kind: SandboxProviderKind,
connection: Connection,
}
impl DriverInventoryProvider {
#[must_use]
pub fn new(connected: ConnectedProvider) -> Self {
Self {
kind: connected.kind,
connection: Connection::Connected(owned(connected.provider)),
}
}
/// An inventory over a provider connected through
/// [`connect_provider`] on first use.
#[must_use]
pub fn lazy(
kind: SandboxProviderKind,
settings: ServerSandboxProviderSettings,
options: ProviderConnectOptions,
) -> Self {
Self {
kind,
connection: Connection::Lazy(Box::new(LazyConnection {
settings,
options,
provider: OnceCell::new(),
})),
}
}
async fn provider(&self) -> crate::Result<&Arc<dyn DriverProvider>> {
match &self.connection {
Connection::Connected(provider) => Ok(provider),
Connection::Lazy(lazy) => {
lazy.provider
.get_or_try_init(|| async {
connect_provider(&self.kind, &lazy.settings, &lazy.options)
.await
.map(|connected| owned(connected.provider))
.map_err(|error| {
crate::Error::context(
format!("Failed to connect to the {} provider", self.kind),
error,
)
})
})
.await
}
}
}
async fn describe_managed(
&self,
id: &str,
) -> crate::Result<Option<sandbox_driver::SandboxStatus>> {
// An id the driver cannot even name is not one of ours.
let Ok(sandbox_id) = SandboxId::try_new(id) else {
return Ok(None);
};
let handle = match self.provider().await?.attach(&sandbox_id, None).await {
Ok(handle) => handle,
// Unknown to the provider, or not fabro's: neither is in the
// inventory.
Err(DriverError::NotFound { .. } | DriverError::NotOwned { .. }) => return Ok(None),
Err(error) => {
return Err(crate::Error::context(
format!("Failed to look up {} sandbox '{id}'", self.kind),
error,
));
}
};
let status = handle.describe().await.map_err(|error| {
crate::Error::context(
format!("Failed to describe {} sandbox '{id}'", self.kind),
error,
)
})?;
if status.state == sandbox_driver::SandboxState::Deleted {
return Ok(None);
}
Ok(Some(status))
}
}
/// The provider narrowed to fabro's sandboxes.
fn owned(provider: Arc<dyn DriverProvider>) -> Arc<dyn DriverProvider> {
Arc::new(OwnedProvider::new(
provider,
managed_labels::ownership(None),
))
}
#[async_trait]
impl SandboxProvider for DriverInventoryProvider {
fn kind(&self) -> SandboxProviderKind {
self.kind.clone()
}
async fn list(&self) -> crate::Result<Vec<SandboxInfo>> {
let statuses = self
.provider()
.await?
.list(&SandboxFilter::default())
.await
.map_err(|error| {
crate::Error::context(format!("Failed to list {} sandboxes", self.kind), error)
})?;
Ok(statuses
.iter()
.map(|status| details::info_from_status(&self.kind, status))
.collect())
}
async fn get(&self, id: &str) -> crate::Result<Option<SandboxInfo>> {
Ok(self
.describe_managed(id)
.await?
.map(|status| details::info_from_status(&self.kind, &status)))
}
async fn delete(&self, id: &str) -> crate::Result<()> {
// Missing or already deleted is an idempotent success; the scope
// refuses a sandbox that is not fabro's, which must never be
// deleted here.
let Ok(sandbox_id) = SandboxId::try_new(id) else {
return Ok(());
};
match self.provider().await?.delete(&sandbox_id, None).await {
Ok(()) => Ok(()),
Err(DriverError::NotOwned { .. }) => Err(crate::Error::message(format!(
"Refusing to delete {} sandbox '{id}' because it is missing label {}={}",
self.kind,
managed_labels::MANAGED_LABEL,
managed_labels::MANAGED_LABEL_VALUE
))),
Err(error) => Err(crate::Error::context(
format!("Failed to delete {} sandbox '{id}'", self.kind),
error,
)),
}
}
}
#[cfg(test)]
mod tests {
use sandbox_driver::{SandboxSource, SandboxSpec};
use sandbox_driver_host::HostProvider;
use super::*;
fn inventory() -> (DriverInventoryProvider, Arc<HostProvider>) {
let host = Arc::new(HostProvider::new());
let provider = DriverInventoryProvider::new(ConnectedProvider {
kind: SandboxProviderKind::try_new("host").unwrap(),
provider: host.clone(),
});
(provider, host)
}
#[tokio::test]
async fn lists_and_deletes_only_fabro_managed_sandboxes() {
let (inventory, host) = inventory();
let ours = host
.create(
&SandboxSpec::new(SandboxSource::HostDirectory)
.label(managed_labels::MANAGED_LABEL, "true"),
None,
)
.await
.unwrap();
let theirs = host
.create(&SandboxSpec::new(SandboxSource::HostDirectory), None)
.await
.unwrap();
let listed = inventory.list().await.unwrap();
assert_eq!(listed.len(), 1);
assert_eq!(listed[0].id, ours.id().as_str());
assert_eq!(listed[0].provider.as_str(), "host");
assert!(inventory.get(ours.id().as_str()).await.unwrap().is_some());
assert!(inventory.get(theirs.id().as_str()).await.unwrap().is_none());
let refused = inventory.delete(theirs.id().as_str()).await.unwrap_err();
assert!(
refused.to_string().contains("Refusing to delete"),
"{refused}"
);
inventory.delete(ours.id().as_str()).await.unwrap();
assert!(inventory.get(ours.id().as_str()).await.unwrap().is_none());
inventory.delete(ours.id().as_str()).await.unwrap();
inventory.delete("never-existed").await.unwrap();
}
}

152
lib/components/fabro-sandbox/src/provider_sandbox.rs
View file

@ -1,96 +1,107 @@
//! Run sandboxes on any provider fabro can name: a bundled kind in process
//! or a sandbox-driver plugin executable.
//!
//! One path builds them all. The environment's [`SandboxOptions`] become
//! the driver spec once, the provider is connected through the single
//! One path builds them all. The environment's spec arrives built (see
//! [`crate::environment`]), the provider is connected through the single
//! construction function, and a bundled provider adds only what its
//! backend needs on top: Docker its fixed working directory and default
//! image, Daytona the snapshot it creates sandboxes from and its lifecycle
//! timers. A plugin gets the spec as is, laid out inside the working
//! directory the provider chooses.
//! image, Daytona its fixed working directory, default snapshot, and
//! lifecycle timers, the Host the designated directory it works in,
//! created when missing. A plugin gets the spec as is, trimmed to what it
//! can honor, laid out inside the working directory the provider chooses.
use std::path::PathBuf;
use std::sync::Arc;
use fabro_github::GitHubCredentials;
use fabro_types::{BundledProvider, RunId, SandboxProviderKind};
use sandbox_driver::{EventContext, OwnedProvider, SandboxId, SandboxProvider};
use sandbox_driver::{
EventContext, OwnedProvider, SandboxId, SandboxProvider, SandboxSource,
SandboxSpec as DriverSpec,
};
use tokio::fs;
use crate::driver::{ProviderAccess, connect_provider};
use crate::driver_sandbox::{LayoutSource, RepoWorkspace, RunSandbox};
use crate::options::{self, SandboxOptions};
use crate::environment::{self, CloneRequest};
use crate::sandbox_spec::SandboxSpec;
use crate::{daytona, docker, managed_labels};
/// A sandbox for a run on `kind`. The sandbox is created by `initialize`;
/// construction validates the clone request and connects the provider, so
/// a bad spec, a missing credential, or a missing plugin executable fails
/// before any backend call.
#[expect(
clippy::too_many_arguments,
reason = "mirrors SandboxSpec::Provider; clone inputs are validated together"
)]
/// a bad request, a missing credential, or a missing plugin executable
/// fails before any backend call.
pub async fn provider_sandbox(
kind: SandboxProviderKind,
access: &ProviderAccess,
options: SandboxOptions,
spec: DriverSpec,
clone: &CloneRequest,
github_app: Option<&GitHubCredentials>,
run_id: Option<RunId>,
clone_origin_url: Option<String>,
clone_branch: Option<String>,
clone_tag: Option<String>,
clone_commit_sha: Option<String>,
) -> crate::Result<RunSandbox> {
let workspace = RepoWorkspace::plan(
layout_source(&kind),
options.skip_clone,
clone_origin_url.as_deref(),
clone_branch.as_deref(),
clone_tag.as_deref(),
clone_commit_sha.as_deref(),
options.clone_depth,
github_app,
)?;
let workspace = RepoWorkspace::plan(layout_source(&kind), clone, github_app)?;
let provider = connect(&kind, access, run_id.as_ref()).await?;
let base = options::base_spec(&options, run_id.as_ref());
let mut spec = spec;
if let Some(run_id) = &run_id {
spec = spec.name(environment::run_name(run_id));
}
Ok(match kind.bundled() {
Some(BundledProvider::Docker) => {
let (spec, _image) = docker::overlay(base, &options);
RunSandbox::pending(kind, provider, spec, workspace)
RunSandbox::pending(kind, provider, docker::overlay(spec), workspace)
}
Some(BundledProvider::Daytona) => {
let credentials = access
.daytona
.as_ref()
.ok_or_else(|| crate::Error::message(MISSING_DAYTONA_CREDENTIALS))?;
let plan = daytona::create_plan(
Arc::clone(&provider),
credentials.api_key.clone(),
base,
options,
run_id,
);
RunSandbox::pending_with_plan(kind, provider, Box::new(plan), workspace)
}
Some(BundledProvider::Local) => {
return Err(crate::Error::message(
"local sandboxes are built from a working directory, not a provider spec",
));
}
None => {
let mut spec = base;
spec.network = options::supported_network(spec.network, provider.capabilities());
Some(BundledProvider::Daytona) => RunSandbox::pending(
kind,
provider,
daytona::overlay(spec, run_id.as_ref()),
workspace,
),
Some(BundledProvider::Local) | None => {
if kind.is_local() {
designate_directory(&spec).await?;
}
let capabilities = provider.capabilities();
spec.network = environment::supported_network(spec.network, capabilities);
spec.timers = environment::supported_timers(spec.timers, capabilities);
RunSandbox::pending(kind, provider, spec, workspace)
}
})
}
/// The Host provider works in a designated directory in place and needs it
/// to exist. A run may point at a fresh scratch path, so the directory is
/// created before the provider sees the spec.
async fn designate_directory(spec: &DriverSpec) -> crate::Result<()> {
let Some(directory) = &spec.working_directory else {
return Ok(());
};
fs::create_dir_all(directory).await.map_err(|error| {
crate::Error::context(
format!("Failed to create working directory {directory}"),
error,
)
})
}
/// A sandbox on this host at `working_directory`, ready to use: the `local`
/// kind, built through the provider path with default settings and
/// initialized. For the agent CLI and tests; a run builds its sandbox from
/// its [`SandboxSpec`] and initializes it itself.
pub async fn local_sandbox(working_directory: impl Into<PathBuf>) -> crate::Result<RunSandbox> {
let spec = SandboxSpec::local(working_directory, ProviderAccess::default());
let sandbox =
provider_sandbox(spec.kind, &spec.access, spec.spec, &spec.clone, None, None).await?;
sandbox.initialize().await?;
Ok(sandbox)
}
/// Reattach to a run's sandbox on `kind` by its persisted id. The driver
/// reports the sandbox's lifecycle from here on through `events`.
///
/// The sandbox must carry fabro's managed label and, when a run id is
/// known, the matching run label: the provider shares its backend with
/// every other application, and fabro never operates on a sandbox it did
/// not create. The ownership scope the provider is connected through
/// refuses anything else.
/// On a shared backend the sandbox must carry fabro's managed label and,
/// when a run id is known, the matching run label: fabro never operates on
/// a sandbox it did not create, and the ownership scope the provider is
/// connected through refuses anything else. A local sandbox attaches by
/// the id the Host provider derives from its directory.
pub async fn attach_provider_sandbox(
kind: SandboxProviderKind,
access: &ProviderAccess,
@ -117,24 +128,20 @@ pub async fn attach_provider_sandbox(
working_directory,
clone_origin_url,
);
let sandbox = RunSandbox::attached(kind.clone(), handle, workspace);
if kind.bundled() == Some(BundledProvider::Daytona) {
if let Some(snapshot) = status.source {
sandbox.set_snapshot(snapshot);
}
let sandbox = RunSandbox::attached(kind, handle, workspace);
if let Some(snapshot) = status.snapshot {
sandbox.set_snapshot(snapshot);
}
Ok(sandbox)
}
/// The image the run record names for a sandbox on `kind`: the
/// environment's, or Docker's default when the environment names none.
pub(crate) fn recorded_image(
kind: &SandboxProviderKind,
options: &SandboxOptions,
) -> Option<String> {
match kind.bundled() {
Some(BundledProvider::Docker) => Some(docker::effective_image(options)),
_ => options.image.clone(),
pub(crate) fn recorded_image(kind: &SandboxProviderKind, spec: &DriverSpec) -> Option<String> {
match (kind.bundled(), &spec.source) {
(Some(BundledProvider::Docker), _) => Some(docker::effective_image(spec)),
(_, SandboxSource::Image { reference }) => Some(reference.clone()),
_ => None,
}
}
@ -179,6 +186,11 @@ async fn connect(
.map_err(|error| {
crate::Error::context(format!("Failed to connect to the {kind} provider"), error)
})?;
// A local sandbox is a directory the caller designated; it carries no
// labels, and nothing else shares the host's directories with fabro.
if kind.bundled() == Some(BundledProvider::Local) {
return Ok(connected.provider);
}
Ok(Arc::new(OwnedProvider::new(
connected.provider,
managed_labels::ownership(run_id),

629
lib/components/fabro-sandbox/src/push_credentials.rs
View file

@ -1,629 +0,0 @@
//! Shared push-credential state for clone-based sandbox providers.
//!
//! Docker and Daytona embed GitHub credentials into the cloned repository's
//! `origin` remote and refresh them before pushes. Both providers hold this
//! state so the compare → `set-url` → record sequence, the generation
//! tracking, and the refresh-error logging behave identically across
//! providers. The token cache itself sits below the providers, in
//! [`fabro_github::token_source::InstallationTokenSource`].
use std::future::Future;
use std::sync::Arc;
use fabro_github::GitHubCredentials;
use fabro_github::token_source::{InstallationTokenSource, ResolvedToken, TokenSnapshot};
use fabro_redact::DisplaySafeUrl;
pub use fabro_types::run_event::GitCredentialRefreshError as RefreshErrorKind;
use tokio::sync::{Mutex, MutexGuard};
use crate::redact;
use crate::sandbox::{RefreshOutcome, RemoteCredentialAction};
/// Build the shared installation-token source for a clone-based sandbox.
///
/// Returns `None` when there are no managed credentials or no GitHub origin
/// to scope them to. Minted tokens carry the same `contents: write`
/// permission the clone token uses.
pub(crate) fn build_token_source(
github_app: Option<&GitHubCredentials>,
clone_origin_url: Option<&str>,
) -> crate::Result<Option<Arc<InstallationTokenSource>>> {
let Some(creds) = github_app else {
return Ok(None);
};
let Some(origin_url) = clone_origin_url.filter(|url| !url.trim().is_empty()) else {
return Ok(None);
};
let normalized = fabro_github::normalize_repo_origin_url(origin_url);
let Ok((owner, repo)) = fabro_github::parse_github_owner_repo(&normalized) else {
// Non-GitHub origins never clone in these providers, so there is no
// remote to keep credentials fresh for.
return Ok(None);
};
InstallationTokenSource::for_repository(
creds,
owner,
repo,
serde_json::json!({ "contents": "write" }),
)
.map(Some)
.map_err(|err| crate::Error::context_anyhow("Failed to build GitHub token source", err))
}
/// Push-credential state one provider instance tracks for its `origin`
/// remote.
pub(crate) struct PushCredentialState {
source: Option<Arc<InstallationTokenSource>>,
/// Serializes compare → `set-url` → record. The token source's
/// single-flight ends before the sandbox exec, so without this lock a
/// refresh-ahead tick and a push could both see the old embedded
/// generation and race on `.git/config.lock`. Holds the last
/// successfully embedded token: its secret is already in the remote URL
/// inside the sandbox, so retaining it adds no exposure, and it is what
/// a push falls back to when a refresh fails. The tracked value is local
/// belief, not ground truth — agent code inside the sandbox can rewrite
/// `origin`.
embedded: Mutex<Option<ResolvedToken>>,
}
impl PushCredentialState {
pub(crate) fn new(source: Option<Arc<InstallationTokenSource>>) -> Self {
Self {
source,
embedded: Mutex::new(None),
}
}
pub(crate) fn source(&self) -> Option<&Arc<InstallationTokenSource>> {
self.source.as_ref()
}
/// Record the token embedded in `origin` outside the refresh path — the
/// clone is the first operation to embed a token, and it seeds this
/// state so the first refresh compares against the clone token instead
/// of believing nothing was ever embedded.
pub(crate) async fn record_embedded(&self, token: ResolvedToken) {
*self.embedded.lock().await = Some(token);
}
/// Refresh the credentials embedded in `origin`.
///
/// Resolves through the shared source, skips the `set-url` exec when the
/// resolved generation is already embedded, and records the new
/// generation only after `set_url` succeeds. `set_url` receives the
/// authenticated URL to embed and runs under the embed lock.
pub(crate) async fn refresh<F, Fut>(
&self,
origin_url: &str,
set_url: F,
) -> crate::Result<RefreshOutcome>
where
F: FnOnce(DisplaySafeUrl) -> Fut,
Fut: Future<Output = crate::Result<()>>,
{
let Some(source) = &self.source else {
return Ok(RefreshOutcome::none());
};
let mut embedded = self.embedded.lock().await;
let resolved = match source.resolve().await {
Ok(resolved) => resolved,
Err(err) => {
// The refresh-error path is defined, not incidental: the push
// proceeds with the last embedded token, so log which one
// that is instead of losing the credential state.
if let Some(prev) = embedded.as_ref() {
tracing::warn!(
error = %format!("{err:#}"),
generation = prev.snapshot.generation,
provenance = %prev.snapshot.provenance,
token_age_ms = prev.snapshot.age_ms(),
"GitHub token refresh failed; origin keeps the last embedded credentials"
);
} else {
tracing::warn!(
error = %format!("{err:#}"),
"GitHub token refresh failed and no credentials were ever embedded"
);
}
return Err(crate::Error::context_anyhow(
"Failed to refresh push credentials",
err,
));
}
};
if embedded
.as_ref()
.is_some_and(|prev| prev.snapshot.generation == resolved.snapshot.generation)
{
return Ok(RefreshOutcome::unchanged(resolved.snapshot));
}
let auth_url = fabro_github::embed_token_in_url(origin_url, resolved.token.expose())
.map_err(|err| {
crate::Error::context_anyhow("Failed to build authenticated origin URL", err)
})?;
set_url(auth_url).await?;
let snapshot = resolved.snapshot;
*embedded = Some(resolved);
Ok(RefreshOutcome::embedded(snapshot))
}
}
/// What [`CredentialLease::ensure_embedded`] did for one push attempt.
#[derive(Debug, Clone, Copy)]
pub(crate) struct EnsureOutcome {
pub action: RemoteCredentialAction,
/// The token embedded in the remote right now — never an unembedded mint.
pub token: Option<TokenSnapshot>,
pub refresh_error: Option<RefreshErrorKind>,
}
/// Scoped pin of push credentials for one push operation.
///
/// Holds the provider's embed mutex until dropped, so no other refresh can
/// re-embed mid-operation — a refresh-ahead tick crossing the cache margin
/// during a retrying push waits here instead of swapping the remote out from
/// under the pin. Internally retains up to two secrets: the last successfully
/// embedded token (the fallback) and the operation's resolved target, so both
/// drift re-embedding and the refresh-error fallback work. Only non-secret
/// snapshots leave the lease.
///
/// A successful resolve happens at most once per operation and is never
/// replaced; the pin transitions to the target only through a successful
/// embed. The token source's refresh margin exceeds every push plan's elapsed
/// bound, so the pinned token always outlives the operation.
pub(crate) struct CredentialLease<'a> {
source: Option<&'a InstallationTokenSource>,
/// Embed-mutex guard: the last successfully embedded token.
embedded: MutexGuard<'a, Option<ResolvedToken>>,
/// The operation's resolved target, including a cached fallback when a
/// refresh mint failed.
target: Option<ResolvedToken>,
/// Skip an immediate duplicate resolve after lease acquisition already
/// failed. A later push attempt can retry after backoff.
defer_resolve_once: bool,
}
impl PushCredentialState {
/// Acquire the push-credential lease for one push operation.
///
/// Resolves the operation's target token up front. A failed refresh can
/// return a valid cached token; the first attempt uses it, and
/// [`CredentialLease::ensure_embedded`] retries the refresh after push
/// backoff. A resolve with no cached or embedded token fails acquisition.
pub(crate) async fn lease(&self) -> crate::Result<CredentialLease<'_>> {
let embedded = self.embedded.lock().await;
let Some(source) = self.source.as_deref() else {
return Ok(CredentialLease {
source: None,
embedded,
target: None,
defer_resolve_once: false,
});
};
match source.resolve().await {
Ok(resolved) => {
let defer_resolve_once = resolved.refresh_failed;
Ok(CredentialLease {
source: Some(source),
embedded,
target: Some(resolved),
defer_resolve_once,
})
}
Err(err) => {
if let Some(prev) = embedded.as_ref() {
tracing::warn!(
error = %format!("{err:#}"),
generation = prev.snapshot.generation,
provenance = %prev.snapshot.provenance,
token_age_ms = prev.snapshot.age_ms(),
"token resolve failed; push pins the last embedded credentials"
);
Ok(CredentialLease {
source: Some(source),
embedded,
target: None,
defer_resolve_once: true,
})
} else {
tracing::warn!(
error = %format!("{err:#}"),
"token resolve failed and no credentials were ever embedded"
);
Err(crate::Error::message(
"Failed to refresh push credentials: token_mint_failed",
))
}
}
}
}
}
impl CredentialLease<'_> {
/// Non-secret description of the token embedded in the remote right now.
pub(crate) fn snapshot(&self) -> Option<TokenSnapshot> {
self.embedded.as_ref().map(|token| token.snapshot)
}
/// Embed the pinned generation if the remote does not carry it.
///
/// One call covers the initial embed, a deferred embed after an earlier
/// failure, and drift repair (`force` re-embeds even when the tracked
/// generation matches, for remotes rewritten inside the sandbox). While
/// the lease has no target, this retries the failed `resolve()` first —
/// retrying a failed resolve discards no fresh token, so it cannot
/// restart any replication clock. Refresh failures are recorded, never
/// propagated: the push proceeds with the last embedded token.
pub(crate) async fn ensure_embedded(
&mut self,
sandbox: &crate::RunSandbox,
origin_url: &str,
force: bool,
) -> crate::Result<EnsureOutcome> {
let Some(source) = self.source else {
return Ok(EnsureOutcome {
action: RemoteCredentialAction::None,
token: None,
refresh_error: None,
});
};
let mut refresh_error = self.defer_resolve_once.then_some(RefreshErrorKind::Mint);
if self.defer_resolve_once {
self.defer_resolve_once = false;
} else if self
.target
.as_ref()
.is_none_or(|resolved| resolved.refresh_failed)
{
match source.resolve().await {
Ok(resolved) => {
refresh_error = resolved.refresh_failed.then_some(RefreshErrorKind::Mint);
self.target = Some(resolved);
}
Err(err) => {
tracing::warn!(
error = %format!("{err:#}"),
"token resolve retry failed; pushing with the last embedded token"
);
refresh_error = Some(RefreshErrorKind::Mint);
}
}
}
let Some(desired) = self.target.as_ref().or(self.embedded.as_ref()).cloned() else {
// Managed credentials with nothing resolved or embedded:
// acquisition fails before any attempt runs, so pushes never see
// this state.
return Ok(EnsureOutcome {
action: RemoteCredentialAction::None,
token: None,
refresh_error,
});
};
let embedded_generation = self
.embedded
.as_ref()
.map(|token| token.snapshot.generation);
if !force && embedded_generation == Some(desired.snapshot.generation) {
return Ok(EnsureOutcome {
action: RemoteCredentialAction::Unchanged,
token: Some(desired.snapshot),
refresh_error,
});
}
match set_url_via_exec(sandbox, origin_url, &desired).await {
Ok(()) => {
let snapshot = desired.snapshot;
*self.embedded = Some(desired);
Ok(EnsureOutcome {
action: RemoteCredentialAction::Embedded,
token: Some(snapshot),
refresh_error,
})
}
Err(err) => {
if matches!(
&err,
crate::Error::Exec { result, .. }
if result.termination != fabro_types::CommandTermination::Exited
) {
return Err(err);
}
tracing::warn!(
error = %crate::display_for_log(&err),
"embedding push credentials in origin failed; pushing with the last embedded token"
);
Ok(EnsureOutcome {
action: RemoteCredentialAction::Unchanged,
token: self.snapshot(),
refresh_error: Some(RefreshErrorKind::SetUrl),
})
}
}
}
}
/// Rewrite `origin` with the token embedded, through the sandbox's uniform
/// exec surface.
async fn set_url_via_exec(
sandbox: &crate::RunSandbox,
origin_url: &str,
token: &ResolvedToken,
) -> crate::Result<()> {
let auth_url =
fabro_github::embed_token_in_url(origin_url, token.token.expose()).map_err(|err| {
crate::Error::context(
"Failed to build authenticated origin URL",
RedactedSetUrlError(fabro_redact::redact_string(&format!("{err:#}"))),
)
})?;
set_auth_url_via_exec(sandbox, auth_url).await
}
pub(crate) async fn set_auth_url_via_exec(
sandbox: &crate::RunSandbox,
auth_url: DisplaySafeUrl,
) -> crate::Result<()> {
let command = format!(
"git -c maintenance.auto=0 remote set-url origin {}",
crate::shell_quote(auth_url.as_raw_url().as_str())
);
let result = sandbox
.exec_command(&command, 10_000, None, None, None)
.await
.map_err(|err| {
let message = redact::redact_auth_url(&crate::display_for_log(&err), Some(&auth_url));
crate::Error::context(
"Failed to refresh push credentials: set_url_exec_failed",
RedactedSetUrlError(message),
)
})?;
if !result.is_success() {
return Err(result.into_exec_error_with_redactor(
"git remote set-url origin (refresh push credentials)",
|s| redact::redact_auth_url(s, Some(&auth_url)),
));
}
Ok(())
}
#[derive(Debug, thiserror::Error)]
#[error("{0}")]
struct RedactedSetUrlError(String);
#[cfg(test)]
mod tests {
use std::sync::atomic::{AtomicUsize, Ordering};
use chrono::Utc;
use fabro_github::InstallationToken;
use fabro_github::test_support::{InstallationTokenMinter, installation_token_source};
use tokio::time::sleep;
use super::*;
use crate::sandbox::RemoteCredentialAction;
struct FixedMinter {
calls: AtomicUsize,
ttl: chrono::Duration,
}
#[async_trait::async_trait]
impl InstallationTokenMinter for FixedMinter {
async fn mint(&self) -> anyhow::Result<InstallationToken> {
let call = self.calls.fetch_add(1, Ordering::SeqCst) + 1;
Ok(InstallationToken {
token: format!("ghs_gen{call}"),
expires_at: Utc::now() + self.ttl,
})
}
}
struct FailingMinter;
#[async_trait::async_trait]
impl InstallationTokenMinter for FailingMinter {
async fn mint(&self) -> anyhow::Result<InstallationToken> {
Err(anyhow::anyhow!("mint failed"))
}
}
fn minting_state(ttl: chrono::Duration) -> PushCredentialState {
PushCredentialState::new(Some(installation_token_source(
"owner/repo",
Arc::new(FixedMinter {
calls: AtomicUsize::new(0),
ttl,
}),
)))
}
const ORIGIN: &str = "https://github.com/owner/repo";
/// Long enough for a blocked task to be observably pending on paused time.
const SHORT_WAIT: std::time::Duration = std::time::Duration::from_secs(5);
/// A refresh-ahead tick crossing the cache margin during a push waits on
/// the embed mutex until the operation releases the lease, so the remote
/// can never be swapped out from under the pinned generation.
#[tokio::test(start_paused = true)]
async fn refresh_waits_for_the_lease_to_release() {
let state = std::sync::Arc::new(minting_state(chrono::Duration::minutes(60)));
let lease = state.lease().await.expect("lease acquires");
let refresh_task = {
let state = std::sync::Arc::clone(&state);
tokio::spawn(async move {
state
.refresh(ORIGIN, |_| async { Ok(()) })
.await
.expect("refresh succeeds after the lease releases")
})
};
// The refresh must be blocked while the lease holds the embed mutex.
sleep(SHORT_WAIT).await;
assert!(
!refresh_task.is_finished(),
"refresh must wait on the embed mutex"
);
drop(lease);
let outcome = refresh_task.await.expect("refresh task completes");
// The lease's resolve minted generation 1; the deferred refresh
// reuses it (the operation never embedded, so the refresh embeds).
assert_eq!(outcome.token().unwrap().generation, 1);
}
#[tokio::test]
async fn refresh_without_managed_credentials_reports_none() {
let state = PushCredentialState::new(None);
let outcome = state
.refresh(ORIGIN, |_| async { panic!("set-url must not run") })
.await
.unwrap();
assert_eq!(outcome, RefreshOutcome::none());
}
#[tokio::test]
async fn refresh_embeds_a_new_generation_and_skips_matching_ones() {
let state = minting_state(chrono::Duration::minutes(60));
let set_url_calls = AtomicUsize::new(0);
let first = state
.refresh(ORIGIN, |auth_url| {
set_url_calls.fetch_add(1, Ordering::SeqCst);
assert!(auth_url.as_raw_url().as_str().contains("ghs_gen1"));
async { Ok(()) }
})
.await
.unwrap();
assert_eq!(first.action(), RemoteCredentialAction::Embedded);
assert_eq!(first.token().unwrap().generation, 1);
// The cached token is fresh, so the second refresh must skip set-url.
let second = state
.refresh(ORIGIN, |_| {
set_url_calls.fetch_add(1, Ordering::SeqCst);
async { Ok(()) }
})
.await
.unwrap();
assert_eq!(second.action(), RemoteCredentialAction::Unchanged);
assert_eq!(second.token().unwrap().generation, 1);
assert_eq!(set_url_calls.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn refresh_embeds_again_when_the_source_mints_a_new_generation() {
// Tokens expire inside the margin, so every resolve re-mints.
let state = minting_state(chrono::Duration::minutes(5));
let set_url_calls = AtomicUsize::new(0);
let first = state
.refresh(ORIGIN, |_| {
set_url_calls.fetch_add(1, Ordering::SeqCst);
async { Ok(()) }
})
.await
.unwrap();
let second = state
.refresh(ORIGIN, |_| {
set_url_calls.fetch_add(1, Ordering::SeqCst);
async { Ok(()) }
})
.await
.unwrap();
assert_eq!(first.token().unwrap().generation, 1);
assert_eq!(second.action(), RemoteCredentialAction::Embedded);
assert_eq!(second.token().unwrap().generation, 2);
assert_eq!(set_url_calls.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn clone_seed_makes_the_first_refresh_a_no_op() {
let state = minting_state(chrono::Duration::minutes(60));
let clone_token = state.source().unwrap().mint_for_clone().await.unwrap();
state.record_embedded(clone_token).await;
let outcome = state
.refresh(ORIGIN, |_| async { panic!("set-url must not run") })
.await
.unwrap();
assert_eq!(outcome.action(), RemoteCredentialAction::Unchanged);
assert_eq!(outcome.token().unwrap().generation, 1);
}
#[tokio::test]
async fn failed_set_url_does_not_record_the_new_generation() {
let state = minting_state(chrono::Duration::minutes(60));
let err = state
.refresh(ORIGIN, |_| async {
Err(crate::Error::message("set-url failed"))
})
.await
.unwrap_err();
assert!(err.to_string().contains("set-url failed"));
// The generation was not recorded, so the retry embeds again instead
// of wrongly skipping.
let retried = state.refresh(ORIGIN, |_| async { Ok(()) }).await.unwrap();
assert_eq!(retried.action(), RemoteCredentialAction::Embedded);
assert_eq!(retried.token().unwrap().generation, 1);
}
#[tokio::test]
async fn static_credentials_seeded_at_clone_skip_set_url() {
let source = InstallationTokenSource::for_origin(
&GitHubCredentials::Pat("ghp_pat".to_string()),
ORIGIN,
serde_json::json!({ "contents": "write" }),
)
.unwrap();
let state = PushCredentialState::new(Some(source));
let clone_token = state.source().unwrap().mint_for_clone().await.unwrap();
state.record_embedded(clone_token).await;
let outcome = state
.refresh(ORIGIN, |_| async { panic!("set-url must not run") })
.await
.unwrap();
assert_eq!(outcome.action(), RemoteCredentialAction::Unchanged);
assert!(outcome.token().unwrap().is_static());
}
#[tokio::test]
async fn mint_failure_preserves_the_mint_error_chain() {
let state = PushCredentialState::new(Some(installation_token_source(
"owner/repo",
Arc::new(FailingMinter),
)));
let err = state
.refresh(ORIGIN, |_| async { panic!("set-url must not run") })
.await
.unwrap_err();
assert_eq!(err.causes(), vec![
"minting GitHub installation access token",
"mint failed"
]);
}
#[test]
fn token_source_requires_managed_credentials_and_a_github_origin() {
assert!(build_token_source(None, Some(ORIGIN)).unwrap().is_none());
let pat = GitHubCredentials::Pat("ghp_pat".to_string());
assert!(build_token_source(Some(&pat), None).unwrap().is_none());
assert!(
build_token_source(Some(&pat), Some("https://gitlab.com/owner/repo"))
.unwrap()
.is_none()
);
assert!(
build_token_source(Some(&pat), Some(ORIGIN))
.unwrap()
.is_some()
);
}
}

96
lib/components/fabro-sandbox/src/reconnect.rs
View file

@ -1,74 +1,52 @@
use std::path::PathBuf;
use anyhow::{Context, Result};
use fabro_types::{BundledProvider, RunId, RunSandboxInstance};
use sandbox_driver::EventContext;
use fabro_types::{RunId, RunSandboxInstance};
use sandbox_driver::{EventContext, PtySession, PtySize};
use crate::driver::ProviderAccess;
use crate::driver_sandbox::{RunSandbox, local_sandbox_with_events};
use crate::driver_sandbox::RunSandbox;
use crate::provider_sandbox;
/// Reconnect to a sandbox from a saved record.
/// Reconnect to a run's sandbox from its saved record.
///
/// `access` carries the provider settings and vault credentials the record's
/// provider needs; the process environment is never consulted.
pub async fn reconnect(record: &RunSandboxInstance, access: &ProviderAccess) -> Result<RunSandbox> {
reconnect_for_run(record, access, None).await
}
/// provider needs; the process environment is never consulted. `run_id`
/// narrows the ownership scope to the run when known, and the driver reports
/// the sandbox's lifecycle from here on through `events`.
pub async fn reconnect_for_run(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
) -> Result<RunSandbox> {
reconnect_for_run_with_events(record, access, run_id, None).await
}
pub async fn reconnect_for_run_with_events(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
events: Option<EventContext>,
) -> Result<RunSandbox> {
reconnect_driver_for_run(record, access, run_id, events).await
}
/// Reconnects as the driver-backed sandbox type, for callers that need a
/// driver facet fabro's [`Sandbox`](crate::Sandbox) trait does not carry
/// (VNC, signed previews, leased SSH).
pub async fn reconnect_driver_for_run(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
events: Option<EventContext>,
) -> Result<RunSandbox> {
let runtime = &record.runtime;
// A local sandbox is its working directory: rebuilding the handle over
// that directory is the reconnect. The per-process Host registry holds
// no state worth attaching to.
let sandbox = if record.provider.bundled() == Some(BundledProvider::Local) {
local_sandbox_with_events(PathBuf::from(&runtime.working_directory), events)
.await
.context("Failed to reconnect local sandbox")?
} else {
let repo_cloned = runtime.repo_cloned.with_context(|| {
format!(
"{} run sandbox missing repo_cloned metadata",
record.provider
)
})?;
provider_sandbox::attach_provider_sandbox(
record.provider.clone(),
access,
&runtime.id,
repo_cloned,
runtime.working_directory.clone(),
runtime.clone_origin_url.clone(),
run_id,
events,
)
.await
.with_context(|| format!("Failed to reconnect {} sandbox", record.provider))?
};
Ok(sandbox)
provider_sandbox::attach_provider_sandbox(
record.provider.clone(),
access,
&runtime.id,
// A record without the flag was written for a sandbox fabro never
// cloned into.
runtime.repo_cloned.unwrap_or(false),
runtime.working_directory.clone(),
runtime.clone_origin_url.clone(),
run_id,
events,
)
.await
.with_context(|| format!("Failed to reconnect {} sandbox", record.provider))
}
/// Opens an interactive shell in a run's sandbox over the driver's Pty
/// facet, reconnecting from the run record first. The session is the
/// driver's own; it is closed by the caller.
pub async fn open_terminal_for_run(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
size: PtySize,
) -> crate::Result<Box<dyn PtySession>> {
let sandbox = reconnect_for_run(record, access, run_id, None)
.await
.map_err(|err| crate::Error::context_anyhow("Failed to reconnect sandbox", err))?;
sandbox.activate().await?;
sandbox.open_terminal(size).await
}

10
lib/components/fabro-sandbox/src/redact.rs
View file

@ -1,17 +1,9 @@
//! Fabro's secret scanner on the text seams pebble and the sandbox expose.
//! Fabro's secret scanner on the text seams pebble exposes.
use std::borrow::Cow;
use pebble_coding_agent::extensions::Redactor;
/// Strips a specific authenticated URL out of `text`, when one is known.
pub fn redact_auth_url(text: &str, auth_url: Option<&fabro_redact::DisplaySafeUrl>) -> String {
let Some(auth_url) = auth_url else {
return text.to_string();
};
text.replace(&auth_url.raw_string(), &auth_url.redacted_string())
}
/// Fabro's secret scanner as pebble's [`Redactor`].
///
/// Pebble calls it where text a process or the operating system wrote leaves

1520
lib/components/fabro-sandbox/src/sandbox.rs
View file

File diff suppressed because it is too large Load diff

368
lib/components/fabro-sandbox/src/sandbox_spec.rs
View file

@ -4,201 +4,138 @@ use std::sync::Arc;
use anyhow::Context as _;
use fabro_github::GitHubCredentials;
use fabro_types::{RunId, RunSandboxInstance, RunSandboxRuntime, SandboxProviderKind};
use sandbox_driver::EventContext;
use sandbox_driver::{EventContext, SandboxSource, SandboxSpec as DriverSpec};
use crate::driver::ProviderAccess;
use crate::driver_sandbox::{LayoutSource, RunSandbox, local_sandbox_with_events};
use crate::options::SandboxOptions;
use crate::driver_sandbox::{LayoutSource, RunSandbox};
use crate::environment::CloneRequest;
use crate::{clone_source, provider_sandbox};
/// Options for sandbox initialization and construction.
/// A run's sandbox on any provider fabro can name: a bundled kind in
/// process or a sandbox-driver plugin. What the environment asked for, and
/// how the repository is cloned into it.
#[derive(Clone, Debug)]
pub enum SandboxSpec {
Local {
working_directory: PathBuf,
},
/// A sandbox on any provider fabro can name: a bundled kind in process
/// or a sandbox-driver plugin.
Provider(Box<ProviderSandboxSpec>),
}
/// A run's sandbox on a provider: what the environment asked for and how
/// the repository is cloned into it.
#[derive(Clone, Debug)]
pub struct ProviderSandboxSpec {
pub kind: SandboxProviderKind,
pub struct SandboxSpec {
pub kind: SandboxProviderKind,
/// The provider settings and vault credentials the kind needs.
pub access: ProviderAccess,
pub options: SandboxOptions,
pub github_app: Option<GitHubCredentials>,
pub run_id: Option<RunId>,
pub clone_origin_url: Option<String>,
pub clone_branch: Option<String>,
pub clone_tag: Option<String>,
pub clone_commit_sha: Option<String>,
pub access: ProviderAccess,
/// The environment's request, as the driver spec every provider
/// starts from.
pub spec: DriverSpec,
pub clone: CloneRequest,
pub github_app: Option<GitHubCredentials>,
pub run_id: Option<RunId>,
}
impl SandboxSpec {
pub fn provider(&self) -> SandboxProviderKind {
match self {
Self::Local { .. } => SandboxProviderKind::LOCAL,
Self::Provider(spec) => spec.kind.clone(),
/// A sandbox on this host at `working_directory`, the fabro `local`
/// kind. The directory is designated: the sandbox uses it in place,
/// never removes it, and clones nothing into it. The Host provider has
/// no image, labels, or lifecycle timers, so the spec names only the
/// directory.
#[must_use]
pub fn local(working_directory: impl Into<PathBuf>, access: ProviderAccess) -> Self {
Self {
kind: SandboxProviderKind::LOCAL,
access,
spec: DriverSpec::new(SandboxSource::HostDirectory)
.working_directory(working_directory.into().display().to_string()),
clone: CloneRequest::none(),
github_app: None,
run_id: None,
}
}
pub fn provider(&self) -> SandboxProviderKind {
self.kind.clone()
}
pub fn provider_name(&self) -> String {
self.provider().to_string()
self.kind.to_string()
}
/// The directory the spec designates on the provider, when it names one.
#[must_use]
pub fn working_directory(&self) -> Option<&str> {
self.spec.working_directory.as_deref()
}
/// The image the run record names for this sandbox: the environment's,
/// or the provider's default when the environment names none. A local
/// sandbox has no image.
/// or the provider's default when the environment names none.
pub fn image(&self) -> Option<String> {
match self {
Self::Local { .. } => None,
Self::Provider(spec) => provider_sandbox::recorded_image(&spec.kind, &spec.options),
}
provider_sandbox::recorded_image(&self.kind, &self.spec)
}
/// Build initialized sandbox metadata for persistence.
pub fn to_run_sandbox_instance(
&self,
sandbox: &RunSandbox,
run_id: RunId,
) -> RunSandboxInstance {
pub fn to_run_sandbox_instance(&self, sandbox: &RunSandbox) -> RunSandboxInstance {
let working_directory = sandbox.working_directory().to_string();
let id = {
let info = sandbox.sandbox_info();
if info.is_empty() {
format!("local:{run_id}")
} else {
info
}
};
match self {
Self::Provider(spec) => {
let ProviderSandboxSpec {
kind,
options,
clone_origin_url,
clone_branch,
..
} = spec.as_ref();
let repo_cloned = clone_source::repo_cloned_for_record(
options.skip_clone,
let id = sandbox.sandbox_info();
let clone_origin_url = &self.clone.origin_url;
let repo_cloned =
clone_source::repo_cloned_for_record(self.clone.skip, clone_origin_url.as_deref());
// A fixed layout is known before the sandbox exists; a
// provider-chosen one only from the sandbox.
let layout = match provider_sandbox::layout_source(&self.kind) {
LayoutSource::Fixed(fixed) => {
let repo = runtime_layout_metadata(
repo_cloned,
clone_origin_url.as_deref(),
&fixed.workspace_root,
&fixed.repos_root,
);
// A fixed layout is known before the sandbox exists; a
// provider-chosen one only from the sandbox.
let layout = match provider_sandbox::layout_source(kind) {
LayoutSource::Fixed(fixed) => {
let repo = runtime_layout_metadata(
repo_cloned,
clone_origin_url.as_deref(),
&fixed.workspace_root,
&fixed.repos_root,
);
Some(crate::SandboxWorkspaceLayout {
workspace_root: fixed.workspace_root,
repos_root: fixed.repos_root,
primary_repo_path: repo
.as_ref()
.map(|layout| layout.primary_repo_path.clone()),
primary_repo_link: repo
.as_ref()
.map(|layout| layout.primary_repo_link.clone()),
})
}
LayoutSource::ProviderWorkingDirectory => sandbox.workspace_layout(),
};
RunSandboxInstance {
provider: kind.clone(),
image: provider_sandbox::recorded_image(kind, options),
snapshot: sandbox.snapshot_info(),
runtime: RunSandboxRuntime {
id,
working_directory,
repo_cloned,
clone_origin_url: clone_source::clean_clone_origin_for_record(
clone_origin_url.as_deref(),
),
clone_branch: clone_branch.clone(),
workspace_root: layout.as_ref().map(|layout| layout.workspace_root.clone()),
repos_root: layout.as_ref().map(|layout| layout.repos_root.clone()),
primary_repo_path: layout
.as_ref()
.and_then(|layout| layout.primary_repo_path.clone()),
primary_repo_link: layout
.as_ref()
.and_then(|layout| layout.primary_repo_link.clone()),
},
}
Some(crate::SandboxWorkspaceLayout {
workspace_root: fixed.workspace_root,
repos_root: fixed.repos_root,
primary_repo_path: repo.as_ref().map(|layout| layout.primary_repo_path.clone()),
primary_repo_link: repo.as_ref().map(|layout| layout.primary_repo_link.clone()),
})
}
Self::Local { .. } => RunSandboxInstance {
provider: self.provider(),
image: None,
snapshot: None,
runtime: RunSandboxRuntime {
id,
working_directory,
repo_cloned: None,
clone_origin_url: None,
clone_branch: None,
workspace_root: None,
repos_root: None,
primary_repo_path: None,
primary_repo_link: None,
},
LayoutSource::ProviderWorkingDirectory => sandbox.workspace_layout(),
};
RunSandboxInstance {
provider: self.kind.clone(),
image: self.image(),
snapshot: sandbox.snapshot_info(),
runtime: RunSandboxRuntime {
id,
working_directory,
repo_cloned,
clone_origin_url: clone_source::clean_clone_origin_for_record(
clone_origin_url.as_deref(),
),
clone_branch: self.clone.branch.clone(),
workspace_root: layout.as_ref().map(|layout| layout.workspace_root.clone()),
repos_root: layout.as_ref().map(|layout| layout.repos_root.clone()),
primary_repo_path: layout
.as_ref()
.and_then(|layout| layout.primary_repo_path.clone()),
primary_repo_link: layout
.as_ref()
.and_then(|layout| layout.primary_repo_link.clone()),
},
}
}
/// Builds the sandbox. The driver reports its lifecycle through
/// `events`: the local sandbox's from creation here, a provider
/// sandbox's from `initialize` on.
/// Builds the sandbox; `initialize` creates it on the provider. The
/// driver reports its lifecycle through `events` from then on.
pub async fn build(
&self,
events: Option<EventContext>,
) -> Result<Arc<RunSandbox>, anyhow::Error> {
match self {
Self::Local { working_directory } => {
let sandbox = local_sandbox_with_events(working_directory.clone(), events)
.await
.context("Failed to create local sandbox")?;
Ok(Arc::new(sandbox))
}
Self::Provider(spec) => {
let ProviderSandboxSpec {
kind,
access,
options,
github_app,
run_id,
clone_origin_url,
clone_branch,
clone_tag,
clone_commit_sha,
} = spec.as_ref();
let mut sandbox = provider_sandbox::provider_sandbox(
kind.clone(),
access,
options.clone(),
github_app.as_ref(),
*run_id,
clone_origin_url.clone(),
clone_branch.clone(),
clone_tag.clone(),
clone_commit_sha.clone(),
)
.await
.with_context(|| format!("Failed to create {kind} sandbox"))?;
if let Some(events) = events {
sandbox.set_events(events);
}
Ok(Arc::new(sandbox))
}
let mut sandbox = provider_sandbox::provider_sandbox(
self.kind.clone(),
&self.access,
self.spec.clone(),
&self.clone,
self.github_app.as_ref(),
self.run_id,
)
.await
.with_context(|| format!("Failed to create {} sandbox", self.kind))?;
if let Some(events) = events {
sandbox.set_events(events);
}
Ok(Arc::new(sandbox))
}
}
@ -216,14 +153,24 @@ fn runtime_layout_metadata(
#[cfg(test)]
mod tests {
use fabro_types::RunId;
use sandbox_driver_testing::ScriptedSandbox;
use super::*;
fn sandbox_at(working_dir: &str) -> RunSandbox {
fn docker_spec(clone: CloneRequest) -> SandboxSpec {
SandboxSpec {
kind: SandboxProviderKind::DOCKER,
access: ProviderAccess::default(),
spec: DriverSpec::new(SandboxSource::HostDirectory),
clone,
github_app: None,
run_id: None,
}
}
fn sandbox_at(kind: SandboxProviderKind, working_dir: &str) -> RunSandbox {
RunSandbox::new(
SandboxProviderKind::DOCKER,
kind,
Arc::new(ScriptedSandbox::with_id_and_working_dir(
"scripted-1",
working_dir,
@ -233,21 +180,14 @@ mod tests {
#[test]
fn docker_run_sandbox_persists_layout_metadata_for_cloned_repo() {
let spec = SandboxSpec::Provider(Box::new(ProviderSandboxSpec {
kind: SandboxProviderKind::DOCKER,
access: ProviderAccess::default(),
options: SandboxOptions::default(),
github_app: None,
run_id: None,
clone_origin_url: Some("git@github.com:brynary/rack-test.git".to_string()),
clone_branch: Some("main".to_string()),
clone_tag: None,
clone_commit_sha: None,
}));
let sandbox = sandbox_at("/workspace/rack-test");
let spec = docker_spec(CloneRequest {
origin_url: Some("git@github.com:brynary/rack-test.git".to_string()),
branch: Some("main".to_string()),
..CloneRequest::default()
});
let sandbox = sandbox_at(SandboxProviderKind::DOCKER, "/workspace/rack-test");
let run_id: RunId = "01HY0000000000000000000000".parse().unwrap();
let record = spec.to_run_sandbox_instance(&sandbox, run_id);
let record = spec.to_run_sandbox_instance(&sandbox);
let runtime = record.runtime;
assert_eq!(runtime.working_directory, "/workspace/rack-test");
@ -272,17 +212,12 @@ mod tests {
#[tokio::test]
async fn invalid_exact_checkout_spec_fails_before_provider_connection() {
let spec = SandboxSpec::Provider(Box::new(ProviderSandboxSpec {
kind: SandboxProviderKind::DOCKER,
access: ProviderAccess::default(),
options: SandboxOptions::default(),
github_app: None,
run_id: None,
clone_origin_url: Some("https://github.com/acme/widgets".to_string()),
clone_branch: Some("main".to_string()),
clone_tag: None,
clone_commit_sha: Some("not-a-sha".to_string()),
}));
let spec = docker_spec(CloneRequest {
origin_url: Some("https://github.com/acme/widgets".to_string()),
branch: Some("main".to_string()),
commit_sha: Some("not-a-sha".to_string()),
..CloneRequest::default()
});
let error = spec
.build(None)
@ -300,24 +235,13 @@ mod tests {
#[test]
fn docker_run_sandbox_omits_primary_repo_metadata_for_empty_workspace() {
let spec = SandboxSpec::Provider(Box::new(ProviderSandboxSpec {
kind: SandboxProviderKind::DOCKER,
access: ProviderAccess::default(),
options: SandboxOptions {
skip_clone: true,
..SandboxOptions::default()
},
github_app: None,
run_id: None,
clone_origin_url: Some("https://gitlab.com/acme/widgets".to_string()),
clone_branch: None,
clone_tag: None,
clone_commit_sha: None,
}));
let sandbox = sandbox_at("/workspace");
let spec = docker_spec(CloneRequest {
origin_url: Some("https://gitlab.com/acme/widgets".to_string()),
..CloneRequest::none()
});
let sandbox = sandbox_at(SandboxProviderKind::DOCKER, "/workspace");
let run_id: RunId = "01HY0000000000000000000000".parse().unwrap();
let record = spec.to_run_sandbox_instance(&sandbox, run_id);
let record = spec.to_run_sandbox_instance(&sandbox);
let runtime = record.runtime;
assert_eq!(runtime.working_directory, "/workspace");
@ -327,4 +251,34 @@ mod tests {
assert!(runtime.primary_repo_path.is_none());
assert!(runtime.primary_repo_link.is_none());
}
#[test]
fn local_spec_designates_the_directory_and_clones_nothing() {
let spec = SandboxSpec::local("/home/dev/project", ProviderAccess::default());
assert_eq!(spec.kind, SandboxProviderKind::LOCAL);
assert_eq!(spec.working_directory(), Some("/home/dev/project"));
assert!(spec.clone.skip);
assert_eq!(spec.clone.origin_url, None);
assert_eq!(spec.image(), None);
assert!(matches!(spec.spec.source, SandboxSource::HostDirectory));
let sandbox = sandbox_at(SandboxProviderKind::LOCAL, "/home/dev/project");
let record = spec.to_run_sandbox_instance(&sandbox);
assert_eq!(record.provider, SandboxProviderKind::LOCAL);
assert_eq!(record.image, None);
assert_eq!(record.snapshot, None);
assert_eq!(record.runtime.id, "scripted-1");
assert_eq!(record.runtime.working_directory, "/home/dev/project");
assert_eq!(record.runtime.repo_cloned, Some(false));
assert_eq!(record.runtime.clone_origin_url, None);
assert_eq!(record.runtime.clone_branch, None);
assert_eq!(
record.runtime.workspace_root.as_deref(),
Some("/home/dev/project")
);
assert!(record.runtime.primary_repo_path.is_none());
assert!(record.runtime.primary_repo_link.is_none());
}
}

92
lib/components/fabro-sandbox/src/terminal.rs
View file

@ -1,92 +0,0 @@
use async_trait::async_trait;
use fabro_types::{BundledProvider, RunId, RunSandboxInstance};
use crate::driver::ProviderAccess;
use crate::reconnect;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct TerminalSize {
pub cols: u16,
pub rows: u16,
}
impl Default for TerminalSize {
fn default() -> Self {
Self {
cols: 120,
rows: 32,
}
}
}
#[async_trait]
pub trait TerminalSession: Send + Sync {
async fn write_input(&self, bytes: &[u8]) -> crate::Result<()>;
async fn read_output(&self) -> crate::Result<Option<Vec<u8>>>;
async fn resize(&self, size: TerminalSize) -> crate::Result<()>;
async fn close(&self) -> crate::Result<()>;
}
/// A terminal over the driver's Pty facet.
pub struct DriverTerminalSession {
session: Box<dyn sandbox_driver::PtySession>,
}
impl DriverTerminalSession {
#[must_use]
pub fn new(session: Box<dyn sandbox_driver::PtySession>) -> Self {
Self { session }
}
}
#[async_trait]
impl TerminalSession for DriverTerminalSession {
async fn write_input(&self, bytes: &[u8]) -> crate::Result<()> {
self.session
.write_input(bytes)
.await
.map_err(|err| crate::Error::context("Failed to write terminal input", err))
}
async fn read_output(&self) -> crate::Result<Option<Vec<u8>>> {
self.session
.read_output()
.await
.map_err(|err| crate::Error::context("Failed to read terminal output", err))
}
async fn resize(&self, size: TerminalSize) -> crate::Result<()> {
self.session
.resize(sandbox_driver::PtySize {
rows: size.rows,
cols: size.cols,
})
.await
.map_err(|err| crate::Error::context("Failed to resize terminal", err))
}
async fn close(&self) -> crate::Result<()> {
self.session
.close()
.await
.map_err(|err| crate::Error::context("Failed to close terminal", err))
}
}
pub async fn open_terminal_for_run(
record: &RunSandboxInstance,
access: &ProviderAccess,
run_id: Option<RunId>,
size: TerminalSize,
) -> crate::Result<Box<dyn TerminalSession>> {
if record.provider.bundled() == Some(BundledProvider::Local) {
return Err(crate::Error::message(
"Local sandboxes do not support embedded terminals",
));
}
let sandbox = reconnect::reconnect_driver_for_run(record, access, run_id, None)
.await
.map_err(|err| crate::Error::context_anyhow("Failed to reconnect sandbox", err))?;
sandbox.activate().await?;
Ok(Box::new(sandbox.open_terminal(size).await?))
}

311
lib/components/fabro-sandbox/src/test_support.rs
View file

@ -1,23 +1,65 @@
//! Test doubles for fabro's sandbox layer.
//!
//! [`MockSandbox`] is a configuration and a recorder over the sandbox
//! driver's scripted double: a test writes down the files, the command
//! answer, and the failures it wants, takes a [`RunSandbox`] from it, and
//! reads back what the code under test ran or wrote. Nothing here fakes
//! [`MockSandbox`] is a configuration over the sandbox driver's scripted
//! double: a test writes down the files, the command answer, and the
//! failures it wants, and takes a [`RunSandbox`] from it. What the code
//! under test ran or wrote is read back from the driver double itself,
//! through [`MockSandbox::driver`]; the few accessors here convert what a
//! spec records into the shape fabro's tests assert on. Nothing here fakes
//! fabro's own logic; every call goes through the real `RunSandbox` and
//! fabro's exec policy, down to the scripted driver.
use std::collections::HashMap;
use std::path::Path;
use std::sync::{Arc, OnceLock};
use std::time::Duration;
use fabro_types::{CommandTermination, SandboxProviderKind};
use sandbox_driver::{GrepMatch, PlatformInfo, SandboxState, Termination, WalkedFile};
pub use sandbox_driver_testing::{ScriptedExec, ScriptedSandbox, ScriptedStdioProcess};
use fabro_types::SandboxProviderKind;
use sandbox_driver::{
ExecResult, GrepMatch, PlatformInfo, SandboxState, StderrTail, Termination, WalkedFile,
};
use sandbox_driver_host::HostProvider;
pub use sandbox_driver_testing::{
ScriptedExec, ScriptedProvider, ScriptedSandbox, ScriptedStdioProcess,
};
use tokio::io::DuplexStream;
use crate::driver::ConnectedProvider;
use crate::driver_sandbox::RunSandbox;
use crate::sandbox::{ExecResult, SandboxFile, StderrCollector};
use crate::managed_labels::{MANAGED_LABEL, MANAGED_LABEL_VALUE};
use crate::sandbox::SandboxFile;
/// The id a run record carries for a local sandbox at `working_directory`,
/// as the Host provider derives it from the canonical path. A record a test
/// writes by hand reconnects the way one fabro wrote would. The directory
/// must exist.
pub async fn local_sandbox_id(working_directory: &Path) -> String {
HostProvider::directory_id(working_directory)
.await
.unwrap_or_else(|| {
panic!(
"no local sandbox id for {}: the directory must exist",
working_directory.display()
)
})
.to_string()
}
/// A driver [`ExecResult`] with the given streams, for scripting a mock
/// sandbox's answers.
#[must_use]
pub fn exec_result(
stdout: &str,
stderr: &str,
exit_code: Option<i32>,
termination: Termination,
duration_ms: u64,
) -> ExecResult {
let mut result = ExecResult::new(termination, exit_code, Duration::from_millis(duration_ms));
result.stdout = stdout.as_bytes().to_vec();
result.stderr = stderr.as_bytes().to_vec();
result
}
// --- MockSandbox ---
@ -71,12 +113,11 @@ impl Default for MockSandbox {
fn default() -> Self {
Self {
files: HashMap::new(),
exec_result: ExecResult {
stdout: "mock output".into(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 10,
exec_result: {
let mut result =
ExecResult::new(Termination::Exited, Some(0), Duration::from_millis(10));
result.stdout = b"mock output".to_vec();
result
},
exec_error: None,
working_dir: "/work",
@ -145,25 +186,16 @@ impl MockSandbox {
) -> &Self {
self.driver().scripted_exec().respond_with(move |spec| {
let command = spec.args.last().map(String::as_str).unwrap_or_default();
responder(command).map(|result| driver_result(&result))
responder(command)
});
self
}
/// Queues the result for the next command, ahead of `exec_result`.
/// Results answer in the order they were pushed.
pub fn push_exec_result(&self, result: &ExecResult) -> &Self {
self.driver()
.scripted_exec()
.push_result(driver_result(result));
self
}
fn built(&self) -> &Built {
self.built.get_or_init(|| {
let driver = Arc::new(self.build_driver());
// An isolated provider: explicit environment passes as the
// caller composed it, as it does for Docker and Daytona runs.
// The kind is nominal for exec: the explicit environment reaches
// the scripted driver as the caller composed it on every provider.
let run = RunSandbox::new_with_platform(
SandboxProviderKind::DOCKER,
Arc::clone(&driver) as Arc<dyn sandbox_driver::Sandbox>,
@ -197,7 +229,7 @@ impl MockSandbox {
let exec = driver.scripted_exec();
match &self.exec_error {
Some(message) => exec.fail_by_default(message.clone()),
None => exec.set_default(driver_result(&self.exec_result)),
None => exec.set_default(self.exec_result.clone()),
};
exec.set_streams_separated(self.streams_separated);
if let Some(message) = &self.stdio_process_error {
@ -240,17 +272,12 @@ impl MockSandbox {
.unwrap_or_default()
}
/// The Bash source of every command run so far, in order.
pub fn captured_commands(&self) -> Vec<String> {
self.recorded()
.iter()
.map(|spec| spec.args.last().cloned().unwrap_or_default())
.collect()
}
/// The last command's Bash source.
/// The last command's Bash source. Every command, in order, is
/// `driver().scripted_exec().commands()`.
pub fn captured_command(&self) -> Option<String> {
self.captured_commands().pop()
self.recorded()
.last()
.and_then(|spec| spec.args.last().cloned())
}
/// The last command's timeout in milliseconds.
@ -270,25 +297,9 @@ impl MockSandbox {
.collect()
}
/// Whether each command was given the run's cancellation to stop on,
/// in order.
pub fn captured_term_stops(&self) -> Vec<bool> {
self.built
.get()
.map(|built| built.driver.scripted_exec().term_stops())
.unwrap_or_default()
}
/// The working directory of every command, in order.
pub fn captured_working_dirs(&self) -> Vec<Option<String>> {
self.recorded()
.iter()
.map(|spec| spec.working_dir.clone())
.collect()
}
/// The explicit variables of the last command as the caller passed them.
/// The exec policy's own `BASH_ENV` blank is not the caller's.
/// The driver's Bash helper records its own `BASH_ENV` blank on the
/// spec; that is not the caller's.
pub fn captured_env_vars(&self) -> Option<HashMap<String, String>> {
self.recorded().last().map(|spec| {
spec.env
@ -299,13 +310,6 @@ impl MockSandbox {
})
}
/// The bytes the last streaming command was fed on standard input.
pub fn captured_stdin(&self) -> Option<Vec<u8>> {
self.built
.get()
.and_then(|built| built.driver.scripted_exec().captured_stdin().pop())
}
/// Every file written so far as `(path, content)`, in order.
pub fn written_files(&self) -> Vec<(String, String)> {
self.built
@ -321,65 +325,6 @@ impl MockSandbox {
})
.unwrap_or_default()
}
/// Every file deleted so far by absolute path, in order.
pub fn deleted_files(&self) -> Vec<String> {
self.built
.get()
.map(|built| built.driver.memory_fs().deletes())
.unwrap_or_default()
}
/// How many times the code under test asked whether a path exists.
pub fn exists_calls(&self) -> usize {
self.built
.get()
.map_or(0, |built| built.driver.memory_fs().exists_calls())
}
pub fn start_count(&self) -> u32 {
self.built
.get()
.map_or(0, |built| built.driver.start_count())
}
pub fn stop_count(&self) -> u32 {
self.built
.get()
.map_or(0, |built| built.driver.stop_count())
}
pub fn delete_count(&self) -> u32 {
self.built
.get()
.map_or(0, |built| built.driver.delete_count())
}
/// How many walks the code under test ran.
pub fn walk_files_was_called(&self) -> bool {
self.built
.get()
.is_some_and(|built| built.driver.scripted_search().walk_calls() > 0)
}
}
/// The driver result fabro's exec policy reads back as `result`.
fn driver_result(result: &ExecResult) -> sandbox_driver::ExecResult {
let termination = match result.termination {
CommandTermination::TimedOut => Termination::TimedOut,
CommandTermination::Cancelled => Termination::Cancelled,
// `CommandTermination` is non-exhaustive; `Exited` and anything newer
// read back as a plain exit.
_ => Termination::Exited,
};
let mut driver = sandbox_driver::ExecResult::new(
termination,
result.exit_code,
Duration::from_millis(result.duration_ms),
);
driver.stdout = result.stdout.clone().into_bytes();
driver.stderr = result.stderr.clone().into_bytes();
driver
}
// --- MockStdioProcess ---
@ -387,21 +332,17 @@ fn driver_result(result: &ExecResult) -> sandbox_driver::ExecResult {
/// A stdio process a test drives, over the driver's scripted process.
///
/// The driver closure receives the process's end of standard input, its
/// end of standard output, and fabro's stderr collector for the process.
/// end of standard output, and the rolling stderr tail the process reports.
pub struct MockStdioProcess {
inner: std::sync::Mutex<Option<ScriptedStdioProcess>>,
}
impl MockStdioProcess {
pub fn new(
driver: impl FnOnce(DuplexStream, DuplexStream, StderrCollector) + Send + 'static,
driver: impl FnOnce(DuplexStream, DuplexStream, StderrTail) + Send + 'static,
) -> Self {
Self {
inner: std::sync::Mutex::new(Some(ScriptedStdioProcess::new(
move |stdin, stdout, tail| {
driver(stdin, stdout, StderrCollector::from_driver_tail(tail));
},
))),
inner: std::sync::Mutex::new(Some(ScriptedStdioProcess::new(driver))),
}
}
@ -426,98 +367,32 @@ impl MockStdioProcess {
}
}
// --- FakeSandboxProvider ---
// --- Inventory doubles ---
pub use fake_provider::{FakeGet, FakeList, FakeSandboxProvider, fake_registry, fake_sandbox_info};
/// A running scripted sandbox carrying fabro's managed label, so an owned
/// inventory lists it and attaches to it.
#[must_use]
pub fn managed_scripted_sandbox(id: &str) -> Arc<ScriptedSandbox> {
Arc::new(
ScriptedSandbox::with_id_and_working_dir(id, "/work")
.state(SandboxState::Running)
.label(MANAGED_LABEL, MANAGED_LABEL_VALUE),
)
}
mod fake_provider {
use std::collections::BTreeMap;
use std::sync::Arc;
use async_trait::async_trait;
use fabro_types::{
SandboxInfo, SandboxNetwork, SandboxProviderKind, SandboxResources, SandboxState,
SandboxTimestamps,
};
use crate::provider::{SandboxProvider, SandboxProviderRegistry};
#[derive(Clone)]
pub enum FakeList {
Ok(Vec<SandboxInfo>),
Err(&'static str),
/// A connected inventory provider of `kind` holding `sandboxes`, over the
/// driver's scripted provider.
#[must_use]
pub fn scripted_inventory_provider(
kind: SandboxProviderKind,
sandboxes: Vec<Arc<ScriptedSandbox>>,
) -> ConnectedProvider {
let provider = ScriptedProvider::new(kind.as_str());
for sandbox in sandboxes {
provider.register(sandbox);
}
#[derive(Clone)]
pub enum FakeGet {
Found(Box<SandboxInfo>),
Missing,
Err(&'static str),
}
pub struct FakeSandboxProvider {
kind: SandboxProviderKind,
list: FakeList,
get: FakeGet,
}
impl FakeSandboxProvider {
pub fn new(kind: SandboxProviderKind, list: FakeList, get: FakeGet) -> Self {
Self { kind, list, get }
}
}
#[async_trait]
impl SandboxProvider for FakeSandboxProvider {
fn kind(&self) -> SandboxProviderKind {
self.kind.clone()
}
async fn list(&self) -> crate::Result<Vec<SandboxInfo>> {
match &self.list {
FakeList::Ok(sandboxes) => Ok(sandboxes.clone()),
FakeList::Err(message) => Err(crate::Error::message(*message)),
}
}
async fn get(&self, _id: &str) -> crate::Result<Option<SandboxInfo>> {
match &self.get {
FakeGet::Found(sandbox) => Ok(Some((**sandbox).clone())),
FakeGet::Missing => Ok(None),
FakeGet::Err(message) => Err(crate::Error::message(*message)),
}
}
async fn delete(&self, _id: &str) -> crate::Result<()> {
Ok(())
}
}
pub fn fake_registry(providers: Vec<FakeSandboxProvider>) -> SandboxProviderRegistry {
SandboxProviderRegistry::new(
providers
.into_iter()
.map(|provider| Arc::new(provider) as Arc<dyn SandboxProvider>)
.collect(),
)
}
pub fn fake_sandbox_info(provider: SandboxProviderKind, id: &str) -> SandboxInfo {
SandboxInfo {
provider,
id: id.to_string(),
display_name: None,
state: SandboxState::Running,
native_state: None,
image: None,
snapshot: None,
region: None,
web_url: None,
working_directory: None,
resources: SandboxResources::default(),
network: SandboxNetwork::unknown(),
labels: BTreeMap::new(),
timestamps: SandboxTimestamps::default(),
}
ConnectedProvider {
kind,
provider: Arc::new(provider),
}
}

173
lib/components/fabro-sandbox/tests/daytona_streaming_live.rs
View file

@ -4,18 +4,19 @@ mod daytona_streaming_live {
use anyhow::{Context, Result, ensure};
use fabro_sandbox::{
CommandOutputCallback, DaytonaCredentials, ExecStreamingResult, ProviderAccess, RunSandbox,
SandboxOptions, SandboxProviderKind, provider_sandbox,
CloneRequest, DaytonaCredentials, ExecControls, ExecSpec, ExecStreamingResult, OutputSink,
OutputStream, ProviderAccess, RunSandbox, SandboxProviderKind, Termination,
provider_sandbox,
};
use fabro_static::EnvVars;
use fabro_types::{CommandOutputStream, CommandTermination};
use sandbox_driver::{SandboxSource, SandboxSpec};
use tokio::sync::Mutex;
use tokio::time::{Instant, sleep};
use tokio_util::sync::CancellationToken;
#[derive(Debug, Clone)]
struct CapturedChunk {
stream: CommandOutputStream,
stream: OutputStream,
text: String,
}
@ -31,14 +32,8 @@ mod daytona_streaming_live {
provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_credentials()?),
SandboxOptions {
skip_clone: true,
..Default::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::none(),
None,
None,
)
@ -48,7 +43,7 @@ mod daytona_streaming_live {
sandbox.initialize().await?;
let smoke_result = run_smoke(Arc::clone(&sandbox)).await;
let cleanup_result = sandbox.cleanup().await.context("clean up Daytona sandbox");
let cleanup_result = sandbox.delete().await.context("clean up Daytona sandbox");
smoke_result?;
cleanup_result?;
@ -71,14 +66,8 @@ mod daytona_streaming_live {
let sandbox = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_credentials()?),
SandboxOptions {
skip_clone: true,
..Default::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::none(),
None,
None,
)
@ -100,7 +89,7 @@ mod daytona_streaming_live {
&format!("exec_command should run Bash-only syntax: {non_streaming:?}"),
)?;
ensure_contains(
&non_streaming.stdout,
&non_streaming.stdout_lossy(),
"two",
"exec_command should report the Bash-only result",
)?;
@ -112,7 +101,7 @@ mod daytona_streaming_live {
&format!("exec_command_streaming should run Bash-only syntax: {streaming:?}"),
)?;
ensure_contains(
&streaming.result.stdout,
&streaming.result.stdout_lossy(),
"two",
"exec_command_streaming should report the Bash-only result",
)?;
@ -133,7 +122,7 @@ mod daytona_streaming_live {
)?;
ensure_eq(
&stdin_result.result.stdout,
&stdin.to_string(),
&stdin.as_bytes().to_vec(),
"exec_command_streaming should preserve exact stdin bytes",
)?;
let stdin_cleanup = sandbox
@ -147,7 +136,7 @@ mod daytona_streaming_live {
)
.await?;
ensure!(
stdin_cleanup.is_success(),
stdin_cleanup.success(),
"Daytona stdin data must stay inert and its temporary file must be deleted: {stdin_cleanup:?}"
);
@ -155,7 +144,7 @@ mod daytona_streaming_live {
}
.await;
let cleanup_result = sandbox.cleanup().await.context("clean up Daytona sandbox");
let cleanup_result = sandbox.delete().await.context("clean up Daytona sandbox");
checks?;
cleanup_result?;
@ -177,20 +166,11 @@ mod daytona_streaming_live {
let sandbox = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_credentials()?),
SandboxOptions {
skip_clone: true,
labels: std::collections::BTreeMap::from([(
"team".to_string(),
"platform".to_string(),
)]),
..Default::default()
},
SandboxSpec::new(SandboxSource::HostDirectory)
.label("team".to_string(), "platform".to_string()),
&CloneRequest::none(),
None,
Some(run_id),
None,
None,
None,
None,
)
.await?;
@ -202,7 +182,7 @@ mod daytona_streaming_live {
.await
.context("describe sandbox")?
.labels;
let cleanup_result = sandbox.cleanup().await.context("clean up Daytona sandbox");
let cleanup_result = sandbox.delete().await.context("clean up Daytona sandbox");
ensure_eq(
&labels.get("sh.fabro.managed").map(String::as_str),
@ -235,16 +215,13 @@ mod daytona_streaming_live {
let sandbox = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_credentials()?),
SandboxOptions {
skip_clone: false,
..Default::default()
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest {
origin_url: Some("https://github.com/brynary/rack-test".to_string()),
..CloneRequest::default()
},
None,
None,
Some("https://github.com/brynary/rack-test".to_string()),
None,
None,
None,
)
.await?;
@ -269,16 +246,16 @@ mod daytona_streaming_live {
None,
)
.await?;
let cleanup_result = sandbox.cleanup().await.context("clean up Daytona sandbox");
let cleanup_result = sandbox.delete().await.context("clean up Daytona sandbox");
ensure!(
result.is_success(),
result.success(),
"layout verification failed: stdout={} stderr={}",
result.stdout,
result.stderr
result.stdout_lossy(),
result.stderr_lossy()
);
ensure_contains(
&result.stdout,
&result.stdout_lossy(),
"true",
"default cwd should be inside the work tree",
)?;
@ -305,14 +282,8 @@ mod daytona_streaming_live {
let sandbox = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_credentials()?),
SandboxOptions {
skip_clone: true,
..Default::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::none(),
None,
None,
)
@ -321,7 +292,7 @@ mod daytona_streaming_live {
sandbox.initialize().await?;
let glob_result = run_glob_checks(&sandbox).await;
let cleanup_result = sandbox.cleanup().await.context("clean up Daytona sandbox");
let cleanup_result = sandbox.delete().await.context("clean up Daytona sandbox");
glob_result?;
cleanup_result?;
@ -343,10 +314,10 @@ mod daytona_streaming_live {
)
.await?;
ensure!(
seed.is_success(),
seed.success(),
"seeding the skills tree failed: stdout={} stderr={}",
seed.stdout,
seed.stderr
seed.stdout_lossy(),
seed.stderr_lossy()
);
// `*/SKILL.md` matches exactly one path segment: only the file one level
@ -383,21 +354,22 @@ mod daytona_streaming_live {
let live_exec = tokio::spawn(async move {
sandbox_for_exec
.exec_command_streaming(fabro_sandbox::ExecStreamingRequest {
timeout_ms: Some(60_000),
cancel_token: Some(cancel_for_exec),
output_callback: Some(callback),
..fabro_sandbox::ExecStreamingRequest::new(
"printf 'live-out\\n'; printf 'live-err\\n' >&2; sleep 30",
)
})
.exec_command_streaming(
ExecSpec::bash("printf 'live-out\\n'; printf 'live-err\\n' >&2; sleep 30")
.timeout(Duration::from_mins(1)),
ExecControls {
term: Some(cancel_for_exec),
sink: Some(callback),
..ExecControls::default()
},
)
.await
});
let saw_live_stdout_and_stderr =
wait_for_chunks(&chunks, Duration::from_secs(20), |chunks| {
contains_chunk(chunks, CommandOutputStream::Stdout, "live-out")
&& contains_chunk(chunks, CommandOutputStream::Stderr, "live-err")
contains_chunk(chunks, OutputStream::Stdout, "live-out")
&& contains_chunk(chunks, OutputStream::Stderr, "live-err")
})
.await;
@ -423,16 +395,16 @@ mod daytona_streaming_live {
);
ensure_eq(
&live_result.result.termination,
&CommandTermination::Cancelled,
&Termination::Cancelled,
"cancelled command should preserve cancellation termination",
)?;
ensure_contains(
&live_result.result.stdout,
&live_result.result.stdout_lossy(),
"live-out",
"cancelled command stdout should preserve partial logs",
)?;
ensure_contains(
&live_result.result.stderr,
&live_result.result.stderr_lossy(),
"live-err",
"cancelled command stderr should preserve partial logs",
)?;
@ -451,25 +423,25 @@ mod daytona_streaming_live {
)?;
ensure_eq(
&nonzero.result.termination,
&CommandTermination::Exited,
&Termination::Exited,
"nonzero command should be represented as a completed process",
)?;
ensure_contains(
&nonzero.result.stdout,
&nonzero.result.stdout_lossy(),
"exit-out",
"nonzero command stdout should be captured",
)?;
ensure_contains(
&nonzero.result.stderr,
&nonzero.result.stderr_lossy(),
"exit-err",
"nonzero command stderr should be captured",
)?;
ensure!(
contains_chunk(&nonzero_chunks, CommandOutputStream::Stdout, "exit-out"),
contains_chunk(&nonzero_chunks, OutputStream::Stdout, "exit-out"),
"nonzero command should stream stdout chunks"
);
ensure!(
contains_chunk(&nonzero_chunks, CommandOutputStream::Stderr, "exit-err"),
contains_chunk(&nonzero_chunks, OutputStream::Stderr, "exit-err"),
"nonzero command should stream stderr chunks"
);
@ -482,16 +454,16 @@ mod daytona_streaming_live {
.await?;
ensure_eq(
&timed_out.result.termination,
&CommandTermination::TimedOut,
&Termination::TimedOut,
"timed-out command should preserve timeout termination",
)?;
ensure_contains(
&timed_out.result.stdout,
&timed_out.result.stdout_lossy(),
"timeout-out",
"timed-out command stdout should preserve partial logs",
)?;
ensure_contains(
&timed_out.result.stderr,
&timed_out.result.stderr_lossy(),
"timeout-err",
"timed-out command stderr should preserve partial logs",
)?;
@ -517,13 +489,15 @@ mod daytona_streaming_live {
) -> Result<(ExecStreamingResult, Vec<CapturedChunk>)> {
let chunks = Arc::new(Mutex::new(Vec::new()));
let callback = capture_callback(Arc::clone(&chunks));
let mut spec = ExecSpec::bash(command).timeout(Duration::from_millis(timeout_ms));
if let Some(stdin) = stdin {
spec = spec.stdin(stdin);
}
let result = sandbox
.exec_command_streaming(fabro_sandbox::ExecStreamingRequest {
timeout_ms: Some(timeout_ms),
cancel_token,
stdin,
output_callback: Some(callback),
..fabro_sandbox::ExecStreamingRequest::new(command)
.exec_command_streaming(spec, ExecControls {
term: cancel_token,
sink: Some(callback),
..ExecControls::default()
})
.await?;
let chunks = chunks.lock().await.clone();
@ -531,7 +505,7 @@ mod daytona_streaming_live {
Ok((result, chunks))
}
fn capture_callback(chunks: Arc<Mutex<Vec<CapturedChunk>>>) -> CommandOutputCallback {
fn capture_callback(chunks: Arc<Mutex<Vec<CapturedChunk>>>) -> OutputSink {
Arc::new(move |stream, bytes| {
let chunks = Arc::clone(&chunks);
Box::pin(async move {
@ -559,16 +533,11 @@ mod daytona_streaming_live {
reason = "live smoke tests take Daytona credentials from the developer's environment"
)]
fn live_credentials() -> Result<DaytonaCredentials> {
Ok(DaytonaCredentials {
api_key: std::env::var(EnvVars::DAYTONA_API_KEY)
.context("DAYTONA_API_KEY must be set")?,
api_url: std::env::var(EnvVars::DAYTONA_API_URL)
.or_else(|_| std::env::var(EnvVars::DAYTONA_SERVER_URL))
.ok(),
organization_id: std::env::var(EnvVars::DAYTONA_ORGANIZATION_ID).ok(),
target: None,
http_client: None,
})
let api_key =
std::env::var(EnvVars::DAYTONA_API_KEY).context("DAYTONA_API_KEY must be set")?;
Ok(DaytonaCredentials::from_api_key(api_key, |name| {
std::env::var(name).ok()
}))
}
fn daytona_access(credentials: DaytonaCredentials) -> ProviderAccess {
@ -597,7 +566,7 @@ mod daytona_streaming_live {
}
}
fn contains_chunk(chunks: &[CapturedChunk], stream: CommandOutputStream, text: &str) -> bool {
fn contains_chunk(chunks: &[CapturedChunk], stream: OutputStream, text: &str) -> bool {
chunks
.iter()
.any(|chunk| chunk.stream == stream && chunk.text.contains(text))

207
lib/components/fabro-sandbox/tests/docker_streaming.rs
View file

@ -1,12 +1,13 @@
//! Docker sandbox behaviour through the sandbox-driver Docker provider.
use std::collections::BTreeMap;
use std::sync::Arc;
use std::time::Duration;
use fabro_sandbox::{
CommandOutputCallback, ExecStreamingRequest, ProviderAccess, SandboxOptions,
SandboxProviderKind, provider_sandbox,
CloneRequest, ExecControls, ExecSpec, OutputSink, ProviderAccess, SandboxProviderKind,
Termination, provider_sandbox,
};
use sandbox_driver::{SandboxSource, SandboxSpec};
use tokio::process::Command;
use tokio::sync::Mutex;
@ -22,7 +23,7 @@ async fn docker_image_available(image: &str) -> bool {
.is_ok_and(|status| status.success())
}
fn capture_bytes(chunks: Arc<Mutex<Vec<u8>>>) -> CommandOutputCallback {
fn capture_bytes(chunks: Arc<Mutex<Vec<u8>>>) -> OutputSink {
Arc::new(move |_stream, bytes| {
let chunks = Arc::clone(&chunks);
Box::pin(async move {
@ -43,15 +44,10 @@ async fn streaming_timeout_terminates_docker_exec_before_returning() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -67,15 +63,17 @@ async fn streaming_timeout_terminates_docker_exec_before_returning() {
let marker = "fabro_streaming_timeout_sentinel";
let command = format!("trap '' HUP TERM; echo start; sleep 5 # {marker}");
let result = sandbox
.exec_command_streaming(ExecStreamingRequest {
timeout_ms: Some(200),
output_callback: Some(capture_bytes(Arc::clone(&chunks))),
..ExecStreamingRequest::new(&command)
})
.exec_command_streaming(
ExecSpec::bash(&command).timeout(Duration::from_millis(200)),
ExecControls {
sink: Some(capture_bytes(Arc::clone(&chunks))),
..ExecControls::default()
},
)
.await
.expect("streaming command should return a timeout result");
assert!(result.result.is_timed_out());
assert_eq!(result.result.termination, Termination::TimedOut);
assert!(
String::from_utf8_lossy(&chunks.lock().await).contains("start"),
"stream should include output emitted before timeout"
@ -94,14 +92,14 @@ async fn streaming_timeout_terminates_docker_exec_before_returning() {
.await
.expect("process probe should run");
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
let probe = probe.stdout_lossy();
assert!(
!probe.stdout.contains(marker),
"timed-out docker exec should be terminated before returning, found: {}",
probe.stdout
!probe.contains(marker),
"timed-out docker exec should be terminated before returning, found: {probe}"
);
}
@ -116,15 +114,10 @@ async fn streaming_command_receives_exact_stdin_and_eof() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -137,11 +130,12 @@ async fn streaming_command_receives_exact_stdin_and_eof() {
let stdin = b"first line\n$(touch /tmp/must-not-run)\nlast line".to_vec();
let result = sandbox
.exec_command_streaming(ExecStreamingRequest {
timeout_ms: Some(10_000),
stdin: Some(stdin.clone()),
..ExecStreamingRequest::new("cat")
})
.exec_command_streaming(
ExecSpec::bash("cat")
.timeout(Duration::from_secs(10))
.stdin(stdin.clone()),
ExecControls::default(),
)
.await
.expect("streaming command should read stdin and finish at EOF");
let injection_probe = sandbox
@ -150,19 +144,19 @@ async fn streaming_command_receives_exact_stdin_and_eof() {
.expect("injection probe should run");
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
assert!(
result.result.is_success(),
result.result.success(),
"stdin command failed: stdout={} stderr={}",
result.result.stdout,
result.result.stderr
result.result.stdout_lossy(),
result.result.stderr_lossy()
);
assert_eq!(result.result.stdout.as_bytes(), stdin);
assert_eq!(result.result.stdout, stdin);
assert!(
injection_probe.is_success(),
injection_probe.success(),
"stdin bytes must not be evaluated as shell source"
);
}
@ -178,17 +172,15 @@ async fn cloned_docker_sandbox_uses_repos_checkout_and_workspace_symlink() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: false,
..SandboxOptions::default()
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest {
origin_url: Some("https://github.com/brynary/rack-test".to_string()),
..CloneRequest::default()
},
None,
None,
Some("https://github.com/brynary/rack-test".to_string()),
None,
None,
None,
)
.await
.expect("docker sandbox should construct");
@ -215,17 +207,17 @@ async fn cloned_docker_sandbox_uses_repos_checkout_and_workspace_symlink() {
.await
.expect("layout verification command should run");
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
assert!(
result.is_success(),
result.success(),
"layout verification failed: stdout={} stderr={}",
result.stdout,
result.stderr
result.stdout_lossy(),
result.stderr_lossy()
);
assert!(result.stdout.contains("true"));
assert!(result.stdout_lossy().contains("true"));
}
// Both command paths must evaluate the same interpreter, so Bash-only syntax
@ -244,16 +236,11 @@ async fn docker_runs_clean_bash_through_both_command_paths() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
env: BTreeMap::from([("BASH_ENV".to_string(), "/tmp/fabro-bash-env".to_string())]),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
})
.env_var("BASH_ENV".to_string(), "/tmp/fabro-bash-env".to_string()),
&CloneRequest::none(),
None,
None,
)
@ -276,7 +263,7 @@ async fn docker_runs_clean_bash_through_both_command_paths() {
)
.await
.expect("startup-file fixture should be created");
assert!(setup.is_success());
assert!(setup.success());
// Arrays, `[[ ]]`, and `${arr[@]}` are Bash-only; `shopt -q login_shell`
// proves the command did not run under a login shell. Exact output also
@ -291,33 +278,35 @@ async fn docker_runs_clean_bash_through_both_command_paths() {
let chunks = Arc::new(Mutex::new(Vec::new()));
let streaming = sandbox
.exec_command_streaming(ExecStreamingRequest {
timeout_ms: Some(10_000),
output_callback: Some(capture_bytes(Arc::clone(&chunks))),
..ExecStreamingRequest::new(command)
})
.exec_command_streaming(
ExecSpec::bash(command).timeout(Duration::from_secs(10)),
ExecControls {
sink: Some(capture_bytes(Arc::clone(&chunks))),
..ExecControls::default()
},
)
.await
.expect("streaming command should run");
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
assert!(
non_streaming.is_success(),
non_streaming.success(),
"non-streaming Bash-only command failed: stdout={} stderr={}",
non_streaming.stdout,
non_streaming.stderr
non_streaming.stdout_lossy(),
non_streaming.stderr_lossy()
);
assert_eq!(non_streaming.stdout.trim(), "two");
assert_eq!(non_streaming.stdout_lossy().trim(), "two");
assert!(
streaming.result.is_success(),
streaming.result.success(),
"streaming Bash-only command failed: stdout={} stderr={}",
streaming.result.stdout,
streaming.result.stderr
streaming.result.stdout_lossy(),
streaming.result.stderr_lossy()
);
assert_eq!(streaming.result.stdout.trim(), "two");
assert_eq!(streaming.result.stdout_lossy().trim(), "two");
assert_eq!(String::from_utf8_lossy(&chunks.lock().await).trim(), "two");
}
@ -339,15 +328,10 @@ async fn docker_glob_matches_patterns_containing_a_path_separator() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -379,15 +363,15 @@ async fn docker_glob_matches_patterns_containing_a_path_separator() {
let recursive = sandbox.glob("**/SKILL.md", Some("skills")).await;
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
assert!(
seed.is_success(),
seed.success(),
"seeding the skills tree failed: stdout={} stderr={}",
seed.stdout,
seed.stderr
seed.stdout_lossy(),
seed.stderr_lossy()
);
let one_level = one_level.expect("glob should run");
@ -424,15 +408,10 @@ async fn docker_runtime_directory_is_private_and_outside_workspace() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -471,12 +450,13 @@ async fn docker_runtime_directory_is_private_and_outside_workspace() {
let readback = sandbox.read_file_text(&blob_path).await;
sandbox
.cleanup()
.delete()
.await
.expect("docker cleanup should succeed");
assert!(modes.is_success(), "stat failed: {}", modes.stderr);
let modes: Vec<&str> = modes.stdout.split_whitespace().collect();
assert!(modes.success(), "stat failed: {}", modes.stderr_lossy());
let modes = modes.stdout_lossy();
let modes: Vec<&str> = modes.split_whitespace().collect();
assert_eq!(
modes,
["700", "600"],
@ -503,15 +483,10 @@ async fn docker_sandbox_satisfies_pebbles_environment_contract() {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(image.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: image.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -531,7 +506,7 @@ async fn docker_sandbox_satisfies_pebbles_environment_contract() {
}
.await;
sandbox
.cleanup()
.delete()
.await
.expect("docker sandbox should clean up");
outcome.expect("the Docker sandbox satisfies pebble's environment contract");

20
lib/components/fabro-sandbox/tests/driver_bench.rs
View file

@ -36,8 +36,7 @@ use std::sync::Arc;
use std::time::{Duration, Instant};
use fabro_sandbox::{
ProviderAccess, RunSandbox, SandboxOptions, SandboxProviderKind, local_sandbox,
provider_sandbox,
CloneRequest, ProviderAccess, RunSandbox, SandboxProviderKind, local_sandbox, provider_sandbox,
};
use sandbox_driver::{
ExecSpec, GrepOptions, Sandbox as DriverHandle, SandboxProvider, SandboxSource, SandboxSpec,
@ -279,7 +278,7 @@ async fn unpack_fabro(sandbox: &RunSandbox, repo: &Repository) {
)
.await
.expect("unpack exec");
assert!(result.is_success(), "unpack failed: {}", result.stderr);
assert!(result.success(), "unpack failed: {}", result.stderr_lossy());
}
async fn unpack_driver(sandbox: &dyn DriverHandle, repo: &Repository) {
@ -363,15 +362,10 @@ async fn agent_tool_call_latency_through_the_driver() {
let fabro_docker = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(IMAGE.to_owned()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: IMAGE.to_owned(),
}),
&CloneRequest::none(),
None,
None,
)
@ -380,7 +374,7 @@ async fn agent_tool_call_latency_through_the_driver() {
fabro_docker.initialize().await.expect("fabro docker init");
unpack_fabro(&fabro_docker, &repo).await;
rows.extend(bench_fabro("fabro Docker (driver-backed)", &fabro_docker, &repo).await);
fabro_docker.cleanup().await.expect("fabro docker cleanup");
fabro_docker.delete().await.expect("fabro docker cleanup");
let docker_provider = Arc::new(DockerProvider::connect().await.expect("docker connect"));
let container = docker_provider

2
lib/components/fabro-workflow/src/artifact.rs
View file

@ -1239,7 +1239,7 @@ mod tests {
.unwrap();
assert_eq!(
env.exists_calls(),
env.driver().memory_fs().exists_calls(),
1,
"sandbox locality should be probed once per resolution pass"
);

4
lib/components/fabro-workflow/src/event.rs
View file

@ -1,9 +1,9 @@
mod convert;
mod driver_events;
mod emitter;
mod events;
mod names;
mod redaction;
mod sandbox_bridge;
mod sink;
mod stored_fields;
#[cfg(test)]
@ -12,13 +12,13 @@ mod test_support;
pub use fabro_types::{EventBody, RunNoticeCode, RunNoticeLevel};
pub use self::convert::{to_run_event, to_run_event_at};
pub use self::driver_events::DriverEventRecorder;
pub use self::emitter::Emitter;
pub use self::events::{Event, SandboxLifecycle};
pub use self::names::event_name;
pub use self::redaction::{
build_redacted_event_payload, event_payload_from_redacted_json, redacted_event_json,
};
pub use self::sandbox_bridge::SandboxEventBridge;
pub use self::sink::{
RunEventLogger, RunEventPersistenceError, RunEventSink, StoreProgressLogger, append_event,
append_event_if, append_event_to_sink, create_run,

203
lib/components/fabro-workflow/src/event/convert.rs
View file

@ -52,8 +52,6 @@ fn git_push_attempt_props(
.token
.and_then(|token| token.age_at(attempt.started_at))
.map(|age| u64::try_from(age.as_millis()).unwrap_or(u64::MAX)),
credential_action: attempt.credential_action,
refresh_error: attempt.refresh_error,
})
.collect()
}
@ -684,91 +682,8 @@ fn event_body_from_event(event: &Event) -> EventBody {
causes: causes.clone(),
duration_ms: *duration_ms,
}),
SandboxLifecycle::StartStarted { provider } => {
EventBody::SandboxStartStarted(fabro_types::SandboxStartStartedProps {
provider: provider.clone(),
})
}
SandboxLifecycle::StartCompleted {
provider,
duration_ms,
} => EventBody::SandboxStartCompleted(fabro_types::SandboxStartCompletedProps {
provider: provider.clone(),
duration_ms: *duration_ms,
}),
SandboxLifecycle::StartFailed {
provider,
error,
causes,
} => EventBody::SandboxStartFailed(fabro_types::SandboxStartFailedProps {
provider: provider.clone(),
error: error.clone(),
causes: causes.clone(),
}),
SandboxLifecycle::StopStarted { provider } => {
EventBody::SandboxStopStarted(fabro_types::SandboxStopStartedProps {
provider: provider.clone(),
})
}
SandboxLifecycle::StopCompleted {
provider,
duration_ms,
} => EventBody::SandboxStopCompleted(fabro_types::SandboxStopCompletedProps {
provider: provider.clone(),
duration_ms: *duration_ms,
}),
SandboxLifecycle::StopFailed {
provider,
error,
causes,
} => EventBody::SandboxStopFailed(fabro_types::SandboxStopFailedProps {
provider: provider.clone(),
error: error.clone(),
causes: causes.clone(),
}),
SandboxLifecycle::DeleteStarted { provider } => {
EventBody::SandboxDeleteStarted(fabro_types::SandboxDeleteStartedProps {
provider: provider.clone(),
})
}
SandboxLifecycle::DeleteCompleted {
provider,
duration_ms,
} => EventBody::SandboxDeleteCompleted(fabro_types::SandboxDeleteCompletedProps {
provider: provider.clone(),
duration_ms: *duration_ms,
}),
SandboxLifecycle::DeleteFailed {
provider,
error,
causes,
} => EventBody::SandboxDeleteFailed(fabro_types::SandboxDeleteFailedProps {
provider: provider.clone(),
error: error.clone(),
causes: causes.clone(),
}),
SandboxLifecycle::SnapshotPulling { name } => {
EventBody::SnapshotPulling(fabro_types::SnapshotNameProps { name: name.clone() })
}
SandboxLifecycle::SnapshotCreating { name } => {
EventBody::SnapshotCreating(fabro_types::SnapshotNameProps { name: name.clone() })
}
SandboxLifecycle::SnapshotReady { name, duration_ms } => {
EventBody::SnapshotReady(fabro_types::SnapshotCompletedProps {
name: name.clone(),
duration_ms: *duration_ms,
})
}
SandboxLifecycle::SnapshotFailed {
name,
error,
causes,
} => EventBody::SnapshotFailed(fabro_types::SnapshotFailedProps {
name: name.clone(),
error: error.clone(),
causes: causes.clone(),
}),
},
Event::SandboxDriver { event } => EventBody::sandbox_driver(event.clone()),
Event::SandboxInitialized {
working_directory,
provider,
@ -1298,22 +1213,49 @@ mod tests {
}
#[test]
fn run_event_sandbox_stop_and_delete_use_distinct_event_names() {
let stopped = to_run_event(&fixtures::RUN_5, &Event::Sandbox {
event: SandboxLifecycle::StopCompleted {
provider: "docker".to_string(),
duration_ms: 10,
},
fn run_event_driver_events_are_named_from_the_subject_action_and_phase() {
let stopped = to_run_event(&fixtures::RUN_5, &Event::SandboxDriver {
event: driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 1},
"occurred_at": "2026-05-09T12:00:00Z",
"provider": "docker",
"subject": {"type": "sandbox", "id": "container-1"},
"type": "operation_completed",
"action": "stop",
"duration": {"secs": 0, "nanos": 10_000_000}
})),
});
let deleted = to_run_event(&fixtures::RUN_5, &Event::Sandbox {
event: SandboxLifecycle::DeleteCompleted {
provider: "docker".to_string(),
duration_ms: 20,
},
let building = to_run_event(&fixtures::RUN_5, &Event::SandboxDriver {
event: driver_event(serde_json::json!({
"id": {"source_id": "test", "sequence": 2},
"occurred_at": "2026-05-09T12:00:01Z",
"provider": "daytona",
"subject": {"type": "snapshot", "name": "sandbox-driver-abc"},
"type": "operation_started",
"action": "create"
})),
});
assert_eq!(stopped.event_name(), "sandbox.stop.completed");
assert_eq!(deleted.event_name(), "sandbox.delete.completed");
assert_eq!(building.event_name(), "snapshot.create.started");
let properties = stopped.properties().unwrap();
assert_eq!(properties["action"], "stop");
assert_eq!(properties["subject"]["id"], "container-1");
assert_eq!(properties["duration"]["nanos"], 10_000_000);
// The stored form reads back as the driver's event.
let round_trip: RunEvent = serde_json::from_value(serde_json::to_value(&stopped).unwrap())
.expect("a stored driver event decodes");
assert!(matches!(
&round_trip.body,
EventBody::SandboxDriver { name, event }
if name == "sandbox.stop.completed"
&& matches!(event.body, sandbox_driver::EventBody::OperationCompleted { .. })
));
}
fn driver_event(value: serde_json::Value) -> sandbox_driver::Event {
serde_json::from_value(value).expect("a driver event")
}
#[test]
@ -1770,26 +1712,22 @@ mod tests {
expires_at,
},
}),
credential_action: Some(fabro_sandbox::RemoteCredentialAction::Embedded),
refresh_error: None,
},
// Terminal classified failure with a refresh error: the last
// attempt carries its classification too.
fabro_sandbox::PushAttempt {
attempt: 2,
started_at: started_at + chrono::Duration::seconds(3),
success: false,
retry_reason: Some(fabro_sandbox::GitRetryReason::TransientInfra),
exec_output_tail: Some(exec_tail()),
token: Some(fabro_sandbox::TokenSnapshot {
attempt: 2,
started_at: started_at + chrono::Duration::seconds(3),
success: false,
retry_reason: Some(fabro_sandbox::GitRetryReason::TransientInfra),
exec_output_tail: Some(exec_tail()),
token: Some(fabro_sandbox::TokenSnapshot {
generation: 14,
provenance: fabro_sandbox::TokenProvenance::Reused {
minted_at,
expires_at,
},
}),
credential_action: Some(fabro_sandbox::RemoteCredentialAction::Unchanged),
refresh_error: Some(fabro_sandbox::RefreshErrorKind::SetUrl),
},
];
let expected_attempts = git_push_attempt_props(&runtime_attempts);
@ -1808,11 +1746,8 @@ mod tests {
assert_eq!(serialized[0]["token_generation"], 14);
assert_eq!(serialized[0]["token_provenance"], "minted");
assert_eq!(serialized[0]["token_age_ms"], 180);
assert_eq!(serialized[0]["credential_action"], "embedded");
assert!(serialized[0].get("refresh_error").is_none());
assert_eq!(serialized[1]["classified_reason"], "transient_infra");
assert_eq!(serialized[1]["token_provenance"], "reused");
assert_eq!(serialized[1]["refresh_error"], "set_url");
// The provenance enum never nests in stored events.
assert!(serialized[0].get("token").is_none());
@ -1826,20 +1761,43 @@ mod tests {
}
}
/// Attempts stored by earlier releases carried `credential_action` and
/// `refresh_error` from the origin-URL credential design. The fields are
/// gone; the stored events still read.
#[test]
fn stored_attempts_with_retired_credential_fields_still_deserialize() {
let json = serde_json::json!({
"attempt": 1,
"started_at": "2026-03-30T12:00:01.000Z",
"success": true,
"token_generation": 3,
"token_provenance": "reused",
"token_age_ms": 120,
"credential_action": "embedded",
"refresh_error": "set_url"
});
let props: ::fabro_types::run_event::GitPushAttemptProps =
serde_json::from_value(json).unwrap();
assert_eq!(props.attempt, 1);
assert_eq!(props.token_generation, Some(3));
assert_eq!(
props.token_provenance,
Some(::fabro_types::run_event::GitTokenProvenance::Reused)
);
}
#[test]
fn successful_single_attempt_push_omits_failure_fields() {
let attempts = vec![fabro_sandbox::PushAttempt {
attempt: 1,
started_at: Utc::now(),
success: true,
retry_reason: None,
exec_output_tail: None,
token: Some(fabro_sandbox::TokenSnapshot {
attempt: 1,
started_at: Utc::now(),
success: true,
retry_reason: None,
exec_output_tail: None,
token: Some(fabro_sandbox::TokenSnapshot {
generation: 0,
provenance: fabro_sandbox::TokenProvenance::Static,
}),
credential_action: Some(fabro_sandbox::RemoteCredentialAction::Unchanged),
refresh_error: None,
}];
let stored = to_run_event(&fixtures::RUN_1, &Event::GitPush {
branch: "fabro/run/run-1".to_string(),
@ -1852,12 +1810,7 @@ mod tests {
let attempt = &json["properties"]["attempts"][0];
assert_eq!(attempt["success"], true);
assert_eq!(attempt["token_provenance"], "static");
for absent in [
"classified_reason",
"exec_output_tail",
"token_age_ms",
"refresh_error",
] {
for absent in ["classified_reason", "exec_output_tail", "token_age_ms"] {
assert!(attempt.get(absent).is_none(), "{absent} should be omitted");
}
}

36
lib/components/fabro-workflow/src/event/driver_events.rs Normal file
View file

@ -0,0 +1,36 @@
//! The sandbox driver's events for a run's sandbox, kept as run events.
//!
//! A run's sandbox is created or attached with a driver [`EventContext`]
//! whose observer is a [`DriverEventRecorder`]. Everything the driver
//! reports about the sandbox — the operations it performs and their
//! outcome, progress inside a create such as an image pull, snapshot
//! builds, state observations, notices — is stored whole as an
//! [`Event::SandboxDriver`], named from the event (see
//! `fabro_types::sandbox_driver_event_name`).
//!
//! [`EventContext`]: sandbox_driver::EventContext
use std::sync::Arc;
use async_trait::async_trait;
use sandbox_driver::{Event as DriverEvent, EventObserver};
use super::{Emitter, Event};
/// Records every event the driver reports as a run event.
pub struct DriverEventRecorder {
emitter: Arc<Emitter>,
}
impl DriverEventRecorder {
pub fn new(emitter: Arc<Emitter>) -> Self {
Self { emitter }
}
}
#[async_trait]
impl EventObserver for DriverEventRecorder {
async fn observe(&self, event: DriverEvent) {
self.emitter.emit(&Event::SandboxDriver { event });
}
}

155
lib/components/fabro-workflow/src/event/events.rs
View file

@ -513,11 +513,17 @@ pub enum Event {
status: String,
duration_ms: u64,
},
/// A fact about the run's sandbox: the pipeline bringing it up, or a
/// driver operation on it.
/// A fact about the run's sandbox from the pipeline bringing it up.
Sandbox {
event: SandboxLifecycle,
},
/// An event the sandbox driver reported about the run's sandbox (an
/// operation and its outcome, progress inside a create, a state
/// observation, a notice), kept whole. Named from the event; see
/// `fabro_types::sandbox_driver_event_name`.
SandboxDriver {
event: sandbox_driver::Event,
},
/// Emitted after the sandbox has been initialized (by engine lifecycle).
SandboxInitialized {
working_directory: String,
@ -763,7 +769,7 @@ pub enum Event {
/// Initializing, ready, and failed are the pipeline's view of bringing the
/// sandbox up — create, activate, and prepare the workspace as one step.
/// The rest are the sandbox driver's own operations and snapshot work,
/// translated from its events by [`super::SandboxEventBridge`].
/// the driver's own events are kept whole as [`Event::SandboxDriver`].
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum SandboxLifecycle {
Initializing {
@ -784,68 +790,11 @@ pub enum SandboxLifecycle {
causes: Vec<String>,
duration_ms: u64,
},
StartStarted {
provider: String,
},
StartCompleted {
provider: String,
duration_ms: u64,
},
StartFailed {
provider: String,
error: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
causes: Vec<String>,
},
StopStarted {
provider: String,
},
StopCompleted {
provider: String,
duration_ms: u64,
},
StopFailed {
provider: String,
error: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
causes: Vec<String>,
},
DeleteStarted {
provider: String,
},
DeleteCompleted {
provider: String,
duration_ms: u64,
},
DeleteFailed {
provider: String,
error: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
causes: Vec<String>,
},
/// The provider is pulling the image the sandbox is created from.
SnapshotPulling {
name: String,
},
/// The provider is building or activating the snapshot.
SnapshotCreating {
name: String,
},
SnapshotReady {
name: String,
duration_ms: u64,
},
SnapshotFailed {
name: String,
error: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
causes: Vec<String>,
},
}
impl SandboxLifecycle {
pub fn trace(&self) {
use tracing::{debug, error, info, warn};
use tracing::{debug, error, info};
match self {
Self::Initializing { provider } => {
debug!(provider, "Sandbox initializing");
@ -865,70 +814,6 @@ impl SandboxLifecycle {
} => {
error!(provider, error, causes = ?causes, duration_ms, "Sandbox init failed");
}
Self::StartStarted { provider } => {
info!(provider, "Sandbox start started");
}
Self::StartCompleted {
provider,
duration_ms,
} => {
info!(provider, duration_ms, "Sandbox start completed");
}
Self::StartFailed {
provider,
error,
causes,
} => {
warn!(provider, error, causes = ?causes, "Sandbox start failed");
}
Self::StopStarted { provider } => {
info!(provider, "Sandbox stop started");
}
Self::StopCompleted {
provider,
duration_ms,
} => {
info!(provider, duration_ms, "Sandbox stop completed");
}
Self::StopFailed {
provider,
error,
causes,
} => {
warn!(provider, error, causes = ?causes, "Sandbox stop failed");
}
Self::DeleteStarted { provider } => {
info!(provider, "Sandbox delete started");
}
Self::DeleteCompleted {
provider,
duration_ms,
} => {
info!(provider, duration_ms, "Sandbox delete completed");
}
Self::DeleteFailed {
provider,
error,
causes,
} => {
warn!(provider, error, causes = ?causes, "Sandbox delete failed");
}
Self::SnapshotPulling { name } => {
debug!(name, "Snapshot pulling");
}
Self::SnapshotCreating { name } => {
debug!(name, "Snapshot creating");
}
Self::SnapshotReady { name, duration_ms } => {
info!(name, duration_ms, "Snapshot ready");
}
Self::SnapshotFailed {
name,
error,
causes,
} => {
error!(name, error, causes = ?causes, "Snapshot failed");
}
}
}
}
@ -1484,6 +1369,7 @@ impl Event {
}
Self::Agent { event, .. } => event.event.trace(&event.session_id),
Self::Sandbox { event } => event.trace(),
Self::SandboxDriver { event } => trace_driver_event(event),
Self::SandboxInitialized {
working_directory,
provider,
@ -1761,3 +1647,22 @@ impl Event {
}
}
}
/// Traces a sandbox driver event under its run event name.
fn trace_driver_event(event: &sandbox_driver::Event) {
use sandbox_driver::EventBody as Body;
use tracing::{debug, info, warn};
let name = fabro_types::sandbox_driver_event_name(event);
match &event.body {
Body::OperationFailed { error, .. } => {
warn!(event = %name, error = %error.message, "Sandbox driver operation failed");
}
Body::OperationCompleted { duration, .. } => {
let duration_ms = u64::try_from(duration.as_millis()).unwrap_or(u64::MAX);
info!(event = %name, duration_ms, "Sandbox driver operation completed");
}
Body::OperationStarted { .. } => info!(event = %name, "Sandbox driver operation started"),
_ => debug!(event = %name, "Sandbox driver event"),
}
}

25
lib/components/fabro-workflow/src/event/names.rs
View file

@ -1,8 +1,13 @@
use std::borrow::Cow;
use super::{Event, SandboxLifecycle};
#[must_use]
pub fn event_name(event: &Event) -> &'static str {
match event {
pub fn event_name(event: &Event) -> Cow<'static, str> {
let name: &'static str = match event {
Event::SandboxDriver { event } => {
return Cow::Owned(fabro_types::sandbox_driver_event_name(event));
}
Event::RunCreated { .. } => "run.created",
Event::WorkflowRunStarted { .. } => "run.started",
Event::RunSubmitted { .. } => "run.submitted",
@ -67,19 +72,6 @@ pub fn event_name(event: &Event) -> &'static str {
SandboxLifecycle::Initializing { .. } => "sandbox.initializing",
SandboxLifecycle::Ready { .. } => "sandbox.ready",
SandboxLifecycle::InitializeFailed { .. } => "sandbox.failed",
SandboxLifecycle::StartStarted { .. } => "sandbox.start.started",
SandboxLifecycle::StartCompleted { .. } => "sandbox.start.completed",
SandboxLifecycle::StartFailed { .. } => "sandbox.start.failed",
SandboxLifecycle::StopStarted { .. } => "sandbox.stop.started",
SandboxLifecycle::StopCompleted { .. } => "sandbox.stop.completed",
SandboxLifecycle::StopFailed { .. } => "sandbox.stop.failed",
SandboxLifecycle::DeleteStarted { .. } => "sandbox.delete.started",
SandboxLifecycle::DeleteCompleted { .. } => "sandbox.delete.completed",
SandboxLifecycle::DeleteFailed { .. } => "sandbox.delete.failed",
SandboxLifecycle::SnapshotPulling { .. } => "sandbox.snapshot.pulling",
SandboxLifecycle::SnapshotCreating { .. } => "sandbox.snapshot.creating",
SandboxLifecycle::SnapshotReady { .. } => "sandbox.snapshot.ready",
SandboxLifecycle::SnapshotFailed { .. } => "sandbox.snapshot.failed",
},
Event::SandboxInitialized { .. } => "sandbox.initialized",
Event::SetupStarted { .. } => "setup.started",
@ -112,7 +104,8 @@ pub fn event_name(event: &Event) -> &'static str {
Event::PullRequestLinked { .. } => "pull_request.linked",
Event::PullRequestUnlinked { .. } => "pull_request.unlinked",
Event::PullRequestFailed { .. } => "pull_request.failed",
}
};
Cow::Borrowed(name)
}
#[cfg(test)]

192
lib/components/fabro-workflow/src/event/sandbox_bridge.rs
View file

@ -1,192 +0,0 @@
//! The sandbox driver's events for a run's sandbox, as workflow events.
//!
//! A run's sandbox is created or attached with a driver [`EventContext`]
//! whose observer is a [`SandboxEventBridge`]. The driver reports every
//! operation it performs — start, stop, delete, the image pull inside a
//! create, snapshot builds — and the bridge turns the ones fabro records
//! on a run into [`SandboxLifecycle`] events. Everything else the driver
//! reports (state observations, notices, other operations) is not a run
//! event and is dropped here.
use std::collections::HashMap;
use std::sync::{Arc, Mutex, PoisonError};
use std::time::{Duration, Instant};
use async_trait::async_trait;
use sandbox_driver::{
Action, ErrorReport, Event as DriverEvent, EventBody as DriverEventBody, EventObserver,
EventSubject, OperationId, ProgressCode,
};
use super::{Emitter, Event, SandboxLifecycle};
/// Emits the workflow's sandbox lifecycle events from the driver's.
pub struct SandboxEventBridge {
emitter: Arc<Emitter>,
/// Fabro's name for the provider, which is what the run records; the
/// driver's own kind name can differ (`host` for a `local` run).
provider: String,
/// The image the sandbox is created from, named on pull events.
image: Option<String>,
/// Creates that pulled an image, by operation, with when the pull began.
pulls: Mutex<HashMap<OperationId, Instant>>,
}
impl SandboxEventBridge {
pub fn new(emitter: Arc<Emitter>, provider: impl Into<String>, image: Option<String>) -> Self {
Self {
emitter,
provider: provider.into(),
image,
pulls: Mutex::new(HashMap::new()),
}
}
/// The lifecycle event a driver event stands for, if fabro records one.
fn translate(&self, event: &DriverEvent) -> Option<SandboxLifecycle> {
match &event.subject {
EventSubject::Sandbox { .. } => self.translate_sandbox(event),
EventSubject::Snapshot { id, name } => {
let name = name
.clone()
.or_else(|| id.as_ref().map(ToString::to_string))
.unwrap_or_default();
match &event.body {
DriverEventBody::OperationStarted { .. } => {
Some(SandboxLifecycle::SnapshotCreating { name })
}
DriverEventBody::OperationCompleted { duration, .. } => {
Some(SandboxLifecycle::SnapshotReady {
name,
duration_ms: duration_ms(*duration),
})
}
DriverEventBody::OperationFailed { error, .. } => {
Some(SandboxLifecycle::SnapshotFailed {
name,
error: error.message.clone(),
causes: error.causes.clone(),
})
}
_ => None,
}
}
_ => None,
}
}
fn translate_sandbox(&self, event: &DriverEvent) -> Option<SandboxLifecycle> {
let provider = self.provider.clone();
match &event.body {
DriverEventBody::OperationStarted { action } => match action {
Action::Start => Some(SandboxLifecycle::StartStarted { provider }),
Action::Stop => Some(SandboxLifecycle::StopStarted { provider }),
Action::Delete => Some(SandboxLifecycle::DeleteStarted { provider }),
_ => None,
},
DriverEventBody::OperationProgress { action, progress } => {
if *action != Action::Create || progress.code.as_str() != ProgressCode::IMAGE_PULL {
return None;
}
// The first pull report of a create opens the pull; later
// ones are the same pull's progress.
let operation_id = event.operation_id.clone()?;
let mut pulls = self.pulls.lock().unwrap_or_else(PoisonError::into_inner);
if pulls.contains_key(&operation_id) {
return None;
}
pulls.insert(operation_id, Instant::now());
Some(SandboxLifecycle::SnapshotPulling {
name: self
.image
.clone()
.or_else(|| progress.message.clone())
.unwrap_or_default(),
})
}
DriverEventBody::OperationCompleted { action, duration } => match action {
Action::Create => {
let pulled = self.take_pull(event.operation_id.as_ref())?;
Some(SandboxLifecycle::SnapshotReady {
name: self.image.clone().unwrap_or_default(),
duration_ms: duration_ms(pulled.elapsed()),
})
}
Action::Start => Some(SandboxLifecycle::StartCompleted {
provider,
duration_ms: duration_ms(*duration),
}),
Action::Stop => Some(SandboxLifecycle::StopCompleted {
provider,
duration_ms: duration_ms(*duration),
}),
Action::Delete => Some(SandboxLifecycle::DeleteCompleted {
provider,
duration_ms: duration_ms(*duration),
}),
_ => None,
},
DriverEventBody::OperationFailed { action, error, .. } => match action {
Action::Create => {
self.take_pull(event.operation_id.as_ref())?;
Some(SandboxLifecycle::SnapshotFailed {
name: self.image.clone().unwrap_or_default(),
error: error.message.clone(),
causes: error.causes.clone(),
})
}
Action::Start => Some(failed(error, |error, causes| {
SandboxLifecycle::StartFailed {
provider,
error,
causes,
}
})),
Action::Stop => Some(failed(error, |error, causes| {
SandboxLifecycle::StopFailed {
provider,
error,
causes,
}
})),
Action::Delete => Some(failed(error, |error, causes| {
SandboxLifecycle::DeleteFailed {
provider,
error,
causes,
}
})),
_ => None,
},
_ => None,
}
}
/// When the create `operation_id` began pulling its image, if it did.
fn take_pull(&self, operation_id: Option<&OperationId>) -> Option<Instant> {
self.pulls
.lock()
.unwrap_or_else(PoisonError::into_inner)
.remove(operation_id?)
}
}
fn failed(
error: &ErrorReport,
build: impl FnOnce(String, Vec<String>) -> SandboxLifecycle,
) -> SandboxLifecycle {
build(error.message.clone(), error.causes.clone())
}
fn duration_ms(duration: Duration) -> u64 {
u64::try_from(duration.as_millis()).unwrap_or(u64::MAX)
}
#[async_trait]
impl EventObserver for SandboxEventBridge {
async fn observe(&self, event: DriverEvent) {
if let Some(lifecycle) = self.translate(&event) {
self.emitter.emit(&Event::Sandbox { event: lifecycle });
}
}
}

126
lib/components/fabro-workflow/src/handler/command.rs
View file

@ -2,8 +2,11 @@ use std::path::Path;
use async_trait::async_trait;
use fabro_graphviz::graph::{ContextKeyAttr, Graph, Node};
use fabro_sandbox::sandbox::{CommandOutputCallback, ExecStreamingRequest};
use fabro_types::{CommandTermination, StageTiming};
use fabro_sandbox::{
ExecControls, ExecResultExt, ExecSpec, OutputSink, Termination, TransportError,
command_termination,
};
use fabro_types::StageTiming;
use fabro_util::shell::shell_quote;
use super::structured_output::{self, StructuredOutputError};
@ -108,7 +111,7 @@ impl Handler for CommandHandler {
let cancel_token = services.run.cancel_token().child_token();
let stage_id = stage_scope.stage_id();
let recorder = CommandLogRecorder::create(run_dir, &stage_id).await?;
let output_callback: CommandOutputCallback = {
let sink: OutputSink = {
let recorder = recorder.clone();
std::sync::Arc::new(move |_stream, bytes| {
let recorder = recorder.clone();
@ -116,21 +119,26 @@ impl Handler for CommandHandler {
recorder
.append(&bytes)
.await
.map_err(|err| fabro_sandbox::Error::message(err.to_string()))
.map_err(|err| TransportError::new(err.to_string()).into())
})
})
};
let mut spec =
ExecSpec::bash(&command).timeout(std::time::Duration::from_millis(timeout_ms));
for (key, value) in env_vars.into_iter().flatten() {
spec = spec.env_var(key, value);
}
if let Some(stdin) = stdin {
spec = spec.stdin(stdin);
}
let result = services
.run
.sandbox
.exec_command_streaming(ExecStreamingRequest {
timeout_ms: Some(timeout_ms),
env_vars,
cancel_token: Some(cancel_token.clone()),
stdin,
output_callback: Some(output_callback),
..ExecStreamingRequest::new(&command)
.exec_command_streaming(spec, ExecControls {
term: Some(cancel_token.clone()),
sink: Some(sink),
..ExecControls::default()
})
.await;
cancel_token.cancel();
@ -148,28 +156,31 @@ impl Handler for CommandHandler {
&Event::CommandCompleted {
node_id: node.id.clone(),
output: finalized.output_ref.clone(),
exit_code: result.exit_code,
duration_ms: result.duration_ms,
termination: result.termination,
exit_code: result.program_exit_code(),
duration_ms: result.duration_ms(),
termination: command_termination(result.termination),
output_bytes: finalized.output_bytes,
live_streaming: streaming.live_streaming,
},
&stage_scope,
);
if result.termination == CommandTermination::TimedOut {
if result.termination == Termination::TimedOut {
let mut reason = format!("Script timed out after {timeout_ms}ms: {script}");
append_output_tail(&mut reason, &finalized.output_text);
return Err(Error::handler(reason));
}
if result.termination == CommandTermination::Cancelled {
if matches!(
result.termination,
Termination::Cancelled | Termination::Killed
) {
let mut reason = format!("Script cancelled: {script}");
append_output_tail(&mut reason, &finalized.output_text);
return Err(Error::handler(reason));
}
if result.exit_code == Some(0) {
if result.success() {
let validation = output_schema.as_ref().map(|schema| {
(
schema,
@ -191,7 +202,7 @@ impl Handler for CommandHandler {
keys::COMMAND_OUTPUT.to_string(),
serde_json::json!(finalized.output_ref),
);
outcome.timing = Some(StageTiming::active_only(0, result.duration_ms));
outcome.timing = Some(StageTiming::active_only(0, result.duration_ms()));
if let Some((schema, Ok(validated))) = validation {
structured_output::apply_validated_output(node, schema, &validated, &mut outcome);
}
@ -199,7 +210,7 @@ impl Handler for CommandHandler {
} else {
let mut reason = format!(
"Script failed with exit code: {}",
result.exit_code.unwrap_or(-1)
result.program_exit_code().unwrap_or(-1)
);
append_output_tail(&mut reason, &finalized.output_text);
let mut outcome = Outcome::fail_classify(reason);
@ -207,7 +218,7 @@ impl Handler for CommandHandler {
keys::COMMAND_OUTPUT.to_string(),
serde_json::json!(finalized.output_ref),
);
outcome.timing = Some(StageTiming::active_only(0, result.duration_ms));
outcome.timing = Some(StageTiming::active_only(0, result.duration_ms()));
Ok(outcome)
}
}
@ -321,7 +332,8 @@ mod tests {
use bytes::Bytes;
use fabro_graphviz::graph::AttrValue;
use fabro_sandbox::test_support::MockSandbox;
use fabro_sandbox::Termination;
use fabro_sandbox::test_support::{MockSandbox, exec_result};
use fabro_store::{Database, RunDatabase, StageId};
use fabro_types::{Graph, RunProjection, RunSpec, WorkflowSettings, fixtures, test_support};
use object_store::memory::InMemory;
@ -842,13 +854,7 @@ mod tests {
#[tokio::test]
async fn command_invalid_output_schema_fails_before_execution() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: String::new(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 1,
},
exec_result: exec_result("", "", Some(0), Termination::Exited, 1),
..Default::default()
};
let handler = CommandHandler;
@ -1460,7 +1466,7 @@ mod tests {
assert_eq!(outcome.status, StageOutcome::Succeeded);
assert_eq!(
mock.captured_stdin(),
mock.driver().scripted_exec().captured_stdin().pop(),
Some(serde_json::to_vec(&parallel_results).unwrap())
);
assert!(
@ -1496,7 +1502,11 @@ mod tests {
assert_eq!(outcome.status, StageOutcome::Succeeded);
assert_eq!(
mock.captured_stdin().as_deref(),
mock.driver()
.scripted_exec()
.captured_stdin()
.pop()
.as_deref(),
Some(b"first\nlast".as_slice())
);
}
@ -1590,13 +1600,7 @@ mod tests {
#[tokio::test]
async fn executes_script_via_sandbox() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: "SANDBOX_MARKER\n".into(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 5,
},
exec_result: exec_result("SANDBOX_MARKER\n", "", Some(0), Termination::Exited, 5),
..Default::default()
};
@ -1633,13 +1637,7 @@ mod tests {
#[tokio::test]
async fn executes_python_script_via_sandbox() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: "PYTHON_SANDBOX\n".into(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 5,
},
exec_result: exec_result("PYTHON_SANDBOX\n", "", Some(0), Termination::Exited, 5),
..Default::default()
};
@ -1679,13 +1677,7 @@ mod tests {
#[tokio::test]
async fn passes_env_vars_to_sandbox() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: String::new(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 5,
},
exec_result: exec_result("", "", Some(0), Termination::Exited, 5),
..Default::default()
};
@ -1717,13 +1709,7 @@ mod tests {
#[tokio::test]
async fn refreshes_github_token_for_each_command_stage_when_near_expiry() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: String::new(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 5,
},
exec_result: exec_result("", "", Some(0), Termination::Exited, 5),
..Default::default()
};
let minter = std::sync::Arc::new(RefreshingMinter {
@ -1772,13 +1758,7 @@ mod tests {
#[tokio::test]
async fn passes_run_cancellation_to_sandbox() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: String::new(),
stderr: String::new(),
exit_code: Some(0),
termination: CommandTermination::Exited,
duration_ms: 5,
},
exec_result: exec_result("", "", Some(0), Termination::Exited, 5),
..Default::default()
};
@ -1800,19 +1780,19 @@ mod tests {
.await
.unwrap();
assert_eq!(spy.captured_term_stops(), vec![true]);
assert_eq!(spy.driver().scripted_exec().term_stops(), vec![true]);
}
#[tokio::test]
async fn script_handler_timeout_error_includes_output_tails() {
let spy = MockSandbox {
exec_result: fabro_sandbox::sandbox::ExecResult {
stdout: "partial stdout\n".into(),
stderr: "partial stderr\n".into(),
exit_code: None,
termination: CommandTermination::TimedOut,
duration_ms: 50,
},
exec_result: exec_result(
"partial stdout\n",
"partial stderr\n",
None,
Termination::TimedOut,
50,
),
..Default::default()
};

174
lib/components/fabro-workflow/src/handler/llm/acp.rs
View file

@ -13,7 +13,7 @@ use fabro_acp::{
};
use fabro_github::token_source::REFRESH_MARGIN;
use fabro_graphviz::graph::Node;
use fabro_sandbox::{RefreshOutcome, RunSandbox};
use fabro_sandbox::{RunSandbox, TokenSnapshot};
use fabro_static::EnvVars;
use fabro_types::{AgentBackend, SessionCapability, StageId, StageTiming};
use fabro_util::time::elapsed_ms;
@ -41,8 +41,9 @@ const REFRESH_INTERVAL_DEFAULT: Duration = Duration::from_mins(45);
/// Floor for expiry-driven rescheduling, so a token already inside the cache
/// margin cannot pin the loop in a hot cycle.
const REFRESH_RESCHEDULE_FLOOR: Duration = Duration::from_secs(30);
/// Upper bound on a single push-credential refresh (token mint + `git remote
/// set-url` exec). The turn-entry refresh runs before the ACP process spawns
/// Upper bound on a single credential refresh (token mint + rewriting the
/// checkout's credential store). The turn-entry refresh runs before the ACP
/// process spawns
/// and the ACP node uses `NodeTimeoutPolicy::HandlerManaged`, so without this
/// bound a stalled GitHub API call would hang node entry indefinitely.
const REFRESH_MINT_TIMEOUT: Duration = Duration::from_secs(30);
@ -77,9 +78,8 @@ fn parse_refresh_enabled(raw: Option<&str>) -> bool {
)
}
/// Parse the refresh-ahead loop interval. `None` disables the loop (explicit
/// `0`, mirroring the codebase's `set_autostop_interval` "0 to disable"
/// convention). Unset/empty or an unparsable value falls back to the default.
/// Parse the refresh-ahead loop interval. `None` disables the loop (an
/// explicit `0`). Unset/empty or an unparsable value falls back to the default.
fn parse_refresh_interval(raw: Option<&str>) -> Option<Duration> {
match raw.map(str::trim) {
None | Some("") => Some(REFRESH_INTERVAL_DEFAULT),
@ -114,10 +114,10 @@ fn push_cred_refresh_interval() -> Option<Duration> {
/// 45-minute sleep would leave the embedded token expired until the next
/// tick. Schedule from the token's own `expires_at` instead: wake when the
/// cache margin opens, so that tick re-mints. `None` disables the loop —
/// static credentials cannot be re-minted by waiting.
fn next_refresh_delay(outcome: &RefreshOutcome) -> Option<Duration> {
let token = outcome.token()?;
let expires_at = token.expires_at()?;
/// static credentials cannot be re-minted by waiting, and a sandbox without
/// managed credentials has nothing to renew.
fn next_refresh_delay(token: Option<&TokenSnapshot>) -> Option<Duration> {
let expires_at = token?.expires_at()?;
let margin = chrono::Duration::from_std(REFRESH_MARGIN).unwrap_or(chrono::Duration::MAX);
let until_margin = ((expires_at - margin) - chrono::Utc::now())
.to_std()
@ -125,11 +125,11 @@ fn next_refresh_delay(outcome: &RefreshOutcome) -> Option<Duration> {
Some(until_margin.max(REFRESH_RESCHEDULE_FLOOR))
}
/// Background loop that keeps the sandbox's push credentials fresh for the
/// Background loop that keeps the checkout's git credentials fresh for the
/// duration of one ACP turn, so a single turn that outlives the
/// installation-token TTL still pushes with a fresh token. Bounded by
/// `cancel` (the drop-guard cancels it at turn end). Each successful tick
/// reschedules from the embedded token's expiry ([`next_refresh_delay`]); a
/// reschedules from the installed token's expiry ([`next_refresh_delay`]); a
/// failed or timed-out tick retries after a shorter delay so a transient
/// error does not leave a longer-than-interval window with an expired token.
async fn refresh_ahead_loop<Fut>(
@ -138,7 +138,7 @@ async fn refresh_ahead_loop<Fut>(
interval: Duration,
initial_delay: Duration,
) where
Fut: Future<Output = fabro_sandbox::Result<RefreshOutcome>> + Send,
Fut: Future<Output = fabro_sandbox::Result<Option<TokenSnapshot>>> + Send,
{
let retry_delay = interval.min(Duration::from_mins(1));
let mut delay = initial_delay;
@ -147,27 +147,21 @@ async fn refresh_ahead_loop<Fut>(
() = cancel.cancelled() => break,
() = sleep(delay) => {
match timeout(REFRESH_MINT_TIMEOUT, refresh()).await {
Ok(Ok(outcome)) => {
match outcome {
RefreshOutcome::Embedded(token) => {
Ok(Ok(token)) => {
match &token {
Some(token) => {
tracing::info!(
generation = token.generation,
"refresh-ahead re-embedded push credentials mid-turn"
"refresh-ahead renewed the checkout's git credentials mid-turn"
);
}
RefreshOutcome::Unchanged(token) => {
None => {
tracing::debug!(
generation = token.generation,
"refresh-ahead tick: embedded push credentials still fresh"
);
}
RefreshOutcome::None => {
tracing::debug!(
"refresh-ahead tick: no managed push credentials to refresh"
"refresh-ahead tick: no managed git credentials to renew"
);
}
}
if let Some(next) = next_refresh_delay(&outcome) {
if let Some(next) = next_refresh_delay(token.as_ref()) {
delay = next;
} else {
tracing::debug!(
@ -318,24 +312,15 @@ impl AgentAcpBackend {
let refresh_enabled = push_cred_refresh_enabled();
let refresh_interval = refresh_enabled.then(push_cred_refresh_interval).flatten();
let refresh_schedule = if refresh_enabled {
match timeout(REFRESH_MINT_TIMEOUT, sandbox.refresh_push_credentials()).await {
Ok(Ok(outcome)) => {
match outcome {
RefreshOutcome::Embedded(token) => {
tracing::debug!(
generation = token.generation,
"refreshed sandbox push credentials at ACP turn entry"
);
}
RefreshOutcome::Unchanged(token) => {
tracing::debug!(
generation = token.generation,
"sandbox push credentials already fresh at ACP turn entry"
);
}
RefreshOutcome::None => {}
match timeout(REFRESH_MINT_TIMEOUT, sandbox.refresh_ambient_credentials()).await {
Ok(Ok(token)) => {
if let Some(token) = &token {
tracing::debug!(
generation = token.generation,
"refreshed the checkout's git credentials at ACP turn entry"
);
}
refresh_interval.zip(next_refresh_delay(&outcome))
refresh_interval.zip(next_refresh_delay(token.as_ref()))
}
Ok(Err(e)) => {
tracing::warn!(
@ -363,7 +348,7 @@ impl AgentAcpBackend {
AbortOnDrop(tokio::spawn(refresh_ahead_loop(
move || {
let sandbox = Arc::clone(&sandbox);
async move { sandbox.refresh_push_credentials().await }
async move { sandbox.refresh_ambient_credentials().await }
},
cancel_token.child_token(),
interval,
@ -659,11 +644,9 @@ mod tests {
use fabro_acp::{AcpError, AcpProcessExit};
use fabro_graphviz::graph::{AttrValue, Node};
use fabro_sandbox::test_support::MockSandbox;
use fabro_sandbox::{
RefreshOutcome, RemoteCredentialAction, RunSandbox, TokenProvenance, TokenSnapshot,
local_sandbox, shell_quote,
};
use fabro_sandbox::{RunSandbox, TokenProvenance, TokenSnapshot, local_sandbox};
use fabro_types::{CommandTermination, EventBody, ExecOutputTail};
use fabro_util::shell;
use tokio_util::sync::CancellationToken;
use super::{
@ -721,25 +704,21 @@ mod tests {
}
#[tokio::test]
async fn refresh_reports_no_action_without_managed_credentials() {
// A mock sandbox has no cloned workspace and so no managed push
// credentials: refresh is a no-op that must report no remote action
// and no token — the signal the refresh-ahead loop relies on to log
// at debug rather than falsely claim a re-embed.
async fn refresh_reports_no_token_without_managed_credentials() {
// A mock sandbox has no cloned workspace and so no managed
// credentials: refresh is a no-op that must report no token — the
// signal the refresh-ahead loop relies on to stop rather than claim
// a renewal.
let sandbox = MockSandbox::linux().sandbox();
assert_eq!(
sandbox.refresh_push_credentials().await.unwrap(),
RefreshOutcome::none()
);
assert_eq!(sandbox.refresh_ambient_credentials().await.unwrap(), None);
}
fn minted_outcome(
action: RemoteCredentialAction,
fn minted_token(
generation: u64,
minted_ago: chrono::Duration,
expires_in: chrono::Duration,
reused: bool,
) -> RefreshOutcome {
) -> TokenSnapshot {
let now = chrono::Utc::now();
let minted_at = now - minted_ago;
let expires_at = now + expires_in;
@ -754,34 +733,28 @@ mod tests {
expires_at,
}
};
let token = TokenSnapshot {
TokenSnapshot {
generation,
provenance,
};
match action {
RemoteCredentialAction::Embedded => RefreshOutcome::embedded(token),
RemoteCredentialAction::Unchanged => RefreshOutcome::unchanged(token),
RemoteCredentialAction::None => RefreshOutcome::none(),
}
}
fn static_outcome() -> RefreshOutcome {
RefreshOutcome::unchanged(TokenSnapshot {
fn static_token() -> TokenSnapshot {
TokenSnapshot {
generation: 0,
provenance: TokenProvenance::Static,
})
}
}
#[test]
fn next_refresh_delay_schedules_from_token_expiry_minus_margin() {
let outcome = minted_outcome(
RemoteCredentialAction::Embedded,
let outcome = minted_token(
1,
chrono::Duration::zero(),
chrono::Duration::minutes(60),
false,
);
let delay = next_refresh_delay(&outcome).unwrap();
let delay = next_refresh_delay(Some(&outcome)).unwrap();
// Expiry minus the 10-minute refresh margin: ~50 minutes out.
assert!(delay > Duration::from_mins(49), "{delay:?}");
assert!(delay <= Duration::from_mins(50), "{delay:?}");
@ -789,35 +762,37 @@ mod tests {
#[test]
fn next_refresh_delay_floors_when_the_margin_is_already_open() {
let outcome = minted_outcome(
RemoteCredentialAction::Unchanged,
let outcome = minted_token(
1,
chrono::Duration::minutes(55),
chrono::Duration::minutes(5),
true,
);
assert_eq!(next_refresh_delay(&outcome), Some(REFRESH_RESCHEDULE_FLOOR));
assert_eq!(
next_refresh_delay(Some(&outcome)),
Some(REFRESH_RESCHEDULE_FLOOR)
);
}
#[test]
fn next_refresh_delay_disables_the_loop_for_static_credentials() {
assert_eq!(next_refresh_delay(&static_outcome()), None);
assert_eq!(next_refresh_delay(Some(&static_token())), None);
}
#[test]
fn next_refresh_delay_disables_the_loop_without_managed_credentials() {
assert_eq!(next_refresh_delay(&RefreshOutcome::none()), None);
assert_eq!(next_refresh_delay(None), None);
}
/// Scripted refresh outcomes, recording when each refresh tick lands on
/// the (paused) tokio clock.
struct ScriptedRefresh {
script: Mutex<std::collections::VecDeque<RefreshOutcome>>,
script: Mutex<std::collections::VecDeque<TokenSnapshot>>,
ticks: Mutex<Vec<tokio::time::Instant>>,
}
impl ScriptedRefresh {
fn new(script: Vec<RefreshOutcome>) -> Arc<Self> {
fn new(script: Vec<TokenSnapshot>) -> Arc<Self> {
Arc::new(Self {
script: Mutex::new(script.into()),
ticks: Mutex::new(Vec::new()),
@ -831,19 +806,21 @@ mod tests {
/// The refresh the loop calls: answers the next scripted outcome.
fn refresher(
self: &Arc<Self>,
) -> impl Fn() -> std::future::Ready<fabro_sandbox::Result<RefreshOutcome>> + Send {
) -> impl Fn() -> std::future::Ready<fabro_sandbox::Result<Option<TokenSnapshot>>> + Send
{
let this = Arc::clone(self);
move || {
this.ticks
.lock()
.expect("ticks lock")
.push(tokio::time::Instant::now());
std::future::ready(Ok(this
.script
.lock()
.expect("script lock")
.pop_front()
.expect("refresh script exhausted")))
std::future::ready(Ok(Some(
this.script
.lock()
.expect("script lock")
.pop_front()
.expect("refresh script exhausted"),
)))
}
}
}
@ -860,24 +837,21 @@ mod tests {
let sandbox = ScriptedRefresh::new(vec![
// Minute 45: cache still fresh (expires minute 60, margin opens
// minute 50).
minted_outcome(
RemoteCredentialAction::Unchanged,
minted_token(
1,
chrono::Duration::minutes(45),
chrono::Duration::minutes(15),
true,
),
// Minute ~50: margin open → the source minted generation 2.
minted_outcome(
RemoteCredentialAction::Embedded,
minted_token(
2,
chrono::Duration::zero(),
chrono::Duration::minutes(60),
false,
),
// Minute ~100: generation 2 still fresh.
minted_outcome(
RemoteCredentialAction::Unchanged,
minted_token(
2,
chrono::Duration::minutes(50),
chrono::Duration::minutes(10),
@ -915,16 +889,14 @@ mod tests {
#[tokio::test(start_paused = true)]
async fn refresh_ahead_honors_the_expiry_based_initial_delay() {
let interval = Duration::from_mins(45);
let entry_outcome = minted_outcome(
RemoteCredentialAction::Unchanged,
let entry_outcome = minted_token(
1,
chrono::Duration::minutes(45),
chrono::Duration::minutes(15),
true,
);
let initial_delay = next_refresh_delay(&entry_outcome).unwrap();
let sandbox = ScriptedRefresh::new(vec![minted_outcome(
RemoteCredentialAction::Embedded,
let initial_delay = next_refresh_delay(Some(&entry_outcome)).unwrap();
let sandbox = ScriptedRefresh::new(vec![minted_token(
2,
chrono::Duration::zero(),
chrono::Duration::minutes(60),
@ -966,7 +938,7 @@ mod tests {
"acp.command".to_string(),
AttrValue::String(format!(
"python3 {}",
shell_quote(&script_path.to_string_lossy())
shell::shell_quote(&script_path.to_string_lossy())
)),
);
@ -1072,7 +1044,7 @@ mod tests {
"acp.command".to_string(),
AttrValue::String(format!(
"python3 {}",
shell_quote(&script_path.to_string_lossy())
shell::shell_quote(&script_path.to_string_lossy())
)),
);
@ -1135,7 +1107,7 @@ mod tests {
"acp.command".to_string(),
AttrValue::String(format!(
"python3 {}",
shell_quote(&script_path.to_string_lossy())
shell::shell_quote(&script_path.to_string_lossy())
)),
);
@ -1219,7 +1191,7 @@ mod tests {
"acp.command".to_string(),
AttrValue::String(format!(
"python3 {}",
shell_quote(&script_path.to_string_lossy())
shell::shell_quote(&script_path.to_string_lossy())
)),
);

66
lib/components/fabro-workflow/src/handler/llm/changed_files.rs
View file

@ -1,26 +1,28 @@
use std::collections::HashSet;
use std::sync::Arc;
use fabro_sandbox::{RunSandbox, shell_quote};
const DIFF_MARKER: &str = "__FABRO_CHANGED_FILES_DIFF__";
const UNTRACKED_MARKER: &str = "__FABRO_CHANGED_FILES_UNTRACKED__";
use fabro_sandbox::RunSandbox;
use fabro_util::shell;
use sandbox_driver::{Git as _, GitDiffOptions, GitRevisionRange};
/// The paths the working tree changed against `HEAD`, plus the untracked
/// files git does not ignore, sorted and deduplicated. A sandbox without
/// git, or a working directory that is not a repository, has no changed
/// files.
pub async fn detect_changed_files(sandbox: &Arc<RunSandbox>) -> Vec<String> {
let Ok(git) = sandbox.git() else {
return Vec::new();
};
let repo = sandbox.working_directory();
let mut files: Vec<String> = Vec::new();
let command = format!(
"printf '%s\\n' {diff}; git diff --name-only || true; \
printf '%s\\n' {untracked}; git ls-files --others --exclude-standard || true",
diff = shell_quote(DIFF_MARKER),
untracked = shell_quote(UNTRACKED_MARKER),
);
if let Ok(result) = sandbox
.exec_command(&command, 30_000, None, None, None)
if let Ok(entries) = git
.diff_entries(repo, &GitDiffOptions::new(GitRevisionRange::new("HEAD")))
.await
{
if result.is_success() {
files.extend(parse_changed_files(&result.stdout));
}
files.extend(entries.into_iter().map(|entry| entry.path));
}
if let Ok(untracked) = git.untracked_files(repo).await {
files.extend(untracked);
}
files.sort();
@ -42,42 +44,20 @@ pub async fn files_touched_since(
let last_file_touched = if files_touched.is_empty() {
None
} else {
let quoted_files: Vec<String> =
files_touched.iter().map(|file| shell_quote(file)).collect();
let quoted_files: Vec<String> = files_touched
.iter()
.map(|file| shell::shell_quote(file))
.collect();
let cmd = format!("ls -t {} | head -1", quoted_files.join(" "));
sandbox
.exec_command(&cmd, 5_000, None, None, None)
.await
.ok()
.and_then(|result| {
let trimmed = result.stdout.trim().to_string();
(result.is_success() && !trimmed.is_empty()).then_some(trimmed)
let trimmed = result.stdout_lossy().trim().to_string();
(result.success() && !trimmed.is_empty()).then_some(trimmed)
})
};
(files_touched, last_file_touched)
}
fn parse_changed_files(stdout: &str) -> impl Iterator<Item = String> + '_ {
stdout.lines().filter_map(|line| {
let trimmed = line.trim();
(!trimmed.is_empty() && trimmed != DIFF_MARKER && trimmed != UNTRACKED_MARKER)
.then(|| trimmed.to_string())
})
}
#[cfg(test)]
mod tests {
use super::parse_changed_files;
#[test]
fn parse_changed_files_ignores_section_markers() {
let files = parse_changed_files(
"__FABRO_CHANGED_FILES_DIFF__\nsrc/main.rs\n\
__FABRO_CHANGED_FILES_UNTRACKED__\nREADME.md\n",
)
.collect::<Vec<_>>();
assert_eq!(files, vec!["src/main.rs", "README.md"]);
}
}

8
lib/components/fabro-workflow/src/lifecycle/git.rs
View file

@ -87,10 +87,10 @@ pub(crate) struct PushResult {
pub(crate) async fn push_run_branch(
sandbox: &fabro_sandbox::RunSandbox,
branch: &str,
plan: &fabro_sandbox::RetryPlan,
policy: &fabro_sandbox::GitRetryPolicy,
) -> Result<fabro_sandbox::PushReport, fabro_sandbox::PushError> {
sandbox
.git_push_ref(&format!("refs/heads/{branch}:refs/heads/{branch}"), plan)
.git_push_ref(&format!("refs/heads/{branch}:refs/heads/{branch}"), policy)
.await
}
@ -333,9 +333,9 @@ impl RunLifecycle<WorkflowGraph> for GitLifecycle {
.as_ref()
.and_then(|g| g.run_branch.as_ref())
{
let plan = fabro_sandbox::RetryPlan::checkpoint_push();
let policy = fabro_sandbox::checkpoint_push_policy();
let (push_ok, exec_output_tail, attempts) =
match push_run_branch(self.sandbox.as_ref(), branch, &plan).await {
match push_run_branch(self.sandbox.as_ref(), branch, &policy).await {
Ok(report) => {
self.sandbox_git.record_successful_push();
(true, None, report.attempts)

203
lib/components/fabro-workflow/src/operations/start.rs
View file

@ -10,8 +10,7 @@ use fabro_llm::credentials::readiness;
use fabro_llm::lithos_catalog::Catalog;
use fabro_mcp::config::McpServerSettings;
use fabro_sandbox::{
DaytonaCredentials, ProviderAccess, ProviderSandboxSpec, SandboxOptions, SandboxSpec,
local_working_directory_from_environment, options_from_environment,
CloneRequest, DaytonaCredentials, ProviderAccess, SandboxSpec, sandbox_spec_for_environment,
};
use fabro_static::EnvVars;
#[cfg(test)]
@ -506,10 +505,17 @@ impl RunSession {
)));
}
}
let daytona = vault_guard
.get(EnvVars::DAYTONA_API_KEY)
.map(|api_key| DaytonaCredentials::from_api_key(api_key.to_string(), process_env_var));
let access = ProviderAccess {
providers: services.sandbox_providers.clone(),
daytona,
};
let sandbox = match sandbox_provider.bundled() {
Some(BundledProvider::Local) if dry_run_clone_target => SandboxSpec::Local {
working_directory: dry_run_workspace_for_target(persisted).await?,
},
Some(BundledProvider::Local) if dry_run_clone_target => {
SandboxSpec::local(dry_run_workspace_for_target(persisted).await?, access)
}
Some(BundledProvider::Local) => match record.target.as_ref() {
Some(target @ (RunTarget::Git(_) | RunTarget::None {})) => {
return Err(Error::engine(format!(
@ -517,44 +523,39 @@ impl RunSession {
target.kind_name()
)));
}
Some(RunTarget::Folder { path }) => SandboxSpec::Local {
working_directory: folder_working_directory_from_record(record, path).await?,
},
Some(RunTarget::Folder { path }) => SandboxSpec::local(
folder_working_directory_from_record(record, path).await?,
access,
),
None => {
let working_directory = local_working_directory_from_environment(
&resolved.environment,
record.source_directory.as_deref().map(Path::new),
)
.map_err(|err| {
Error::engine_with_source(
"Failed to resolve local environment working directory",
err,
)
})?;
SandboxSpec::Local { working_directory }
let working_directory = resolved
.environment
.local_working_directory(record.source_directory.as_deref().map(Path::new))
.map_err(|err| {
Error::engine_with_source(
"Failed to resolve local environment working directory",
err,
)
})?;
SandboxSpec::local(working_directory, access)
}
},
_ => {
let daytona = vault_guard.get(EnvVars::DAYTONA_API_KEY).map(|api_key| {
DaytonaCredentials::from_api_key(api_key.to_string(), process_env_var)
});
let access = ProviderAccess {
providers: services.sandbox_providers.clone(),
daytona,
};
let mut options = resolve_sandbox_options(resolved, secret_lookup)?;
options.skip_clone |= clone_source.skip_clone;
SandboxSpec::Provider(Box::new(ProviderSandboxSpec {
let spec = resolve_sandbox_spec(resolved, secret_lookup)?;
let mut clone = CloneRequest::from_settings(&resolved.clone);
clone.skip |= clone_source.skip_clone;
clone.origin_url = clone_source.origin_url;
clone.branch = clone_source.branch;
clone.tag = clone_source.tag;
clone.commit_sha = clone_source.commit_sha;
SandboxSpec {
kind: sandbox_provider.clone(),
access,
options,
spec,
clone,
github_app: services.github_app.clone(),
run_id: Some(record.run_id),
clone_origin_url: clone_source.origin_url,
clone_branch: clone_source.branch,
clone_tag: clone_source.tag,
clone_commit_sha: clone_source.commit_sha,
}))
}
}
};
@ -802,20 +803,20 @@ fn resolve_sandbox_provider(settings: &ResolvedRunSettings) -> SandboxProviderKi
settings.environment.provider.clone()
}
/// The environment's sandbox options with its variables resolved through
/// the vault.
fn resolve_sandbox_options(
/// The environment's sandbox spec with its variables resolved through the
/// vault.
fn resolve_sandbox_spec(
settings: &ResolvedRunSettings,
secrets_lookup: impl FnMut(&str) -> Option<String>,
) -> Result<SandboxOptions, Error> {
) -> Result<sandbox_driver::SandboxSpec, Error> {
let env = settings
.environment
.resolve_env(secrets_lookup)
.map_err(|err| Error::engine_with_source("failed to resolve environment variables", err))?
.into_iter()
.collect();
options_from_environment(&settings.environment, &settings.clone, env)
.map_err(|err| Error::engine_with_source("failed to resolve sandbox options", err))
sandbox_spec_for_environment(&settings.environment, env)
.map_err(|err| Error::engine_with_source("failed to resolve sandbox spec", err))
}
fn resolve_start_llm(
@ -1525,9 +1526,9 @@ mod tests {
..RunLayer::default()
});
let options = resolve_sandbox_options(&settings.run, |_| None).unwrap();
assert!(options.skip_clone);
assert_eq!(options.clone_depth, Some(1));
let clone = CloneRequest::from_settings(&settings.run.clone);
assert!(clone.skip);
assert_eq!(clone.depth, Some(1));
}
#[test]
@ -1540,16 +1541,16 @@ mod tests {
..RunLayer::default()
});
let options = resolve_sandbox_options(&settings.run, |_| None).unwrap();
assert_eq!(options.clone_depth, None);
let clone = CloneRequest::from_settings(&settings.run.clone);
assert_eq!(clone.depth, None);
}
#[test]
fn clone_providers_default_to_depth_100() {
let settings = settings_from_run_layer(RunLayer::default());
let options = resolve_sandbox_options(&settings.run, |_| None).unwrap();
assert_eq!(options.clone_depth, Some(100));
let clone = CloneRequest::from_settings(&settings.run.clone);
assert_eq!(clone.depth, Some(100));
}
#[test]
@ -1867,29 +1868,19 @@ mod tests {
..
} = session;
let runtime = sandbox
.to_run_sandbox_instance(&MockSandbox::linux().sandbox(), fixtures::RUN_1)
.to_run_sandbox_instance(&MockSandbox::linux().sandbox())
.runtime;
assert_eq!(runtime.repo_cloned, Some(false));
assert_eq!(runtime.clone_origin_url, None);
assert_eq!(runtime.clone_branch, None);
assert_eq!(runtime.primary_repo_path, None);
assert_eq!(runtime.primary_repo_link, None);
let SandboxSpec::Provider(spec) = sandbox else {
panic!("none target should retain the selected Docker provider");
};
let ProviderSandboxSpec {
kind,
options,
clone_origin_url,
clone_branch,
clone_commit_sha,
..
} = *spec;
let SandboxSpec { kind, clone, .. } = sandbox;
assert_eq!(kind, SandboxProviderKind::DOCKER);
assert!(options.skip_clone);
assert_eq!(clone_origin_url, None);
assert_eq!(clone_branch, None);
assert_eq!(clone_commit_sha, None);
assert!(clone.skip);
assert_eq!(clone.origin_url, None);
assert_eq!(clone.branch, None);
assert_eq!(clone.commit_sha, None);
assert_eq!(sandbox_env.origin_url, None);
assert_eq!(pr_origin_url, None);
}
@ -1936,31 +1927,25 @@ mod tests {
..
} = session;
let runtime = sandbox
.to_run_sandbox_instance(&MockSandbox::linux().sandbox(), fixtures::RUN_1)
.to_run_sandbox_instance(&MockSandbox::linux().sandbox())
.runtime;
assert_eq!(runtime.repo_cloned, Some(false));
assert_eq!(runtime.clone_origin_url, None);
assert_eq!(runtime.clone_branch, None);
assert_eq!(runtime.primary_repo_path, None);
assert_eq!(runtime.primary_repo_link, None);
let SandboxSpec::Provider(spec) = sandbox else {
panic!("none target should retain the selected Daytona provider");
};
let ProviderSandboxSpec {
let SandboxSpec {
kind,
access,
options,
clone_origin_url,
clone_branch,
clone_commit_sha,
clone,
..
} = *spec;
} = sandbox;
assert_eq!(kind, SandboxProviderKind::DAYTONA);
assert!(access.daytona.is_some(), "the vault key reaches the spec");
assert!(options.skip_clone);
assert_eq!(clone_origin_url, None);
assert_eq!(clone_branch, None);
assert_eq!(clone_commit_sha, None);
assert!(clone.skip);
assert_eq!(clone.origin_url, None);
assert_eq!(clone.branch, None);
assert_eq!(clone.commit_sha, None);
assert_eq!(sandbox_env.origin_url, None);
assert_eq!(pr_origin_url, None);
}
@ -2033,12 +2018,20 @@ mod tests {
.await
.unwrap();
let SandboxSpec::Local { working_directory } = session.sandbox else {
panic!("clone target dry-run should execute in a Local scratch sandbox");
};
assert_eq!(
working_directory,
run_dir.join("dry-run-workspace").canonicalize().unwrap()
session.sandbox.kind,
SandboxProviderKind::LOCAL,
"clone target dry-run should execute in a Local scratch sandbox"
);
assert_eq!(
session.sandbox.working_directory().map(Path::new),
Some(
run_dir
.join("dry-run-workspace")
.canonicalize()
.unwrap()
.as_path()
)
);
assert_eq!(session.sandbox_env.origin_url, None);
assert_eq!(session.pr_origin_url, None);
@ -2147,11 +2140,14 @@ mod tests {
.await
.unwrap();
let SandboxSpec::Local { working_directory } = session.sandbox else {
panic!("folder target should retain the selected Local provider");
};
assert_eq!(working_directory, canonical_folder);
assert_ne!(working_directory, environment_cwd);
assert_eq!(
session.sandbox.kind,
SandboxProviderKind::LOCAL,
"folder target should retain the selected Local provider"
);
let working_directory = session.sandbox.working_directory().map(Path::new);
assert_eq!(working_directory, Some(canonical_folder.as_path()));
assert_ne!(working_directory, Some(environment_cwd.as_path()));
assert_eq!(session.sandbox_env.origin_url.as_deref(), Some(origin_url));
assert_eq!(session.pr_origin_url.as_deref(), Some(origin_url));
}
@ -2216,10 +2212,15 @@ mod tests {
.await
.unwrap();
let SandboxSpec::Local { working_directory } = session.sandbox else {
panic!("legacy Local run should retain the selected Local provider");
};
assert_eq!(working_directory, environment_cwd);
assert_eq!(
session.sandbox.kind,
SandboxProviderKind::LOCAL,
"legacy Local run should retain the selected Local provider"
);
assert_eq!(
session.sandbox.working_directory().map(Path::new),
Some(environment_cwd.as_path())
);
}
#[tokio::test]
@ -2338,17 +2339,21 @@ mod tests {
..RunLayer::default()
});
let options = resolve_sandbox_options(&settings.run, |_| None).unwrap();
let spec = resolve_sandbox_spec(&settings.run, |_| None).unwrap();
assert_eq!(options.image.as_deref(), Some("ubuntu:24.04"));
assert_eq!(options.cpu, Some(4));
assert_eq!(options.memory_bytes, Some(2_000_000_000));
assert!(matches!(
options.network,
fabro_sandbox::NetworkPolicy::Block
&spec.source,
fabro_sandbox::SandboxSource::Image { reference } if reference == "ubuntu:24.04"
));
assert_eq!(spec.resources.cpu_cores, Some(4));
assert_eq!(
options.env,
spec.resources.memory_mb,
Some(1908),
"2 GB rounds up to whole mebibytes"
);
assert!(matches!(spec.network, fabro_sandbox::NetworkPolicy::Block));
assert_eq!(
spec.env,
std::collections::BTreeMap::from([("NODE_ENV".to_string(), "test".to_string())])
);
}

40
lib/components/fabro-workflow/src/pipeline/execute/tests.rs
View file

@ -15,8 +15,8 @@ use fabro_auth::test_support as auth_test_support;
use fabro_graphviz::graph::{AttrValue, Edge, Graph, Node};
use fabro_hooks::HookSettings;
use fabro_interview::AutoApproveInterviewer;
use fabro_sandbox::test_support::MockSandbox;
use fabro_sandbox::{RunSandbox, SandboxSpec};
use fabro_sandbox::test_support::{MockSandbox, local_sandbox_id};
use fabro_sandbox::{ProviderAccess, RunSandbox, SandboxSpec};
use fabro_store::Database;
use fabro_types::settings::run::RunModelControls;
use fabro_types::{
@ -263,9 +263,10 @@ async fn execute_test_run_with_options(
run_store: run_store.into(),
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: std::env::current_dir().unwrap(),
},
sandbox: SandboxSpec::local(
std::env::current_dir().unwrap(),
ProviderAccess::default(),
),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -325,9 +326,10 @@ async fn execute_runs_start_to_exit_and_returns_final_context() {
run_store: run_store.into(),
dry_run: false,
emitter: test_emitter_arc("run-test"),
sandbox: SandboxSpec::Local {
working_directory: std::env::current_dir().unwrap(),
},
sandbox: SandboxSpec::local(
std::env::current_dir().unwrap(),
ProviderAccess::default(),
),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -412,10 +414,11 @@ async fn resumed_in_flight_node_starts_a_new_stage_execution() {
let run_store = test_run_store(&run_id).await;
seed_created_and_starting(&run_store, &run_options, &graph).await;
// Resume reconnects to the previously recorded sandbox.
let working_directory = std::env::current_dir().unwrap();
append_event(&run_store, &run_id, &Event::SandboxInitialized {
working_directory: std::env::current_dir().unwrap().display().to_string(),
working_directory: working_directory.display().to_string(),
provider: fabro_types::SandboxProviderKind::LOCAL,
id: "local".to_string(),
id: local_sandbox_id(&working_directory).await,
image: None,
snapshot: None,
repo_cloned: None,
@ -468,9 +471,10 @@ async fn resumed_in_flight_node_starts_a_new_stage_execution() {
run_store: run_store.into(),
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: std::env::current_dir().unwrap(),
},
sandbox: SandboxSpec::local(
std::env::current_dir().unwrap(),
ProviderAccess::default(),
),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -584,9 +588,7 @@ async fn run_with_lifecycle(
run_store: run_store.into(),
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: PathBuf::from(sandbox.working_directory()),
},
sandbox: SandboxSpec::local(sandbox.working_directory(), ProviderAccess::default()),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -943,10 +945,10 @@ async fn execute_reactivates_sandbox_after_a_stage_can_leave_it_stopped() {
.unwrap();
assert_eq!(outcome.status, StageOutcome::Succeeded);
assert_eq!(sandbox.stop_count(), 1);
assert!(sandbox.walk_files_was_called());
assert_eq!(sandbox.driver().stop_count(), 1);
assert!(sandbox.driver().scripted_search().walk_calls() > 0);
assert_eq!(
sandbox.start_count(),
sandbox.driver().start_count(),
1,
"the stopped sandbox is started again before the walk"
);

8
lib/components/fabro-workflow/src/pipeline/finalize.rs
View file

@ -1644,8 +1644,8 @@ mod tests {
.await
.unwrap();
assert_eq!(sandbox.stop_count(), 1);
assert_eq!(sandbox.delete_count(), 0);
assert_eq!(sandbox.driver().stop_count(), 1);
assert_eq!(sandbox.driver().delete_count(), 0);
}
#[tokio::test]
@ -1678,8 +1678,8 @@ mod tests {
.await
.unwrap();
assert_eq!(sandbox.stop_count(), 0);
assert_eq!(sandbox.delete_count(), 0);
assert_eq!(sandbox.driver().stop_count(), 0);
assert_eq!(sandbox.driver().delete_count(), 0);
}
#[tokio::test]

110
lib/components/fabro-workflow/src/pipeline/initialize.rs
View file

@ -10,20 +10,20 @@ use fabro_hooks::{HookContext, HookDecision, HookEvent, HookExecutionContext, Ho
use fabro_llm::credentials::{CredentialProvider, readiness};
use fabro_llm::lithos_catalog::Catalog;
use fabro_sandbox::{
DaytonaCredentials, GitSetupIntent, ProviderAccess, RunSandbox, SandboxSpec,
reconnect_for_run_with_events, shell_quote,
DaytonaCredentials, ExecResultExt, GitSetupIntent, ProviderAccess, RunSandbox,
reconnect_for_run,
};
use fabro_static::EnvVars;
use fabro_types::RunSandboxKind;
use fabro_util::time::elapsed_ms;
use fabro_vault::Vault;
use sandbox_driver::{CorrelationId, EventContext};
use sandbox_driver::{CorrelationId, EventContext, Git as _};
use tokio::runtime::Handle;
use tokio::sync::RwLock as AsyncRwLock;
use super::types::{InitOptions, Initialized, LlmSpec, Persisted, SandboxEnvSpec};
use crate::error::Error;
use crate::event::{Event, RunNoticeCode, RunNoticeLevel, SandboxEventBridge, SandboxLifecycle};
use crate::event::{DriverEventRecorder, Event, RunNoticeCode, RunNoticeLevel, SandboxLifecycle};
use crate::git::GitAuthor;
use crate::git_bridge;
use crate::handler::llm::{AgentAcpBackend, BackendRouter, PebbleBackend, routing};
@ -79,18 +79,15 @@ async fn configure_sandbox_git_identity(
sandbox: &RunSandbox,
author: &GitAuthor,
) -> Result<(), Error> {
let command = format!(
"git config --local user.name {} && git config --local user.email {}",
shell_quote(&author.name),
shell_quote(&author.email)
);
sandbox
.exec_command(&command, 10_000, None, None, None)
.await
.map_err(|err| Error::engine_with_source("Sandbox git identity setup failed", err))?
.into_result("git config user identity")
let git = sandbox
.git()
.map_err(|err| Error::engine_with_source("Sandbox git identity setup failed", err))?;
let repo = sandbox.working_directory();
for (key, value) in [("user.name", &author.name), ("user.email", &author.email)] {
git.config_set(repo, key, value)
.await
.map_err(|err| Error::engine_with_source("Sandbox git identity setup failed", err))?;
}
Ok(())
}
@ -371,7 +368,7 @@ pub async fn initialize(
.as_ref()
.and_then(|git| git.sha.clone());
if !is_resume
&& !matches!(options.sandbox, SandboxSpec::Local { .. })
&& !options.sandbox.kind.is_local()
&& matches!(
options
.run_options
@ -388,14 +385,12 @@ pub async fn initialize(
);
}
// The driver reports what it does to the run's sandbox; the bridge
// records the operations fabro keeps as run events.
// The driver reports what it does to the run's sandbox; every event is
// kept as a run event.
let provider_name = options.sandbox.provider_name();
let sandbox_events = EventContext::new(Arc::new(SandboxEventBridge::new(
Arc::clone(&options.emitter),
provider_name.clone(),
options.sandbox.image(),
)))
let sandbox_events = EventContext::new(Arc::new(DriverEventRecorder::new(Arc::clone(
&options.emitter,
))))
.correlation_id(CorrelationId::new(options.run_options.run_id.to_string()));
let attach_instance = if is_resume {
let record = options
@ -438,7 +433,7 @@ pub async fn initialize(
DaytonaCredentials::from_api_key(api_key.to_string(), process_env_var)
}),
};
let sandbox = reconnect_for_run_with_events(
let sandbox = reconnect_for_run(
&instance,
&access,
Some(options.run_options.run_id),
@ -465,10 +460,8 @@ pub async fn initialize(
});
if attach_existing {
// Resume needs the full provider health check. `activate()` is the
// lighter access-time operation used after a run is already active.
sandbox
.start()
.activate()
.await
.map_err(|e| Error::engine_with_source("Failed to start sandbox", e))?;
} else {
@ -492,12 +485,16 @@ pub async fn initialize(
error,
));
}
// A local sandbox's id is derived from its directory, which the
// record already names; it is not a name worth showing.
let name = Some(sandbox.sandbox_info())
.filter(|name| !name.is_empty() && !sandbox.kind().is_local());
options.emitter.emit(&Event::Sandbox {
event: SandboxLifecycle::Ready {
provider: provider_name.clone(),
provider: provider_name.clone(),
duration_ms: elapsed_ms(started),
name: Some(sandbox.sandbox_info()).filter(|name| !name.is_empty()),
url: sandbox.console_url().await,
name,
url: sandbox.console_url().await,
},
});
}
@ -522,9 +519,7 @@ pub async fn initialize(
}
if !attach_existing {
let run_sandbox = options
.sandbox
.to_run_sandbox_instance(&sandbox, options.run_options.run_id);
let run_sandbox = options.sandbox.to_run_sandbox_instance(&sandbox);
let runtime = &run_sandbox.runtime;
options.emitter.emit(&Event::SandboxInitialized {
working_directory: runtime.working_directory.clone(),
@ -672,19 +667,19 @@ pub async fn initialize(
}
cancel_token.cancel();
let duration_ms = crate::millis_u64(cmd_start.elapsed());
if !result.is_success() {
let exit_code = result.display_exit_code();
if !result.success() {
let exit_code = result.program_exit_code().unwrap_or(-1);
let exec_output_tail = result.default_redacted_output_tail();
let stderr = result.stderr_lossy();
options.emitter.emit(&Event::SetupFailed {
command: command.clone(),
index,
exit_code,
stderr: result.stderr.clone(),
stderr: stderr.clone(),
exec_output_tail,
});
return Err(Error::engine(format!(
"Setup command failed (exit code {}): {command}\n{}",
exit_code, result.stderr,
"Setup command failed (exit code {exit_code}): {command}\n{stderr}",
)));
}
let exit_code = result.exit_code.unwrap_or(0);
@ -941,7 +936,7 @@ mod tests {
run_store,
dry_run: false,
emitter: Arc::clone(&emitter),
sandbox: SandboxSpec::Local { working_directory },
sandbox: SandboxSpec::local(working_directory, ProviderAccess::default()),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -1072,11 +1067,18 @@ mod tests {
.await
.expect("git identity should configure");
let commands = sandbox.captured_commands();
assert_eq!(commands, vec![
"git config --local user.name 'Fabro Bot' && git config --local user.email \
fabro-bot@example.com"
]);
let commands = sandbox.driver().scripted_exec().commands();
assert_eq!(commands.len(), 2, "{commands:#?}");
assert!(
commands[0].contains("'config' '--local' '--' 'user.name' 'Fabro Bot'"),
"{}",
commands[0]
);
assert!(
commands[1].contains("'config' '--local' '--' 'user.email' 'fabro-bot@example.com'"),
"{}",
commands[1]
);
}
#[tokio::test]
@ -1353,7 +1355,7 @@ mod tests {
"acp.command".to_string(),
AttrValue::String(format!(
"python3 {}",
fabro_sandbox::shell_quote(&script_path.to_string_lossy())
fabro_util::shell::shell_quote(&script_path.to_string_lossy())
)),
);
let mut exit = Node::new("exit");
@ -1385,9 +1387,7 @@ mod tests {
run_store: run_store.into(),
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: temp.path().to_path_buf(),
},
sandbox: SandboxSpec::local(temp.path(), ProviderAccess::default()),
llm: LlmSpec {
model: "fake-acp".to_string(),
provider_id: lithos_llm::catalog::builtin::openai(),
@ -1490,9 +1490,10 @@ mod tests {
run_store: run_store.into(),
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: std::env::current_dir().unwrap(),
},
sandbox: SandboxSpec::local(
std::env::current_dir().unwrap(),
ProviderAccess::default(),
),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),
@ -1634,9 +1635,10 @@ mod tests {
},
dry_run: false,
emitter: emitter.clone(),
sandbox: SandboxSpec::Local {
working_directory: std::env::current_dir().unwrap(),
},
sandbox: SandboxSpec::local(
std::env::current_dir().unwrap(),
ProviderAccess::default(),
),
llm: LlmSpec {
model: "test-model".to_string(),
provider_id: lithos_llm::catalog::builtin::anthropic(),

28
lib/components/fabro-workflow/src/pipeline/publish.rs
View file

@ -100,9 +100,6 @@ fn push_attempt_cause(attempt: &fabro_sandbox::PushAttempt) -> String {
{
let _ = write!(line, " (token age {age_ms}ms)");
}
if let Some(refresh_error) = attempt.refresh_error {
let _ = write!(line, ", refresh error: {refresh_error}");
}
line
}
@ -232,8 +229,8 @@ impl Concluded {
async fn push_final_commit(&self, run_branch: &str) -> Result<(), Error> {
// The terminal push guards the whole run's value, so it gets a real
// retry budget; attempts are nearly free at this point.
let plan = fabro_sandbox::RetryPlan::publish_push();
match push_run_branch(self.services.sandbox.as_ref(), run_branch, &plan).await {
let policy = fabro_sandbox::publish_push_policy();
match push_run_branch(self.services.sandbox.as_ref(), run_branch, &policy).await {
Ok(report) => {
self.services.sandbox_git.record_successful_push();
self.services.emitter.emit(&Event::GitPush {
@ -285,7 +282,6 @@ mod tests {
attempt: u32,
retry_reason: Option<fabro_sandbox::GitRetryReason>,
token_age_ms: Option<u64>,
refresh_error: Option<fabro_sandbox::RefreshErrorKind>,
) -> fabro_sandbox::PushAttempt {
let started_at = Utc::now();
fabro_sandbox::PushAttempt {
@ -302,8 +298,6 @@ mod tests {
expires_at: started_at + chrono::Duration::hours(1),
},
}),
credential_action: Some(fabro_sandbox::RemoteCredentialAction::Unchanged),
refresh_error,
}
}
@ -314,14 +308,12 @@ mod tests {
.iter()
.enumerate()
.map(|(index, reason)| fabro_sandbox::PushAttempt {
attempt: u32::try_from(index).unwrap() + 1,
started_at: Utc::now(),
success: false,
retry_reason: *reason,
exec_output_tail: None,
token: None,
credential_action: None,
refresh_error: None,
attempt: u32::try_from(index).unwrap() + 1,
started_at: Utc::now(),
success: false,
retry_reason: *reason,
exec_output_tail: None,
token: None,
})
.collect()
}
@ -362,13 +354,11 @@ mod tests {
1,
Some(fabro_sandbox::GitRetryReason::TokenReplication),
Some(180),
None,
),
push_attempt(
2,
Some(fabro_sandbox::GitRetryReason::TokenReplication),
Some(3320),
Some(fabro_sandbox::RefreshErrorKind::SetUrl),
),
];
let last_push = Utc::now() - chrono::Duration::seconds(67);
@ -401,7 +391,7 @@ mod tests {
"{attempt_lines:?}"
);
assert!(
attempt_lines[1].contains("refresh error: set_url"),
attempt_lines[1].contains("(token age 3320ms)"),
"{attempt_lines:?}"
);
assert_eq!(

3
lib/components/fabro-workflow/src/run_metadata.rs
View file

@ -24,8 +24,7 @@ pub(crate) fn metadata_push_failure_is_transient(
detail: &str,
token: Option<&TokenSnapshot>,
) -> bool {
let credentials = fabro_sandbox::CredentialContext::from_snapshot(token);
fabro_sandbox::classify_failure(detail, credentials).is_some()
fabro_sandbox::transient_git_failure(detail, token).is_some()
}
#[derive(Debug, thiserror::Error)]

1026
lib/components/fabro-workflow/src/sandbox_git.rs
View file

File diff suppressed because it is too large Load diff

35
lib/components/fabro-workflow/src/sandbox_git_runtime.rs
View file

@ -1,8 +1,9 @@
use fabro_sandbox::{RunSandbox, shell_quote};
use fabro_sandbox::{ExecResult, ExecResultExt, RunSandbox, Termination};
use fabro_util::error::SharedError;
use fabro_util::shell;
use tokio::sync::OnceCell;
use crate::sandbox_git::{GIT_REMOTE, exec_err};
use crate::sandbox_git::GitCommandError;
pub(crate) struct SandboxGitRuntime {
probe: OnceCell<Result<(), SharedError>>,
@ -65,10 +66,10 @@ async fn probe_sandbox_git(sandbox: &RunSandbox) -> Result<(), SharedError> {
GIT_INDEX_FILE={index_q} {git} update-index --add --cacheinfo 100644,$blob,probe.txt\n\
GIT_INDEX_FILE={index_q} {git} write-tree >/dev/null\n\
rm -rf {temp_q}",
temp_q = shell_quote(&temp),
probe_file_q = shell_quote(&probe_file),
index_q = shell_quote(&index),
git = GIT_REMOTE,
temp_q = shell::shell_quote(&temp),
probe_file_q = shell::shell_quote(&probe_file),
index_q = shell::shell_quote(&index),
git = "git -c maintenance.auto=0 -c gc.auto=0",
);
exec_ok(sandbox, &command).await
}
@ -86,7 +87,7 @@ async fn exec_ok(sandbox: &RunSandbox, command: &str) -> Result<(), SharedError>
.map_err(|err| {
SharedError::new(anyhow::Error::new(err).context("sandbox git probe command failed"))
})?;
if result.is_success() {
if result.success() {
Ok(())
} else {
Err(SharedError::new(anyhow::Error::new(exec_err(
@ -94,3 +95,23 @@ async fn exec_ok(sandbox: &RunSandbox, command: &str) -> Result<(), SharedError>
))))
}
}
/// The probe's failure, named by how the command ended; the output tail
/// travels in the source.
fn exec_err(label: &str, result: ExecResult) -> GitCommandError {
let duration_ms = result.duration_ms();
let message = match result.termination {
Termination::TimedOut => format!("{label} timed out after {duration_ms}ms"),
Termination::Cancelled | Termination::Killed => {
format!("{label} cancelled after {duration_ms}ms")
}
_ => format!(
"{label} failed (exit {})",
result.program_exit_code().unwrap_or(-1)
),
};
GitCommandError {
message,
source: result.into_exec_error(label),
}
}

41
lib/components/fabro-workflow/tests/it/cp_integration.rs
View file

@ -14,9 +14,11 @@
reason = "This integration test stages sandbox fixtures with sync std::fs."
)]
use fabro_sandbox::reconnect::reconnect;
use fabro_sandbox::{ProviderAccess, SandboxOptions, provider_sandbox};
use fabro_sandbox::reconnect::reconnect_for_run;
use fabro_sandbox::test_support::local_sandbox_id;
use fabro_sandbox::{CloneRequest, ProviderAccess, provider_sandbox};
use fabro_types::{RunSandboxInstance, RunSandboxRuntime, SandboxProviderKind};
use sandbox_driver::{SandboxSource, SandboxSpec};
const DOCKER_CP_IMAGE: &str = "buildpack-deps:noble";
@ -24,13 +26,13 @@ const DOCKER_CP_IMAGE: &str = "buildpack-deps:noble";
// Local sandbox
// ---------------------------------------------------------------------------
fn local_record(working_directory: &std::path::Path) -> RunSandboxInstance {
async fn local_record(working_directory: &std::path::Path) -> RunSandboxInstance {
RunSandboxInstance {
provider: SandboxProviderKind::LOCAL,
image: None,
snapshot: None,
runtime: RunSandboxRuntime {
id: "local:test".to_string(),
id: local_sandbox_id(working_directory).await,
working_directory: working_directory.to_string_lossy().to_string(),
repo_cloned: None,
clone_origin_url: None,
@ -48,8 +50,8 @@ async fn local_cp_upload_download_round_trip() {
let sandbox_dir = tempfile::tempdir().unwrap();
let scratch = tempfile::tempdir().unwrap();
let record = local_record(sandbox_dir.path());
let sandbox = reconnect(&record, &ProviderAccess::default())
let record = local_record(sandbox_dir.path()).await;
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect local");
@ -81,8 +83,8 @@ async fn local_cp_binary_round_trip() {
let sandbox_dir = tempfile::tempdir().unwrap();
let scratch = tempfile::tempdir().unwrap();
let record = local_record(sandbox_dir.path());
let sandbox = reconnect(&record, &ProviderAccess::default())
let record = local_record(sandbox_dir.path()).await;
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect local");
@ -110,8 +112,8 @@ async fn local_cp_creates_parent_dirs() {
let sandbox_dir = tempfile::tempdir().unwrap();
let scratch = tempfile::tempdir().unwrap();
let record = local_record(sandbox_dir.path());
let sandbox = reconnect(&record, &ProviderAccess::default())
let record = local_record(sandbox_dir.path()).await;
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect local");
@ -187,15 +189,10 @@ async fn docker_cp_container() -> DockerCpContainer {
let sandbox = provider_sandbox(
SandboxProviderKind::DOCKER,
&ProviderAccess::default(),
SandboxOptions {
image: Some(DOCKER_CP_IMAGE.to_string()),
skip_clone: true,
..SandboxOptions::default()
},
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::Image {
reference: DOCKER_CP_IMAGE.to_string(),
}),
&CloneRequest::none(),
None,
None,
)
@ -241,7 +238,7 @@ async fn docker_cp_upload_download_round_trip() {
let scratch = tempfile::tempdir().unwrap();
let record = docker_record(&container.id);
let sandbox = reconnect(&record, &ProviderAccess::default())
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect docker");
@ -272,7 +269,7 @@ async fn docker_cp_binary_round_trip() {
let scratch = tempfile::tempdir().unwrap();
let record = docker_record(&container.id);
let sandbox = reconnect(&record, &ProviderAccess::default())
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect docker");
@ -301,7 +298,7 @@ async fn docker_cp_creates_parent_dirs() {
let scratch = tempfile::tempdir().unwrap();
let record = docker_record(&container.id);
let sandbox = reconnect(&record, &ProviderAccess::default())
let sandbox = reconnect_for_run(&record, &ProviderAccess::default(), None, None)
.await
.expect("reconnect docker");

197
lib/components/fabro-workflow/tests/it/daytona_integration.rs
View file

@ -24,7 +24,7 @@ use std::sync::Arc;
use fabro_graphviz::graph::{AttrValue, Edge, Graph, Node};
use fabro_sandbox::{
DaytonaCredentials, ProviderAccess, RunSandbox, SandboxOptions, SandboxProviderKind,
CloneRequest, DaytonaCredentials, ProviderAccess, RunSandbox, SandboxProviderKind,
provider_sandbox,
};
use fabro_static::EnvVars;
@ -43,6 +43,7 @@ use fabro_workflow::run_options::{GitCheckpointOptions, RunOptions};
use fabro_workflow::runtime_store::RunStoreHandle;
use fabro_workflow::test_support::{WorkflowRunner, test_store_dir};
use object_store::local::LocalFileSystem;
use sandbox_driver::{LifecycleTimers, Resources, SandboxSource, SandboxSpec};
use tokio_util::sync::CancellationToken;
use ulid::Ulid;
@ -192,16 +193,8 @@ fn daytona_access(credentials: DaytonaCredentials) -> ProviderAccess {
}
fn live_daytona_credentials() -> DaytonaCredentials {
DaytonaCredentials {
api_key: std::env::var(EnvVars::DAYTONA_API_KEY)
.expect("DAYTONA_API_KEY must be set"),
api_url: std::env::var(EnvVars::DAYTONA_API_URL)
.or_else(|_| std::env::var(EnvVars::DAYTONA_SERVER_URL))
.ok(),
organization_id: std::env::var(EnvVars::DAYTONA_ORGANIZATION_ID).ok(),
target: None,
http_client: None,
}
let api_key = std::env::var(EnvVars::DAYTONA_API_KEY).expect("DAYTONA_API_KEY must be set");
DaytonaCredentials::from_api_key(api_key, |name| std::env::var(name).ok())
}
async fn create_env() -> RunSandbox {
@ -223,13 +216,10 @@ async fn create_env_with_github_app(
provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_daytona_credentials()),
SandboxOptions::default(),
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::default(),
github_app.as_ref(),
None,
None,
None,
None,
None,
)
.await
.expect("Failed to create Daytona client — is DAYTONA_API_KEY set?")
@ -285,9 +275,9 @@ async fn daytona_exec_command() {
.await
.unwrap();
assert_eq!(result.exit_code, Some(0));
assert!(result.stdout.contains("hello"));
assert!(result.stdout_lossy().contains("hello"));
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -301,9 +291,9 @@ async fn daytona_exec_command_with_pipe() {
.await
.unwrap();
assert_eq!(result.exit_code, Some(0));
assert!(result.stdout.trim().contains('2'));
assert!(result.stdout_lossy().trim().contains('2'));
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -328,10 +318,13 @@ async fn daytona_exec_command_cancelled() {
.unwrap();
assert_eq!(result.exit_code, None);
assert!(result.is_cancelled());
assert_eq!(result.stderr, "Command cancelled");
assert!(matches!(
result.termination,
fabro_sandbox::Termination::Cancelled | fabro_sandbox::Termination::Killed
));
assert_eq!(result.stderr_lossy(), "Command cancelled");
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -361,10 +354,10 @@ async fn daytona_exec_command_local_timeout() {
"Command stalled for longer than the local timeout mechanism"
);
assert_eq!(result.exit_code, None);
assert!(result.is_timed_out());
assert_eq!(result.stderr, "Command timed out locally");
assert_eq!(result.termination, fabro_sandbox::Termination::TimedOut);
assert_eq!(result.stderr_lossy(), "Command timed out locally");
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -389,7 +382,7 @@ async fn daytona_file_round_trip() {
env.delete_file(test_path).await.unwrap();
assert!(!env.file_exists(test_path).await.unwrap());
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -414,33 +407,31 @@ async fn daytona_full_lifecycle() {
assert!(!entries.is_empty());
// Cleanup (deletes sandbox)
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
async fn daytona_snapshot_sandbox() {
let options = SandboxOptions {
auto_stop: Some(std::time::Duration::from_hours(1)),
dockerfile: Some(
"FROM ubuntu:22.04\nRUN apt-get update && apt-get install -y ripgrep".to_string(),
),
cpu: Some(2),
memory_bytes: Some(4_000_000_000),
disk_bytes: Some(10_000_000_000),
..SandboxOptions::default()
};
let mut resources = Resources::default();
resources.cpu_cores = Some(2);
resources.memory_mb = Some(4096);
resources.disk_mb = Some(10_240);
let mut timers = LifecycleTimers::default();
timers.auto_stop_after_idle = Some(std::time::Duration::from_hours(1));
let spec = SandboxSpec::new(SandboxSource::Dockerfile {
content: "FROM ubuntu:22.04\nRUN apt-get update && apt-get install -y ripgrep".to_string(),
})
.resources(resources)
.timers(timers);
let creds = load_github_app_credentials();
let env = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_daytona_credentials()),
options,
spec,
&CloneRequest::default(),
Some(&creds),
None,
None,
None,
None,
None,
)
.await
.expect("Failed to create Daytona client — is DAYTONA_API_KEY set?");
@ -452,9 +443,9 @@ async fn daytona_snapshot_sandbox() {
.await
.unwrap();
assert_eq!(result.exit_code, Some(0));
assert!(result.stdout.contains("ripgrep"));
assert!(result.stdout_lossy().contains("ripgrep"));
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -506,7 +497,7 @@ async fn daytona_artifact_sync_uploads_and_rewrites_pointer() {
remote_content.len()
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
// ---------------------------------------------------------------------------
@ -620,7 +611,7 @@ async fn daytona_pipeline_artifact_offload_and_sync() {
"offloaded value should round-trip through the run store"
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
// ---------------------------------------------------------------------------
@ -667,9 +658,9 @@ async fn setup_daytona_git(sandbox: &RunSandbox) -> (RunId, String, String) {
sha_result.exit_code,
Some(0),
"git rev-parse HEAD failed: {}",
sha_result.stderr
sha_result.stderr_lossy()
);
let base_sha = sha_result.stdout.trim().to_string();
let base_sha = sha_result.stdout_lossy().trim().to_string();
let run_id = RunId::from(Ulid::new());
let branch_name = format!("fabro/run/{run_id}");
@ -684,8 +675,8 @@ async fn setup_daytona_git(sandbox: &RunSandbox) -> (RunId, String, String) {
Some(0),
"git checkout -b failed (exit {:?}): stdout={} stderr={}",
checkout_result.exit_code,
checkout_result.stdout,
checkout_result.stderr
checkout_result.stdout_lossy(),
checkout_result.stderr_lossy()
);
(run_id, base_sha, branch_name)
@ -701,7 +692,7 @@ async fn daytona_git_checkpoint_remote_emits_events() {
let git_check = env
.exec_command("git --version", 10_000, None, None, None)
.await;
if git_check.as_ref().map_or(true, |r| !r.is_success()) {
if git_check.as_ref().map_or(true, |r| !r.success()) {
let install = env
.exec_command(
"apt-get update -qq && apt-get install -y -qq git >/dev/null 2>&1",
@ -716,7 +707,7 @@ async fn daytona_git_checkpoint_remote_emits_events() {
install.exit_code,
Some(0),
"git install failed: {}",
install.stderr
install.stderr_lossy()
);
}
@ -831,7 +822,7 @@ async fn daytona_git_checkpoint_remote_emits_events() {
"checkpoint should have git_commit_sha"
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
// ---------------------------------------------------------------------------
@ -851,7 +842,7 @@ async fn daytona_git_checkpoint_with_shadow_branch() {
let git_check = env
.exec_command("git --version", 10_000, None, None, None)
.await;
if git_check.as_ref().map_or(true, |r| !r.is_success()) {
if git_check.as_ref().map_or(true, |r| !r.success()) {
let install = env
.exec_command(
"apt-get update -qq && apt-get install -y -qq git >/dev/null 2>&1",
@ -866,7 +857,7 @@ async fn daytona_git_checkpoint_with_shadow_branch() {
install.exit_code,
Some(0),
"git install failed: {}",
install.stderr
install.stderr_lossy()
);
}
@ -947,9 +938,9 @@ async fn daytona_git_checkpoint_with_shadow_branch() {
)
.await
.expect("git show should succeed");
assert_eq!(run_json.exit_code, Some(0), "{}", run_json.stderr);
assert_eq!(run_json.exit_code, Some(0), "{}", run_json.stderr_lossy());
let projection: fabro_store::RunProjection =
serde_json::from_slice(run_json.stdout.as_bytes()).expect("run.json should parse");
serde_json::from_slice(run_json.stdout_lossy().as_bytes()).expect("run.json should parse");
let checkpoint = projection
.current_checkpoint()
.cloned()
@ -969,7 +960,7 @@ async fn daytona_git_checkpoint_with_shadow_branch() {
.await
.expect("git log should succeed");
assert_eq!(log_result.exit_code, Some(0));
let commit_msg = log_result.stdout.trim().to_string();
let commit_msg = log_result.stdout_lossy().trim().to_string();
assert!(
commit_msg.contains("Fabro-Checkpoint:"),
"sandbox commit should have Fabro-Checkpoint trailer, got:\n{commit_msg}"
@ -979,7 +970,7 @@ async fn daytona_git_checkpoint_with_shadow_branch() {
"sandbox commit should have Fabro-Run trailer, got:\n{commit_msg}"
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
// ---------------------------------------------------------------------------
@ -1112,7 +1103,7 @@ async fn daytona_asset_collection() {
"artifact scratch cache should not be created"
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -1131,7 +1122,7 @@ async fn daytona_ssh_access() {
"ssh_command should contain 'ssh': {ssh_command}",
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
@ -1173,7 +1164,7 @@ async fn daytona_clone_private_repo_with_github_app_iat() {
"CLAUDE.md should exist after clone"
);
assert!(
result.stdout.contains("EXISTS"),
result.stdout_lossy().contains("EXISTS"),
"clone should have populated the workspace"
);
@ -1181,7 +1172,7 @@ async fn daytona_clone_private_repo_with_github_app_iat() {
let git_check = env
.exec_command("git --version", 10_000, None, None, None)
.await;
if git_check.as_ref().map_or(true, |r| !r.is_success()) {
if git_check.as_ref().map_or(true, |r| !r.success()) {
let install = env
.exec_command(
"apt-get update -qq && apt-get install -y -qq git >/dev/null 2>&1",
@ -1196,7 +1187,7 @@ async fn daytona_clone_private_repo_with_github_app_iat() {
install.exit_code,
Some(0),
"git install failed: {}",
install.stderr
install.stderr_lossy()
);
}
@ -1207,12 +1198,12 @@ async fn daytona_clone_private_repo_with_github_app_iat() {
.unwrap();
assert_eq!(result.exit_code, Some(0));
assert!(
result.stdout.contains("fabro-sh/fabro"),
result.stdout_lossy().contains("fabro-sh/fabro"),
"origin should point to fabro-sh/fabro, got: {}",
result.stdout.trim()
result.stdout_lossy().trim()
);
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
/// E2E: Verify that repos in an installed org get credentials (needed for
@ -1284,7 +1275,7 @@ async fn daytona_git_push_run_branch_to_origin() {
let git_check = env
.exec_command("git --version", 10_000, None, None, None)
.await;
if git_check.as_ref().map_or(true, |r| !r.is_success()) {
if git_check.as_ref().map_or(true, |r| !r.success()) {
let install = env
.exec_command(
"apt-get update -qq && apt-get install -y -qq git >/dev/null 2>&1",
@ -1299,7 +1290,7 @@ async fn daytona_git_push_run_branch_to_origin() {
install.exit_code,
Some(0),
"git install failed: {}",
install.stderr
install.stderr_lossy()
);
}
@ -1376,12 +1367,12 @@ async fn daytona_git_push_run_branch_to_origin() {
ls_result.exit_code,
Some(0),
"git ls-remote failed: {}",
ls_result.stdout
ls_result.stdout_lossy()
);
assert!(
ls_result.stdout.contains(&branch_name),
ls_result.stdout_lossy().contains(&branch_name),
"run branch should exist on origin after push, got: {}",
ls_result.stdout.trim()
ls_result.stdout_lossy().trim()
);
// Clean up the remote branch
@ -1390,15 +1381,15 @@ async fn daytona_git_push_run_branch_to_origin() {
.exec_command(&delete_cmd, 30_000, None, None, None)
.await;
if let Ok(r) = &delete_result {
if !r.is_success() {
if !r.success() {
eprintln!(
"Warning: failed to delete remote branch {branch_name}: {}",
r.stdout
r.stdout_lossy()
);
}
}
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
/// Diagnose toolbox proxy staleness after idle time.
@ -1445,7 +1436,7 @@ async fn daytona_toolbox_idle_diagnostic() {
eprintln!(
"[t=+{sleep_secs}s] OK exit_code={:?} stdout={}",
r.exit_code,
r.stdout.trim()
r.stdout_lossy().trim()
);
}
Err(e) => {
@ -1536,7 +1527,7 @@ async fn daytona_toolbox_idle_diagnostic() {
}
eprintln!("\n=== PASS: all idle durations survived ===");
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
/// E2E test for `fabro cp` against a live Daytona sandbox.
@ -1545,7 +1536,7 @@ async fn daytona_toolbox_idle_diagnostic() {
/// uploads a file, downloads it back, and verifies the round-trip.
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"), live("GITHUB_APP_PRIVATE_KEY"))]
async fn daytona_cp_upload_download_round_trip() {
use fabro_sandbox::reconnect::reconnect;
use fabro_sandbox::reconnect::reconnect_for_run;
use fabro_types::RunSandboxInstance;
// 1. Create and initialize a real Daytona sandbox
@ -1582,7 +1573,7 @@ async fn daytona_cp_upload_download_round_trip() {
daytona: Some(live_daytona_credentials()),
..ProviderAccess::default()
};
let reconnected = reconnect(&record, &access)
let reconnected = reconnect_for_run(&record, &access, None, None)
.await
.expect("reconnect should succeed");
@ -1640,23 +1631,16 @@ async fn daytona_cp_upload_download_round_trip() {
);
// 9. Cleanup
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"))]
async fn daytona_computer_use_browser_screenshot() {
let options = SandboxOptions {
skip_clone: true,
..SandboxOptions::default()
};
let env = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_daytona_credentials()),
options,
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::none(),
None,
None,
)
@ -1686,9 +1670,9 @@ async fn daytona_computer_use_browser_screenshot() {
)
.await
.unwrap();
eprintln!("Browser check: {}", check.stdout.trim());
eprintln!("Browser check: {}", check.stdout_lossy().trim());
if check.stdout.trim() == "NONE" {
if check.stdout_lossy().trim() == "NONE" {
let install_result = env
.exec_command(
"apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq chromium 2>&1",
@ -1699,7 +1683,7 @@ async fn daytona_computer_use_browser_screenshot() {
eprintln!(
"Browser install exit_code={:?}, last_line={}",
install_result.exit_code,
install_result.stdout.lines().last().unwrap_or("")
install_result.stdout_lossy().lines().last().unwrap_or("")
);
assert_eq!(install_result.exit_code, Some(0), "Chromium install failed");
}
@ -1714,7 +1698,7 @@ async fn daytona_computer_use_browser_screenshot() {
)
.await
.unwrap();
let browser = browser_bin.stdout.trim().to_string();
let browser = browser_bin.stdout_lossy().trim().to_string();
eprintln!("Using browser: {browser}");
// 3. Detect the DISPLAY that computer use started
@ -1728,7 +1712,7 @@ async fn daytona_computer_use_browser_screenshot() {
)
.await
.unwrap();
eprintln!("Xvfb process: {}", display_check.stdout.trim());
eprintln!("Xvfb process: {}", display_check.stdout_lossy().trim());
// 4. Launch browser with setsid to fully detach, and log stderr
let launch_cmd = format!(
@ -1756,7 +1740,7 @@ async fn daytona_computer_use_browser_screenshot() {
)
.await
.unwrap();
eprintln!("Chrome processes:\n{}", ps_check.stdout);
eprintln!("Chrome processes:\n{}", ps_check.stdout_lossy());
let stderr_check = env
.exec_command(
@ -1768,7 +1752,7 @@ async fn daytona_computer_use_browser_screenshot() {
)
.await
.unwrap();
eprintln!("Chrome stderr:\n{}", stderr_check.stdout);
eprintln!("Chrome stderr:\n{}", stderr_check.stdout_lossy());
// 5. The desktop is serving: noVNC listens on its port.
let listening = env
@ -1782,29 +1766,22 @@ async fn daytona_computer_use_browser_screenshot() {
.await
.unwrap();
assert!(
listening.is_success(),
listening.success(),
"noVNC should be reachable inside the sandbox"
);
// 7. Cleanup
env.cleanup().await.unwrap();
env.delete().await.unwrap();
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"))]
async fn daytona_playwright_mcp_sandbox_transport() {
// Create sandbox from daytona-medium (has Node.js + Chromium)
let options = SandboxOptions {
skip_clone: true,
..SandboxOptions::default()
};
let sandbox = provider_sandbox(
SandboxProviderKind::DAYTONA,
&daytona_access(live_daytona_credentials()),
options,
None,
None,
None,
None,
SandboxSpec::new(SandboxSource::HostDirectory),
&CloneRequest::none(),
None,
None,
)
@ -1828,7 +1805,7 @@ async fn daytona_playwright_mcp_sandbox_transport() {
"Install exit_code={:?}, last_lines:\n{}",
install.exit_code,
install
.stdout
.stdout_lossy()
.lines()
.rev()
.take(5)
@ -1965,5 +1942,5 @@ async fn daytona_playwright_mcp_sandbox_transport() {
.expect("the agent shuts down");
// 8. Cleanup
sandbox.cleanup().await.unwrap();
sandbox.delete().await.unwrap();
}

13
lib/components/fabro-workflow/tests/it/integration.rs
View file

@ -13625,19 +13625,12 @@ async fn asset_collection_local_sandbox_on_failure() {
async fn asset_collection_docker_sandbox() {
let run_dir = tempfile::tempdir().unwrap();
let options = fabro_sandbox::SandboxOptions {
skip_clone: true,
..Default::default()
};
let sandbox: Arc<fabro_sandbox::RunSandbox> = Arc::new(
fabro_sandbox::provider_sandbox(
fabro_sandbox::SandboxProviderKind::DOCKER,
&fabro_sandbox::ProviderAccess::default(),
options,
None,
None,
None,
None,
sandbox_driver::SandboxSpec::new(sandbox_driver::SandboxSource::HostDirectory),
&fabro_sandbox::CloneRequest::none(),
None,
None,
)
@ -13734,7 +13727,7 @@ async fn asset_collection_docker_sandbox() {
"artifact scratch cache should not be created"
);
sandbox.cleanup().await.unwrap();
sandbox.delete().await.unwrap();
}
#[tokio::test]

39
lib/components/fabro-workflow/tests/it/pebble_agent.rs
View file

@ -1269,14 +1269,8 @@ async fn docker_sandbox_runs_an_agent_stage() {
fabro_sandbox::provider_sandbox(
fabro_sandbox::SandboxProviderKind::DOCKER,
&fabro_sandbox::ProviderAccess::default(),
fabro_sandbox::SandboxOptions {
skip_clone: true,
..Default::default()
},
None,
None,
None,
None,
sandbox_driver::SandboxSpec::new(sandbox_driver::SandboxSource::HostDirectory),
&fabro_sandbox::CloneRequest::none(),
None,
None,
)
@ -1287,7 +1281,7 @@ async fn docker_sandbox_runs_an_agent_stage() {
agent_stage_smoke(Arc::clone(&sandbox), "docker").await;
sandbox.cleanup().await.expect("Docker cleanup failed");
sandbox.delete().await.expect("Docker cleanup failed");
}
#[fabro_macros::e2e_test(live("DAYTONA_API_KEY"))]
@ -1298,31 +1292,20 @@ async fn docker_sandbox_runs_an_agent_stage() {
async fn daytona_sandbox_runs_an_agent_stage() {
use fabro_static::EnvVars;
let api_key = std::env::var(EnvVars::DAYTONA_API_KEY).expect("DAYTONA_API_KEY must be set");
let access = fabro_sandbox::ProviderAccess {
daytona: Some(fabro_sandbox::DaytonaCredentials {
api_key: std::env::var(EnvVars::DAYTONA_API_KEY)
.expect("DAYTONA_API_KEY must be set"),
api_url: std::env::var(EnvVars::DAYTONA_API_URL)
.or_else(|_| std::env::var(EnvVars::DAYTONA_SERVER_URL))
.ok(),
organization_id: std::env::var(EnvVars::DAYTONA_ORGANIZATION_ID).ok(),
target: None,
http_client: None,
}),
daytona: Some(fabro_sandbox::DaytonaCredentials::from_api_key(
api_key,
|name| std::env::var(name).ok(),
)),
..fabro_sandbox::ProviderAccess::default()
};
let sandbox: Arc<RunSandbox> = Arc::new(
fabro_sandbox::provider_sandbox(
fabro_sandbox::SandboxProviderKind::DAYTONA,
&access,
fabro_sandbox::SandboxOptions {
skip_clone: true,
..Default::default()
},
None,
None,
None,
None,
sandbox_driver::SandboxSpec::new(sandbox_driver::SandboxSource::HostDirectory),
&fabro_sandbox::CloneRequest::none(),
None,
None,
)
@ -1333,5 +1316,5 @@ async fn daytona_sandbox_runs_an_agent_stage() {
agent_stage_smoke(Arc::clone(&sandbox), "daytona").await;
sandbox.cleanup().await.expect("Daytona cleanup failed");
sandbox.delete().await.expect("Daytona cleanup failed");
}

1
lib/foundation/fabro-api/Cargo.toml
View file

@ -23,6 +23,7 @@ lithos-llm = { workspace = true, features = ["runtime"] }
progenitor-client = "0.13"
regress = "0.10"
reqwest.workspace = true
sandbox-driver.workspace = true
serde.workspace = true
serde_json.workspace = true
uuid = { workspace = true, features = ["serde"] }

28
lib/foundation/fabro-api/build.rs
View file

@ -658,28 +658,20 @@ fn main() {
"fabro_types::SandboxListResponse",
&[],
),
("SandboxNetwork", "fabro_types::SandboxNetwork", &[]),
// A sandbox's status is the sandbox driver's own type: the API reuses
// it and the types it carries rather than projecting them.
("SandboxStatus", "sandbox_driver::SandboxStatus", &[]),
("SandboxId", "sandbox_driver::SandboxId", &[]),
("SandboxState", "sandbox_driver::SandboxState", &[]),
("SandboxResources", "sandbox_driver::Resources", &[]),
("SandboxNetworkPolicy", "sandbox_driver::NetworkPolicy", &[]),
("SandboxKind", "sandbox_driver::SandboxKind", &[]),
(
"SandboxNetworkPolicy",
"fabro_types::SandboxNetworkPolicy",
&[],
),
(
"SandboxNetworkPolicyMode",
"fabro_types::SandboxNetworkPolicyMode",
"SandboxWorkspaceOwnership",
"sandbox_driver::WorkspaceOwnership",
&[],
),
("SandboxService", "fabro_types::SandboxService", &[]),
(
"SandboxServiceDiscoverySource",
"fabro_types::SandboxServiceDiscoverySource",
&[],
),
(
"SandboxServiceListMeta",
"fabro_types::SandboxServiceListMeta",
&[],
),
(
"SandboxServiceListResponse",
"fabro_types::SandboxServiceListResponse",

27
lib/foundation/fabro-api/src/lib.rs
View file

@ -62,18 +62,16 @@ pub mod types {
RunPairStatusResponse, RunProjection, RunProvenance, RunRunnableSource, RunSandbox,
RunSandboxFailure, RunSandboxInstance, RunSandboxKind, RunSandboxPlan, RunSandboxRuntime,
RunServerProvenance, RunSessionMetadata, RunSize, RunTarget, SandboxDetails, SandboxInfo,
SandboxListMeta, SandboxListResponse, SandboxNetwork, SandboxNetworkPolicy,
SandboxNetworkPolicyMode, SandboxProviderKind, SandboxProviderLookupError,
SandboxResources, SandboxService, SandboxServiceListResponse, SandboxState,
SandboxTimestamps, SecretMetadata, SecretType, ServerSettings, SessionDetail, SessionId,
SessionStatus, SessionSummary, SessionTurn, SkillActivationSource, SkillSummary,
SkillsProjection, StageCompletion, StageContextWindow, StageContextWindowUnavailableReason,
StageHandler, StageId, StageInferenceProjection, StageModelUsage, StageOutcome,
StageProjection, StageState, StageToolBatchProjection, SubAgentProjection, SubAgentStatus,
SystemActorKind, SystemIntegrationStatus, SystemIntegrationsResponse, TodoListProjection,
ToolCategory, ToolSource, ToolSummary, TurnId, UpdateVariableRequest, UserPrincipal,
Variable, VariableListResponse, WorkflowPath, WorkflowSettings, WorkflowVersion,
WorkflowVersionId,
SandboxListMeta, SandboxListResponse, SandboxProviderKind, SandboxProviderLookupError,
SandboxService, SandboxServiceListResponse, SecretMetadata, SecretType, ServerSettings,
SessionDetail, SessionId, SessionStatus, SessionSummary, SessionTurn,
SkillActivationSource, SkillSummary, SkillsProjection, StageCompletion, StageContextWindow,
StageContextWindowUnavailableReason, StageHandler, StageId, StageInferenceProjection,
StageModelUsage, StageOutcome, StageProjection, StageState, StageToolBatchProjection,
SubAgentProjection, SubAgentStatus, SystemActorKind, SystemIntegrationStatus,
SystemIntegrationsResponse, TodoListProjection, ToolCategory, ToolSource, ToolSummary,
TurnId, UpdateVariableRequest, UserPrincipal, Variable, VariableListResponse, WorkflowPath,
WorkflowSettings, WorkflowVersion, WorkflowVersionId,
};
pub use lithos_llm::catalog::{ModelHandle, ProviderId};
pub use lithos_llm::types::{
@ -83,6 +81,11 @@ pub mod types {
ToolDefinition as CompletionToolDefinition,
ToolDefinitionKind as CompletionToolDefinitionKind,
};
/// A sandbox's status on the API is the sandbox driver's own type.
pub use sandbox_driver::{
NetworkPolicy as SandboxNetworkPolicy, Resources as SandboxResources, SandboxId,
SandboxKind, SandboxState, SandboxStatus, WorkspaceOwnership as SandboxWorkspaceOwnership,
};
pub use crate::generated::types::*;
}

169
lib/foundation/fabro-api/tests/sandbox_details_round_trip.rs
View file

@ -1,38 +1,60 @@
use std::any::{TypeId, type_name};
use std::collections::BTreeMap;
use std::time::SystemTime;
use chrono::{TimeZone, Utc};
use chrono::DateTime;
use fabro_api::types::{
SandboxDetails as ApiSandboxDetails, SandboxNetwork as ApiSandboxNetwork,
SandboxNetworkPolicy as ApiSandboxNetworkPolicy,
SandboxNetworkPolicyMode as ApiSandboxNetworkPolicyMode,
SandboxProviderKind as ApiSandboxProvider, SandboxResources as ApiSandboxResources,
SandboxState as ApiSandboxState, SandboxTimestamps as ApiSandboxTimestamps,
SandboxDetails as ApiSandboxDetails, SandboxId as ApiSandboxId, SandboxKind as ApiSandboxKind,
SandboxNetworkPolicy as ApiSandboxNetworkPolicy, SandboxProviderKind as ApiSandboxProvider,
SandboxResources as ApiSandboxResources, SandboxState as ApiSandboxState,
SandboxStatus as ApiSandboxStatus, SandboxWorkspaceOwnership as ApiSandboxWorkspaceOwnership,
};
use fabro_types::{
RunSandboxInstance, RunSandboxRuntime, SandboxDetails, SandboxNetwork, SandboxNetworkPolicy,
SandboxNetworkPolicyMode, SandboxProviderKind, SandboxResources, SandboxState,
SandboxTimestamps,
use fabro_types::{RunSandboxInstance, RunSandboxRuntime, SandboxDetails, SandboxProviderKind};
use sandbox_driver::{
NetworkPolicy, Resources, SandboxId, SandboxKind, SandboxState, SandboxStatus,
WorkspaceOwnership,
};
use serde_json::json;
#[test]
fn sandbox_details_reuses_domain_types() {
fn sandbox_details_reuses_the_domain_and_driver_types() {
assert_same_type::<ApiSandboxDetails, SandboxDetails>();
assert_same_type::<ApiSandboxProvider, SandboxProviderKind>();
assert_same_type::<ApiSandboxStatus, SandboxStatus>();
assert_same_type::<ApiSandboxId, SandboxId>();
assert_same_type::<ApiSandboxState, SandboxState>();
assert_same_type::<ApiSandboxResources, SandboxResources>();
assert_same_type::<ApiSandboxTimestamps, SandboxTimestamps>();
assert_same_type::<ApiSandboxNetwork, SandboxNetwork>();
assert_same_type::<ApiSandboxNetworkPolicy, SandboxNetworkPolicy>();
assert_same_type::<ApiSandboxNetworkPolicyMode, SandboxNetworkPolicyMode>();
assert_same_type::<ApiSandboxResources, Resources>();
assert_same_type::<ApiSandboxNetworkPolicy, NetworkPolicy>();
assert_same_type::<ApiSandboxKind, SandboxKind>();
assert_same_type::<ApiSandboxWorkspaceOwnership, WorkspaceOwnership>();
}
#[test]
fn sandbox_details_json_matches_openapi_shape() {
let created_at = Utc.with_ymd_and_hms(2026, 5, 9, 12, 0, 0).unwrap();
let mut status = SandboxStatus::new(
SandboxId::try_new("container-abc123").unwrap(),
SandboxState::Running,
);
status.name = Some("fabro-run-abc".to_string());
status.provider_state = "running".to_string();
let mut resources = Resources::default();
resources.cpu_cores = Some(2);
resources.memory_mb = Some(4096);
status.resources = Some(resources);
status.sandbox_kind = Some(SandboxKind::Container);
status.labels.insert("run".to_string(), "abc".to_string());
status.image = Some("ghcr.io/fabro/sandbox:latest".to_string());
status.network = Some(NetworkPolicy::CidrAllowList {
cidrs: vec!["10.0.0.0/8".to_string()],
});
status.web_url = Some(
"https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9"
.to_string(),
);
status.created_at = Some(SystemTime::from(
DateTime::parse_from_rfc3339("2026-05-09T12:00:00Z").unwrap(),
));
let details = SandboxDetails {
sandbox: RunSandboxInstance {
sandbox: RunSandboxInstance {
provider: SandboxProviderKind::DOCKER,
image: Some("ghcr.io/fabro/sandbox:latest".to_string()),
snapshot: None,
@ -48,27 +70,7 @@ fn sandbox_details_json_matches_openapi_shape() {
primary_repo_link: Some("/workspace/fabro".to_string()),
},
},
state: SandboxState::Running,
native_state: Some("running".to_string()),
region: None,
web_url: Some(
"https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9"
.to_string(),
),
resources: SandboxResources {
cpu_cores: Some(2.0),
memory_bytes: Some(4 * 1024 * 1024 * 1024),
disk_bytes: None,
},
network: SandboxNetwork {
egress: SandboxNetworkPolicy::open(),
ingress: SandboxNetworkPolicy::blocked(),
},
labels: BTreeMap::from([("run".to_string(), "abc".to_string())]),
timestamps: SandboxTimestamps {
created_at: Some(created_at),
last_activity_at: None,
},
status,
};
assert_eq!(
@ -86,75 +88,68 @@ fn sandbox_details_json_matches_openapi_shape() {
"primary_repo_link": "/workspace/fabro"
}
},
"state": "running",
"native_state": "running",
"web_url": "https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9",
"resources": {
"cpu_cores": 2.0,
"memory_bytes": 4_294_967_296_u64,
},
"network": {
"egress": {
"mode": "open",
"cidrs": []
"status": {
"id": "container-abc123",
"name": "fabro-run-abc",
"state": "running",
"provider_state": "running",
"error_reason": null,
"resources": {
"cpu_cores": 2,
"memory_mb": 4096,
"disk_mb": null,
"gpus": null
},
"ingress": {
"mode": "blocked",
"cidrs": []
}
},
"labels": {
"run": "abc"
},
"timestamps": {
"created_at": "2026-05-09T12:00:00Z"
"sandbox_kind": "container",
"region": null,
"labels": { "run": "abc" },
"image": "ghcr.io/fabro/sandbox:latest",
"snapshot": null,
"network": { "cidr_allow_list": { "cidrs": ["10.0.0.0/8"] } },
"workspace_ownership": null,
"web_url": "https://app.daytona.io/dashboard/sandboxes?sandboxId=ad65029a-2d01-421e-8936-49451653fcd9",
"created_at": "2026-05-09T12:00:00Z",
"updated_at": null
}
})
);
}
#[test]
fn sandbox_details_deserializes_when_optional_fields_are_absent() {
fn sandbox_details_deserializes_a_status_with_only_its_required_fields() {
let details: SandboxDetails = serde_json::from_value(json!({
"sandbox": {
"provider": "local",
"runtime": {
"id": "local:01JNQVR7M0EJ5GKAT2SC4ERS1Z",
"id": "host-dir-2f55736572732f636c69656e742f70726f6a656374",
"working_directory": "/Users/client/project"
}
},
"state": "unknown",
"resources": {},
"labels": {},
"timestamps": {}
"status": {
"id": "host-dir-2f55736572732f636c69656e742f70726f6a656374",
"state": "running",
"workspace_ownership": "designated"
}
}))
.unwrap();
assert_eq!(details.sandbox.provider, SandboxProviderKind::LOCAL);
assert_eq!(details.status.state, SandboxState::Running);
assert_eq!(
details.sandbox.runtime.id.as_str(),
"local:01JNQVR7M0EJ5GKAT2SC4ERS1Z"
details.status.workspace_ownership,
Some(WorkspaceOwnership::Designated)
);
assert_eq!(
details.sandbox.runtime.working_directory.as_str(),
"/Users/client/project"
);
assert_eq!(details.state, SandboxState::Unknown);
assert!(details.sandbox.image.is_none());
assert!(details.region.is_none());
assert!(details.native_state.is_none());
assert!(details.labels.is_empty());
assert_eq!(details.resources, SandboxResources::default());
assert_eq!(details.network, SandboxNetwork::unknown());
assert_eq!(details.timestamps, SandboxTimestamps::default());
assert!(details.status.resources.is_none());
assert!(details.status.network.is_none());
assert!(details.status.created_at.is_none());
}
fn assert_same_type<T: 'static, U: 'static>() {
fn assert_same_type<A: 'static, B: 'static>() {
assert_eq!(
TypeId::of::<T>(),
TypeId::of::<U>(),
"{} should be the same type as {}",
type_name::<T>(),
type_name::<U>()
TypeId::of::<A>(),
TypeId::of::<B>(),
"{} should be {}",
type_name::<A>(),
type_name::<B>()
);
}

138
lib/foundation/fabro-api/tests/sandbox_inventory_round_trip.rs
View file

@ -1,17 +1,17 @@
use std::any::{TypeId, type_name};
use std::collections::BTreeMap;
use std::time::SystemTime;
use chrono::{TimeZone, Utc};
use chrono::DateTime;
use fabro_api::types::{
SandboxInfo as ApiSandboxInfo, SandboxListMeta as ApiSandboxListMeta,
SandboxListResponse as ApiSandboxListResponse, SandboxProviderKind as ApiSandboxProviderKind,
SandboxProviderLookupError as ApiSandboxProviderLookupError,
};
use fabro_types::{
SandboxInfo, SandboxListMeta, SandboxListResponse, SandboxNetwork, SandboxNetworkPolicy,
SandboxProviderKind, SandboxProviderLookupError, SandboxResources, SandboxState,
SandboxTimestamps,
SandboxInfo, SandboxListMeta, SandboxListResponse, SandboxProviderKind,
SandboxProviderLookupError,
};
use sandbox_driver::{NetworkPolicy, Resources, SandboxId, SandboxState, SandboxStatus};
use serde_json::json;
#[test]
@ -25,99 +25,95 @@ fn sandbox_inventory_round_trip_reuses_domain_types() {
#[test]
fn sandbox_inventory_round_trip_json_matches_openapi_shape() {
let created_at = Utc.with_ymd_and_hms(2026, 5, 25, 12, 0, 0).unwrap();
let mut status = SandboxStatus::new(
SandboxId::try_new("sandbox-abc123").unwrap(),
SandboxState::Running,
);
status.name = Some("fabro-01KSGHGMCFM8W2FHXNMJ7MVY65".to_string());
status.provider_state = "started".to_string();
let mut resources = Resources::default();
resources.cpu_cores = Some(2);
resources.memory_mb = Some(4096);
resources.disk_mb = Some(20 * 1024);
status.resources = Some(resources);
status.region = Some("us".to_string());
status
.labels
.insert("sh.fabro.managed".to_string(), "true".to_string());
status.snapshot = Some("daytona-medium".to_string());
status.network = Some(NetworkPolicy::Block);
status.web_url =
Some("https://app.daytona.io/dashboard/sandboxes?sandboxId=sandbox-abc123".to_string());
let at = SystemTime::from(DateTime::parse_from_rfc3339("2026-05-25T12:00:00Z").unwrap());
status.created_at = Some(at);
status.updated_at = Some(at);
let response = SandboxListResponse {
data: vec![SandboxInfo {
provider: SandboxProviderKind::DAYTONA,
id: "sandbox-abc123".to_string(),
display_name: Some("fabro-01KSGHGMCFM8W2FHXNMJ7MVY65".to_string()),
state: SandboxState::Running,
native_state: Some("started".to_string()),
image: None,
snapshot: Some("daytona-medium".to_string()),
region: Some("us".to_string()),
web_url: Some(
"https://app.daytona.io/dashboard/sandboxes?sandboxId=sandbox-abc123".to_string(),
),
working_directory: Some("/home/daytona/workspace".to_string()),
resources: SandboxResources {
cpu_cores: Some(2.0),
memory_bytes: Some(4 * 1024 * 1024 * 1024),
disk_bytes: Some(20 * 1024 * 1024 * 1024),
},
network: SandboxNetwork {
egress: SandboxNetworkPolicy::open(),
ingress: SandboxNetworkPolicy::blocked(),
},
labels: BTreeMap::from([(
"sh.fabro.managed".to_string(),
"true".to_string(),
)]),
timestamps: SandboxTimestamps {
created_at: Some(created_at),
last_activity_at: Some(created_at),
},
provider: SandboxProviderKind::DAYTONA,
status,
}],
meta: SandboxListMeta {
provider_errors: vec![SandboxProviderLookupError {
provider: SandboxProviderKind::DOCKER,
message: "Failed to connect to Docker daemon".to_string(),
message: "docker daemon unreachable".to_string(),
}],
},
};
let value = serde_json::to_value(&response).unwrap();
assert_eq!(
serde_json::to_value(&response).unwrap(),
value,
json!({
"data": [{
"provider": "daytona",
"id": "sandbox-abc123",
"display_name": "fabro-01KSGHGMCFM8W2FHXNMJ7MVY65",
"state": "running",
"native_state": "started",
"snapshot": "daytona-medium",
"region": "us",
"web_url": "https://app.daytona.io/dashboard/sandboxes?sandboxId=sandbox-abc123",
"working_directory": "/home/daytona/workspace",
"resources": {
"cpu_cores": 2.0,
"memory_bytes": 4_294_967_296_u64,
"disk_bytes": 21_474_836_480_u64
},
"network": {
"egress": {
"mode": "open",
"cidrs": []
"status": {
"id": "sandbox-abc123",
"name": "fabro-01KSGHGMCFM8W2FHXNMJ7MVY65",
"state": "running",
"provider_state": "started",
"error_reason": null,
"resources": {
"cpu_cores": 2,
"memory_mb": 4096,
"disk_mb": 20480,
"gpus": null
},
"ingress": {
"mode": "blocked",
"cidrs": []
}
},
"labels": {
"sh.fabro.managed": "true"
},
"timestamps": {
"sandbox_kind": null,
"region": "us",
"labels": { "sh.fabro.managed": "true" },
"image": null,
"snapshot": "daytona-medium",
"network": "block",
"workspace_ownership": null,
"web_url": "https://app.daytona.io/dashboard/sandboxes?sandboxId=sandbox-abc123",
"created_at": "2026-05-25T12:00:00Z",
"last_activity_at": "2026-05-25T12:00:00Z"
"updated_at": "2026-05-25T12:00:00Z"
}
}],
"meta": {
"provider_errors": [{
"provider": "docker",
"message": "Failed to connect to Docker daemon"
"message": "docker daemon unreachable"
}]
}
})
);
let decoded: ApiSandboxListResponse = serde_json::from_value(value).unwrap();
assert_eq!(decoded.data[0].provider, SandboxProviderKind::DAYTONA);
assert_eq!(decoded.data[0].status.state, SandboxState::Running);
assert!(matches!(
decoded.data[0].status.network,
Some(NetworkPolicy::Block)
));
}
fn assert_same_type<T: 'static, U: 'static>() {
fn assert_same_type<A: 'static, B: 'static>() {
assert_eq!(
TypeId::of::<T>(),
TypeId::of::<U>(),
"{} should be the same type as {}",
type_name::<T>(),
type_name::<U>()
TypeId::of::<A>(),
TypeId::of::<B>(),
"{} should be {}",
type_name::<A>(),
type_name::<B>()
);
}

29
lib/foundation/fabro-api/tests/sandbox_services_round_trip.rs
View file

@ -4,10 +4,7 @@ use fabro_api::types::{
SandboxService as ApiSandboxService,
SandboxServiceListResponse as ApiSandboxServiceListResponse,
};
use fabro_types::{
SandboxService, SandboxServiceDiscoverySource, SandboxServiceListMeta,
SandboxServiceListResponse,
};
use fabro_types::{SandboxService, SandboxServiceListResponse};
use serde_json::json;
#[test]
@ -22,15 +19,9 @@ fn sandbox_services_json_matches_openapi_shape() {
data: vec![SandboxService {
port: 3000,
addresses: vec!["127.0.0.1:3000".to_string(), "[::]:3000".to_string()],
processes: vec![
r#"users:(("node",pid=42,fd=23))"#.to_string(),
r#"users:(("vite",pid=84,fd=19))"#.to_string(),
],
processes: vec!["node".to_string()],
preview_supported: true,
}],
meta: SandboxServiceListMeta {
source: SandboxServiceDiscoverySource::Ss,
},
};
assert_eq!(
@ -39,27 +30,19 @@ fn sandbox_services_json_matches_openapi_shape() {
"data": [{
"port": 3000,
"addresses": ["127.0.0.1:3000", "[::]:3000"],
"processes": [
r#"users:(("node",pid=42,fd=23))"#,
r#"users:(("vite",pid=84,fd=19))"#,
],
"processes": ["node"],
"preview_supported": true
}],
"meta": {
"source": "ss"
}
}]
})
);
}
#[test]
fn sandbox_services_deserializes_empty_response() {
let response: SandboxServiceListResponse =
serde_json::from_value(json!({ "data": [], "meta": { "source": "procfs" } }))
.expect("empty service response should deserialize");
let response: SandboxServiceListResponse = serde_json::from_value(json!({ "data": [] }))
.expect("empty service response should deserialize");
assert!(response.data.is_empty());
assert_eq!(response.meta.source, SandboxServiceDiscoverySource::Procfs);
}
fn assert_same_type<T: 'static, U: 'static>() {

Some files were not shown because too many files have changed in this diff Show more