Mirror worker tracing into run-scoped runtime/server.log files, expose them through the run logs API, and include run.log in dump exports when available.
Wrap root CLI errors at the main boundary so fatal diagnostics use miette's styled renderer while preserving existing telemetry, exit codes, and auth help hints.
Integrates origin's worker-JWT-auth work (commits 8a6f83bb0..c847a828d)
with the config-boundary refactor that landed locally. Conflicts
resolved:
- commands/dump.rs: take origin's removal of the 500-line in-process
test block (replaced by real-server integration coverage).
- commands/run/runner.rs: keep local's dense WorkflowSettings import,
drop dead SettingsLayer import, pull in origin's ActorRef.
- manifest_builder.rs: adopt origin's lifted working_directory
resolution (fixes#159 - manifest git detection in nested repos),
but via local's resolve_working_directory_from_run API that takes
the dense RunNamespace. Update the regression test's
ManifestBuildInput literal to local's run_overrides/cli_overrides
field shape.
- server.rs: keep origin's jwt_auth_mode/jwt_auth_state/
test_user_subject/issue_test_user_jwt/issue_test_worker_token/
create_run_with_bearer/bearer_request test helpers, adapt
jwt_auth_state to local's create_test_app_state_with_session_key
signature (ServerSettings + RunLayer), keep local's dense
canonical_origin_settings that returns ServerSettings via
server_settings_from_toml. Rewrite
build_app_state_requires_session_secret_for_worker_tokens against
the dense AppStateConfig (resolved_settings +
resolved_runtime_settings_for_tests).
Post-merge verification: workspace builds clean, cargo +nightly
fmt --check all clean, cargo +nightly clippy --workspace
--all-targets -- -D warnings clean, cargo nextest run --workspace
4560 tests passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FINALIZE loaded the run event log twice: once via build_conclusion_from_store
for stage durations, then again to count ArtifactCaptured events. Merged into
a single walk feeding both the conclusion and the artifact count.
Collapsed six near-identical pipeline::execute + emit_terminal + flush blocks
in test_support into one execute_and_emit_terminal helper. Also trimmed
narrative comments that described caller ordering, control flow, or the fix
commit rather than non-obvious invariants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `WorkflowRunCompleted` / `WorkflowRunFailed` event was emitted from
`EventLifecycle::on_run_end`, a callback the executor fires at the end of
the EXECUTE phase. But the run isn't done at that point — RETRO and
FINALIZE still need to run, and FINALIZE writes the meta branch's finalize
commit. Observers that treat the event as "done" (CLI attach, daemon SSE
consumers) could observe terminal state and act on it before the worker
flushed its remaining writes.
The recovery scenario test exposed this: it deletes the meta branch
right after `fabro run` returns, then asserts the branch is empty. On
loaded CI runners the worker's finalize commit landed after the delete,
recreating the branch and failing the assertion.
Move the terminal event emission to `pipeline::finalize::finalize`, after
`write_finalize_commit`. The lifecycle's `on_run_end` overrides for event
and git become empty (deleted — the trait already provides a no-op
default). Three pieces of cross-cutting state (`final_patch`,
`captured_artifact_count`, the dead `EventLifecycle` reads of
`last_git_sha`) only existed to ferry data from EXECUTE to the terminal
event; deleted those too. The aggregator collapses to a one-line
delegate to `hook.on_run_end`.
`write_finalize_commit` now takes the conclusion as a parameter and
injects it into the projection copy, since the terminal event hasn't run
through the run store yet when the meta branch is written.
`build_terminal_event` is `pub(crate)` so `test_support` helpers (which
stop at EXECUTE) can mirror the production payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The worker subprocess is spawned with env_clear+allowlist by the server, so
the only sensitive value in its env is FABRO_WORKER_TOKEN itself. Read the
token and remove_var it from the process env in main() before Tokio starts
worker threads, then thread it explicitly through runner::execute(&str).
Every descendant (hooks, local sandbox, devcontainer initializeCommand,
MCP stdio, etc.) now inherits a worker env with no bearer in it, so an
unscrubbed spawn site cannot leak the token. This makes the prior denylist
scrub in fabro-hooks and fabro-sandbox redundant — delete it and the shared
WORKER_SECRET_ENV_DENYLIST constant. The sandbox keeps its _api_key/_secret/
_token/_password/_credential suffix heuristic for user-supplied env_vars
hygiene.
Extend the server-dispatched-worker env-leak integration test to also
assert a Bash stage running in the worker does not observe FABRO_WORKER_TOKEN.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the plan at
docs/plans/2026-04-20-003-refactor-unify-run-vocabulary-metadata-plan.md.
- Rename RunRecord to RunSpec and RunProjection.run to .spec everywhere
in Rust source, tests, helpers, test names, and error messages.
- Introduce SerializableProjection wrapper that trims bulky node text
fields (prompt, response, diff, stdout, stderr) for run.json snapshots.
- Collapse metadata-branch and CLI export to one RunDump::from_projection
builder emitting run.json + graph.fabro + stages/{stage_id}/... and
drop legacy top-level start/status/checkpoint/sandbox/retro/conclusion
split files.
- Replace MetadataStore::write_checkpoint with write_snapshot returning
the commit SHA; add read_run_projection/read_run_spec; demote
read_checkpoint/read_start_record to projection-field extractors.
- Switch fork, rewind, rebuild_meta, CLI rewind recovery, and retro
upload to read the unified projection layout.
- Add additive query methods on RunSpec and RunProjection.
Serde-level `alias = "spec"` shim dropped; `rename = "run"` retained to
keep the server API wire format stable per the plan's scope boundary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lift shared client DTOs into fabro-types, move auth/target/error/session
logic into fabro-client, and reduce fabro-cli to orchestration around the
builder-based client path.
This also lands the remaining plan cleanup for ApiError, ServerTarget
canonicalization, and the RunEventStream rename at the CLI boundary.
Add shared axum/reqwest response assertion helpers in fabro-test,
migrate the Rust HTTP test surface to use them, and document the
new rule in the testing strategy.
Extends the workspace clippy.toml — which already bans std:🧵:sleep,
std:🧵:spawn, and std::process::Command::new on Tokio paths — with:
- disallowed-types: std::io::{Read, Write, BufRead, BufReader, BufWriter}
and std::net::{TcpStream, TcpListener, UdpSocket}
- disallowed-methods: std::io::{stdin, stdout, stderr}
Non-blocking std::io items (Error, ErrorKind, Result, IsTerminal, Cursor)
remain allowed. std::fs is intentionally deferred.
Annotates ~24 pre-existing sync call sites with #[expect(..., reason = "...")]
matching the established pattern. All annotations describe why blocking I/O
is intentional in that context (sync CLI command, test helper, pre-fork
flush, etc.), so a future conversion to async will surface as an unfulfilled
lint expectation instead of silently drifting.
Fixes one real Tokio-path issue surfaced by the new lint:
fabro-cli's server-start daemon-health poller (try_connect) was a sync fn
called from async execute_daemon; std::net::TcpStream::connect_timeout
blocked a Tokio worker for up to 100ms per poll iteration. Converted to
tokio::net::{TcpStream, UnixStream} with tokio::time::timeout.
One follow-up flagged in-code: fabro-agent/src/cli.rs's JSON event writer
uses std::io::stdout() inside tokio::spawn. Annotated with a FOLLOW-UP
reason pointing at tokio::io::stdout; left unchanged since volume is low
and scope exceeded this pass.
Verified: clippy clean, cargo +nightly fmt --check clean, full nextest
workspace run (4131 passed, 182 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Stacked cleanup of the `canonicalize blocked run status` work (local
commit `d13cdf374`) plus reconciliation with origin's `canonicalize
paginated run list responses` (origin commit `8ab689da7`). Both efforts
ran in parallel and diverged on the column name (`blocked` vs `waiting`)
and on how the board response is shaped — this PR converges them,
keeping `blocked` as the canonical column id while adopting origin's
`column` field on `RunListItem` and `StoreRunSummary` shape.
Also fixes a production-worker regression introduced by the
canonicalization: the worker's start-precondition only accepted
`Submitted | Starting`, so once runs started transitioning through
`Queued` on the way to `Starting`, every subprocess-worker run failed
with `Precondition failed: cannot start run: status is Queued`. That
cascaded into ~90 failing CLI/server integration tests locally.
## Commits
1. `f65843168` refactor(runs): simplify blocked status follow-ups
2. `1492d956c` chore: resolve clippy warnings
3. `676fd9f44` first merge of origin/main
4. `23fc92a2f` **fix(runs): allow Queued status in start precondition**
← the cascade-fix
5. `36b507a83` refactor: simplify pause/unpause + dedupe web status
tables
6. `8d8d27748` refactor(workflow): encapsulate BlockedStateTracker
inside HumanHandler
7. `1c17fda35` second merge of origin/main — resolves waiting vs blocked
8. `4cd3ef7b1` refactor(workflow): Mutex<usize> → AtomicUsize
9. `2e5a58e8a` fix(demo): align run-4 lifecycle status with Blocked
board column
## Test plan
- [x] fmt, clippy, build, doctests all clean
- [x] `cargo nextest run --workspace` — **4092/4092 pass**
- [x] `bun test` — **26/26 pass**, typecheck + production build clean
- [x] Manual CLI repro of the Queued-precondition fix
- [x] Browser smoke test: all 5 columns render with correct
labels/colors, demo run-4 appears in Blocked lane with question text
intact
## Known follow-up (not blocking)
A "paused-while-blocked" run (status `Paused` + `blocked_reason: Some`)
lands in the `running` column because the visible status chooses
`Paused` over `Blocked`. The pending question is not prominent on the
board. Addressing it would require `board_column()` to branch on
`(status, blocked_reason)` rather than just `status` — worth a separate
ticket.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CLI integration tests spawned the real fabro binary while letting the
parent process's env pass through. The pr_view "no credentials" snapshot
failed in CI because the Nightly workflow's minted GITHUB_TOKEN was
inherited by the child and turned the expected "credentials required"
error into a real GitHub API call (404 / 401). On developer laptops the
same leak occurs whenever gh auth login is active.
Introduce apply_test_isolation(cmd, home) in fabro-test: env_clear() +
re-populate PATH, HOME, NO_COLOR, and the FABRO_* test overrides. Route
TestContext::command(), the internal server bootstrap, and the four
ad-hoc spawners in tests/it/cmd/{attach,render_graph,runner,server_start}
through the same helper so the isolation is systemic instead of
per-callsite. Tests that deliberately need a credential (OPENAI_API_KEY,
GITHUB_APP_PRIVATE_KEY, etc.) continue to set it explicitly on the
returned Command; those survive the clear.
Add a regression test that sets sentinel GITHUB_TOKEN and
ANTHROPIC_API_KEY in the parent, spawns /usr/bin/env through the helper,
and asserts the child sees neither credential while still seeing PATH
and the harness's FABRO_NO_UPGRADE_CHECK override.
Verified: the full workspace (4022 tests) passes with GITHUB_TOKEN and
ANTHROPIC_API_KEY set in the parent, which previously broke the
pr_view_reads_pull_request_from_store_without_pull_request_json
snapshot. cargo fmt and nightly clippy are clean.
Conflicts resolved:
- install.rs: kept simplified auth (port 32276, no TLS, no username
in merge_server_settings), adapted to origin's input_source API by
removing username from ServerConfigSelection::Write
- serve.rs: kept ProviderCredentials import from origin, dropped
ClientAuth (removed with mTLS)
- server.rs: kept both imports (ServerAuthMethod + Provider)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Propagate the local dev token through worker subprocesses, share the
same authenticated local-server helper across CLI integration tests,
and clean up the async token wait path so fmt, clippy, and full tests
pass again after the dev-token auth rollout.
Move async subprocess paths to Tokio or spawn_blocking, document the
intentional synchronous std::process::Command callsites, and make CI run
Clippy with --all-targets so the guardrail applies to test code too.
Add a Clippy disallowed-methods guardrail for std::thread sleep/spawn
and convert the CLI polling paths to tokio::time::sleep so they no
longer block Tokio workers. Keep the intentional OS-thread sites with
narrow #[expect(...)] annotations that explain why std::thread is
required there.
Add the shared fabro-http transport crate and route hand-written HTTP client construction through it.
Use FABRO_HTTP_PROXY_POLICY for test no-proxy defaults, remove direct reqwest deps from ordinary crates, and add clippy bans for raw reqwest entrypoints.
Keep project config and checked-in workflows under .fabro so they stay out of
normal repo listings. Update config discovery, CLI project commands, fixtures,
docs, and checked-in workflow paths to use .fabro/project.toml and
.fabro/workflows/*.
Move worker control stdin handling off Tokio's blocking shutdown path so
subprocess workers can exit cleanly after success or cooperative
cancellation even when the parent still holds stdin open.
Add regression coverage for retro-enabled success and SIGTERM-driven
cancellation with stdin intentionally left open.
Extends the stage 6.1 WIP into a compiling state across the workspace.
Most crates and their unit/integration tests now read run.* / cli.* /
server.* v2 layers directly or through targeted bridge helpers.
Key moves in this commit:
fabro-server
- AppState.settings: Arc<RwLock<SettingsFile>> -- all helpers,
create_app_state_with_* factories, and tests updated.
- api_server_settings bridges SettingsFile -> legacy Settings via the
transitional bridge so /api/v1/settings still emits the legacy DTO
shape until Stage 6.6 replaces it with an allow-list DTO.
- get_system_info, get_system_df, get_github_repo, webhook startup, and
other read sites use the v2 accessors (github_app_id_str,
server_web, run_sandbox, run_model_*).
- web_auth.rs wraps each oauth / register / setup-status handler in a
local `bridged` helper that produces a legacy Settings from the v2
state, so the complex oauth mutation flow keeps working until its
Stage 6.6 rewrite.
- diagnostics::check_github_app reads via github_*_str accessors;
check_crypto bridges to the legacy shape inline.
- serve.rs: load_settings returns SettingsFile; apply_serve_overrides /
apply_runtime_settings mutate v2 subtrees directly; the config poll
loop and TLS/webhook startup use bridged() for legacy-shape reads.
- Tests in tests/it/{helpers,api/*,scenario/*} rewritten to construct
SettingsFile via ConfigLayer::parse or v2 struct literals.
fabro-workflow
- Every test fixture in pipeline/{finalize,initialize,pull_request,retro,
execute,persist}, operations/{create,rebuild_meta,start}, run_lookup,
runtime_store, handler/manager_loop, and tests/it/{integration,
daytona_integration}.rs now uses SettingsFile.
- start.rs hooks into the bridge helpers directly via use-imports.
- run_graph / run_graph_from_checkpoint / initialize / finalize /
pull_request calls are Box::pin'd to stay under clippy's large-future
threshold after the v2 tree brought RunOptions size up.
- resolve_run_settings writes resolved model/provider back into
run.model as InterpStrings; tests assert via run_model_*_str().
- preprocess_and_validate pulls vars from run_inputs_as_strings().
fabro-cli
- manifest_builder uses ConfigLayer.combine(...).into() to get a v2
SettingsFile for the manifest goal resolution path; file-based
goal_file handling is deferred to 6.6 when the manifest schema catches
up.
- runner::maybe_build_github_app_credentials and
tests/it/cmd/{create,runner}.rs read from v2 accessors.
- commands/config/mod.rs::merged_config returns SettingsFile; the
server-side retrieve_server_settings is bridged via a stopgap
legacy_settings_to_v2 shim that Stage 6.6 replaces.
- commands/store/dump.rs sample_run_record constructs SettingsFile.
fabro-store, fabro-checkpoint
- Test fixtures constructing RunRecord values updated to SettingsFile.
- fabro-checkpoint/src/author.rs stays (v2 From impl landed in a
previous additive commit).
fabro-config
- effective_settings.rs rewrite compiles and passes its unit tests.
- project::resolve_working_directory takes &SettingsFile.
Build status: `cargo build --workspace --tests`, `cargo clippy
--workspace -- -D warnings`, and `cargo fmt --check --all` all pass.
`cargo nextest run --workspace` passes 3,749 of 3,764 tests; the 15
remaining failures are fabro-cli integration tests whose snapshot +
TOML fixture shapes still need manual updates:
- cmd::config::* (seven tests): fixture TOML files still use v1
top-level keys and the snapshot outputs expect the legacy flat JSON
shape.
- cmd::inspect::* (four tests): run-record JSON snapshots embed the
flat Settings shape.
- cmd::run::dry_run_persists_event_history_in_store and
json_run_implies_auto_approve_for_human_gates: check `settings.dry_run
== Some(true)` directly on the v2 file; should assert
dry_run_enabled() instead.
- cmd::attach::attach_json_errors_without_prompting_for_human_input:
unrelated insta snapshot drift caused by the new SettingsFile JSON
shape leaking into an events-log snapshot.
Follow-up work for this stage also includes:
- Rewriting web_auth.rs register flow to emit v2 TOML directly and to
re-parse the written file back into state.settings so in-memory
state doesn't lag the on-disk file.
- Removing the legacy_settings_to_v2 shim in fabro-cli/config once
the server-side settings endpoint returns v2 shapes (Stage 6.6).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace bin-scoped localhost HTTP tests with command-facing integration
coverage so they run under the intended IT timeout budget without
changing nextest overrides.
Drop scratch-only compatibility paths and legacy test scaffolding now that
SlateDB-backed state is authoritative. This removes scratch file fallbacks,
updates docs and UI labels, and moves tests onto durable store-backed helpers.
Persist pending interviews in run state, deliver accepted answers to workers
through the server-owned control path, and remove the old scratch-file and
WebInterviewer transports.
This also moves Slack onto the canonical server answer flow, adds richer
question metadata to the API and run events, and covers the subprocess
question lifecycle with end-to-end tests.
Tighten the worker upload path so object-backed runs only fail when an
artifact upload is actually attempted without a token, and update CLI
snapshots for the new artifact storage metadata.
Fold in the workspace test and clippy fixes needed to verify the final
artifact upload implementation cleanly across Rust and web targets.
Move subprocess workers fully behind the server-owned run store by
switching worker/server coordination to HTTP-backed run events and
control state. Reconcile stale in-flight runs on boot, terminate live
workers during shutdown, and update process titles to reflect server and
worker lifecycle phases.
Keep local server targeting based on explicit server targets instead of
implicitly deriving a socket from storage_dir. This makes ~/.fabro/fabro.sock
the default local socket again, keeps storage under ~/.fabro/storage, threads
FABRO_CONFIG through server autostart paths, and updates the CLI test harness
for the new split.
Consolidate CLI and server machine defaults under settings.toml,
including loader renames, writer preservation fixes, same-machine
manifest handling, and docs/test updates for the new config model.
Move --storage-dir and --server-url off GlobalArgs and onto the
leaf commands that actually honor them.
This aligns help, parser behavior, and env-var wiring with the
current command architecture while preserving the intended model
and exec targeting semantics.
Rename the hidden detached worker command to __runner, remove launcher
bookkeeping, and resolve runs through a shared server-backed lookup.
This leaves attach, resume, logs, and related commands using the server
as the source of truth and updates the integration coverage around the
new lifecycle.
2026-04-05 12:11:55 -04:00
Renamed from lib/crates/fabro-cli/tests/it/cmd/detached.rs (Browse further)
