Commit graph

394 commits

Author SHA1 Message Date
Bryan Helmkamp
dc856d0884
feat(settings): stage 6.1 consumer migration builds workspace-wide
Extends the stage 6.1 WIP into a compiling state across the workspace.
Most crates and their unit/integration tests now read run.* / cli.* /
server.* v2 layers directly or through targeted bridge helpers.

Key moves in this commit:

fabro-server
- AppState.settings: Arc<RwLock<SettingsFile>> -- all helpers,
  create_app_state_with_* factories, and tests updated.
- api_server_settings bridges SettingsFile -> legacy Settings via the
  transitional bridge so /api/v1/settings still emits the legacy DTO
  shape until Stage 6.6 replaces it with an allow-list DTO.
- get_system_info, get_system_df, get_github_repo, webhook startup, and
  other read sites use the v2 accessors (github_app_id_str,
  server_web, run_sandbox, run_model_*).
- web_auth.rs wraps each oauth / register / setup-status handler in a
  local `bridged` helper that produces a legacy Settings from the v2
  state, so the complex oauth mutation flow keeps working until its
  Stage 6.6 rewrite.
- diagnostics::check_github_app reads via github_*_str accessors;
  check_crypto bridges to the legacy shape inline.
- serve.rs: load_settings returns SettingsFile; apply_serve_overrides /
  apply_runtime_settings mutate v2 subtrees directly; the config poll
  loop and TLS/webhook startup use bridged() for legacy-shape reads.
- Tests in tests/it/{helpers,api/*,scenario/*} rewritten to construct
  SettingsFile via ConfigLayer::parse or v2 struct literals.

fabro-workflow
- Every test fixture in pipeline/{finalize,initialize,pull_request,retro,
  execute,persist}, operations/{create,rebuild_meta,start}, run_lookup,
  runtime_store, handler/manager_loop, and tests/it/{integration,
  daytona_integration}.rs now uses SettingsFile.
- start.rs hooks into the bridge helpers directly via use-imports.
- run_graph / run_graph_from_checkpoint / initialize / finalize /
  pull_request calls are Box::pin'd to stay under clippy's large-future
  threshold after the v2 tree brought RunOptions size up.
- resolve_run_settings writes resolved model/provider back into
  run.model as InterpStrings; tests assert via run_model_*_str().
- preprocess_and_validate pulls vars from run_inputs_as_strings().

fabro-cli
- manifest_builder uses ConfigLayer.combine(...).into() to get a v2
  SettingsFile for the manifest goal resolution path; file-based
  goal_file handling is deferred to 6.6 when the manifest schema catches
  up.
- runner::maybe_build_github_app_credentials and
  tests/it/cmd/{create,runner}.rs read from v2 accessors.
- commands/config/mod.rs::merged_config returns SettingsFile; the
  server-side retrieve_server_settings is bridged via a stopgap
  legacy_settings_to_v2 shim that Stage 6.6 replaces.
- commands/store/dump.rs sample_run_record constructs SettingsFile.

fabro-store, fabro-checkpoint
- Test fixtures constructing RunRecord values updated to SettingsFile.
- fabro-checkpoint/src/author.rs stays (v2 From impl landed in a
  previous additive commit).

fabro-config
- effective_settings.rs rewrite compiles and passes its unit tests.
- project::resolve_working_directory takes &SettingsFile.

Build status: `cargo build --workspace --tests`, `cargo clippy
--workspace -- -D warnings`, and `cargo fmt --check --all` all pass.
`cargo nextest run --workspace` passes 3,749 of 3,764 tests; the 15
remaining failures are fabro-cli integration tests whose snapshot +
TOML fixture shapes still need manual updates:

- cmd::config::* (seven tests): fixture TOML files still use v1
  top-level keys and the snapshot outputs expect the legacy flat JSON
  shape.
- cmd::inspect::* (four tests): run-record JSON snapshots embed the
  flat Settings shape.
- cmd::run::dry_run_persists_event_history_in_store and
  json_run_implies_auto_approve_for_human_gates: check `settings.dry_run
  == Some(true)` directly on the v2 file; should assert
  dry_run_enabled() instead.
- cmd::attach::attach_json_errors_without_prompting_for_human_input:
  unrelated insta snapshot drift caused by the new SettingsFile JSON
  shape leaking into an events-log snapshot.

Follow-up work for this stage also includes:
- Rewriting web_auth.rs register flow to emit v2 TOML directly and to
  re-parse the written file back into state.settings so in-memory
  state doesn't lag the on-disk file.
- Removing the legacy_settings_to_v2 shim in fabro-cli/config once
  the server-side settings endpoint returns v2 shapes (Stage 6.6).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 15:25:59 -04:00
Bryan Helmkamp
7d83f2448c
docs(plans): write Stage 6 handoff for settings TOML redesign
Comprehensive handoff document that the engineer picking up the
settings redesign can use to continue the work. Captures:

- TL;DR of what's landed vs what remains
- Source document references (brainstorm, plan, canonical example)
- Current-state tree map with file-level annotations
- Full dependency chain from TOML -> v2 -> bridge -> consumers
- Commit log for Stages 1-5 (13 commits, 76 files, +6413/-2151)
- Stage 6 broken into six independent subtasks: consumer migration,
  bridge deletion, legacy type deletion, fabro-config shim deletion,
  v2 namespace flattening, OpenAPI + clients + fabro-web rewrite
- Concrete before/after migration patterns with InterpString examples
- Testing gotchas I hit during Stages 1-5: fabro-cli parallel test
  daemon FD limit, insta snapshot pending review, hook shorthand vs
  #[serde(flatten)] duplicate-command collision, fabro-test managed
  marker detection, apply_server_defaults full-shape propagation,
  user layer trust boundary, pre-existing fabro-interview test clippy
  warnings
- Open design questions the next engineer needs to decide: ConfigLayer
  resolve naming, post-layering interpolation pass, fail-closed auth,
  ModelRegistry runtime wiring, run.scm provider depth, serde flatten+
  HashMap+deny_unknown_fields constraint
- Verification recipes (full gate, legacy key sanity grep, bridge
  caller sanity grep)
- Explicit success criteria for Stage 6 completion
2026-04-09 11:54:44 -04:00
Bryan Helmkamp
dba10e5e99
docs: migrate reference and guide examples to v2 config shape
Rewrite every docs/ reference and integration guide example that
previously showed legacy flat TOML (`[llm]`, `[vars]`, `[sandbox]`,
`[setup]`, `[exec]`, `[fabro]`, `[pull_request]`, `[mcp_servers]`,
`[git]`, `[web]`, `[api]`, `[features] retros`, `version = 1`,
top-level `storage_dir`) to use the v2 namespaced schema. Also update
the surrounding prose to describe v2 merge semantics (R22 run.inputs
wholesale replacement, R71 sticky sandbox.env/labels, R30 whole-list
prepare.steps replacement, hook id-based replacement).

Files touched:
- docs/reference/user-configuration.mdx (complete rewrite around
  [cli.*] ownership, [run.*] run-scoped defaults, [cli.target] /
  [cli.exec] / [cli.output] / [cli.updates] / [cli.logging], and
  [run.agent.mcps.<name>] with durations like "10s")
- docs/reference/cli.mdx (settings.toml example uses [cli.exec.*],
  [run.model], [cli.target])
- docs/execution/run-configuration.mdx (full run-config example
  rewritten to use [workflow].graph, [run].goal/working_dir,
  [run.model], [run.prepare.steps], [run.sandbox.daytona.snapshot]
  with Size values, [run.inputs], [run.artifacts], [run.agent.mcps],
  [run.pull_request], [[run.hooks]] with optional id and duration
  timeout; section docs explain the new merge semantics)
- docs/execution/environments.mdx and devcontainers.mdx (sandbox
  examples now use [run.sandbox.*])
- docs/execution/retros.mdx (retros moved to [run.execution] retros
  = true per R31)
- docs/execution/failures.mdx (fallbacks now a single ordered array
  under [run.model].fallbacks)
- docs/workflows/variables.mdx ([vars] → [run.inputs], wholesale
  replacement semantics explained)
- docs/administration/server-configuration.mdx (full reference
  rewritten around [server.listen]/[server.api]/[server.web]/
  [server.auth]/[server.storage]/[server.scheduler]/[server.logging]/
  [server.integrations])
- docs/api-reference/overview.mdx (auth strategies now enabled via
  [server.auth.api.jwt].enabled and [server.auth.api.mtls].enabled;
  listener TLS moved to [server.listen.tls])
- docs/integrations/daytona.mdx, github.mdx (provider config now
  nested under [run.sandbox.daytona] / [server.integrations.github])
- docs/human-tools/ssh-access.mdx (sandbox examples to v2)
- docs/agents/mcp.mdx (Playwright sandbox example to [run.agent.mcps])
- docs/core-concepts/models.mdx (model config and fallbacks array to
  [run.model])

Canonical fabro-cli overrides and server run_manifest now emit
verbose via [cli.output].verbosity = verbose rather than the prior
run.metadata staging. No code changes beyond those Stage 4 fixes that
were already in flight.
2026-04-09 11:30:52 -04:00
Bryan Helmkamp
bc7a926e26
feat(artifacts): remove scratch artifact cache staging
Stage captured artifacts in per-attempt tempdirs and persist them through an
explicit artifact sink instead of writing into run scratch cache.

Server-managed and test-owned runs now write directly to ArtifactStore, while
CLI worker runs keep the staged upload path. The local run summary now prints
durable artifact identifiers and copy hints rather than scratch-cache paths,
and the run-directory docs and integration coverage were updated to match.
2026-04-08 17:42:55 -04:00
Bryan Helmkamp
0344cffce2
feat(cli): add fabro uninstall command
Adds a top-level `fabro uninstall` command that reverses `fabro install`
and `install.sh`. Defaults to dry-run (preview) mode, requiring `--yes`
to execute.

Features:
- Inventory and dry-run preview with sizes and `--json` support
- Server shutdown (guarded — only when server is running)
- Safety guardrails (refuses to delete /, $HOME, or dirs without markers)
- Shell config cleanup (exact `# fabro` sentinel match, PATH validation,
  atomic write via temp+rename)
- Binary status reporting with tailored brew/cargo/manual hints
- Exit code: 0 on success, 1 on critical failure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 16:31:32 -04:00
Bryan Helmkamp
622760952d
Merge remote-tracking branch 'origin/main' 2026-04-08 16:30:26 -04:00
Bryan Helmkamp
dad58c5b4a
plan 2026-04-08 16:24:11 -04:00
Bryan Helmkamp
b101bfd5d5
feat(server): make web ui optional
Add a server-side web.enabled toggle and CLI overrides so Fabro can run
with API and health only while disabling the embedded SPA, browser auth
routes, and web-only helper endpoints.
2026-04-08 16:22:41 -04:00
Bryan Helmkamp
eb4b71cafa
docs(plan): add optional web ui server plan 2026-04-08 16:09:42 -04:00
Bryan Helmkamp
fdb1297442
feat(server): support host-only tcp binds
Accept `--bind <ip>` as a TCP bind request while keeping the default
Unix socket behavior unchanged. Resolve host-only TCP binds inside the
serving process so startup output, server metadata, and status always
reflect the concrete host:port, preferring 32276 and falling back to a
random port with a warning when needed.
2026-04-08 15:43:09 -04:00
Bryan Helmkamp
8bab1335c9
Merge remote-tracking branch 'origin/main' into production-web-ui 2026-04-08 14:44:35 -04:00
Bryan Helmkamp
c58b731171
feat: embed fabro spa and align interruption semantics
Move the built web bundle into an embedded fabro-spa crate so Cargo and
release builds no longer depend on Bun at build time, and preserve the
local dev override path for fast UI iteration.

At the same time, rename interview and agent-level aborted flows to
interrupted, keep cancelled for run-level shutdown, and stop reporting
skipped answers as interruptions in the run event stream.
2026-04-08 14:43:44 -04:00
Bryan Helmkamp
15ccf9aef9
plan 2026-04-08 14:43:00 -04:00
Bryan Helmkamp
fa4efd3689
test: add test plan for production web UI implementation 2026-04-08 04:28:47 -04:00
Bryan Helmkamp
c1abae86a0
plan: revise production web UI plan to fix demo get_run_status, files tab, and Playwright config 2026-04-08 04:17:54 -04:00
Bryan Helmkamp
df44aa1ad6
plan: production web UI implementation plan 2026-04-08 04:06:20 -04:00
Bryan Helmkamp
cb4f7856f6
plan 2026-04-07 23:44:44 -04:00
Bryan Helmkamp
3c9c8cabf0
refactor(artifacts): clean up stale artifact wording
Remove leftover object-backed terminology from the worker uploader,
rename the remaining scratch-fallback test to match current behavior,
and update the old artifact upload plan to reflect the current
no-fallback model.
2026-04-07 23:13:22 -04:00
Bryan Helmkamp
60b469e7a8
refactor(artifacts): remove scratch manifest fallback
Make ArtifactStore the only artifact read path, stop writing
manifest.json into run scratch, and update the CLI summary to
resolve artifact paths from the durable server API.
2026-04-07 22:46:44 -04:00
Bryan Helmkamp
c1507b57a3
refactor(scratch): remove store-backed diff and blob cache writes
Stop writing scratch final.patch files now that diffs are projected from
run state, and remove the unused cache/artifacts/values plumbing while
keeping runtime/blobs materialization intact.

Update tests and run-directory docs to match the current scratch contract.
2026-04-07 22:31:59 -04:00
Bryan Helmkamp
45f8d94df6
refactor(server): stop writing worker stderr scratch logs
Route subprocess worker stderr directly into server tracing and remove
the scratch-file sink. Update the run-directory docs to reflect that
runtime now only documents blob materialization here.
2026-04-07 20:57:52 -04:00
Bryan Helmkamp
4bcd8f7647
refactor(scratch): remove stale scratch file refs
Drop scratch-only compatibility paths and legacy test scaffolding now that
SlateDB-backed state is authoritative. This removes scratch file fallbacks,
updates docs and UI labels, and moves tests onto durable store-backed helpers.
2026-04-07 20:40:06 -04:00
Bryan Helmkamp
326e0c27fa
refactor(interview): move run answers onto control channels
Some checks are pending
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run
TypeScript / Build (push) Waiting to run
Persist pending interviews in run state, deliver accepted answers to workers
through the server-owned control path, and remove the old scratch-file and
WebInterviewer transports.

This also moves Slack onto the canonical server answer flow, adds richer
question metadata to the API and run events, and covers the subprocess
question lifecycle with end-to-end tests.
2026-04-07 19:23:36 -04:00
Bryan Helmkamp
6839ba9e91
plan 2026-04-07 18:37:59 -04:00
Bryan Helmkamp
ffe3854c8f
Merge remote-tracking branch 'origin/main'
Resolve the artifact lifecycle merge by keeping object-backed upload and sandbox sync behavior alongside durable blob-ref normalization.
2026-04-07 17:52:07 -04:00
Bryan Helmkamp
cf94f88791
plans 2026-04-07 17:18:42 -04:00
Bryan Helmkamp
73e9509aa8
feat(workflow): persist offloaded context as global blob refs
Store large context payloads in the global CAS and keep durable
checkpoint state as blob://sha256 refs instead of execution-local file
paths. Resolve and materialize blob refs at execution, output, and export
time so resumed and remote runs can read legacy and new artifacts
consistently.
2026-04-07 17:18:28 -04:00
Bryan Helmkamp
5ced931d1a
feat(artifacts): finish object-backed artifact uploads
Add scoped worker upload tokens and HTTP artifact upload clients.

Support manifest-first multipart stage artifact uploads with validation and checksums.

Gate artifact reads by run capability while preserving legacy scratch fallback.
2026-04-07 16:56:37 -04:00
Bryan Helmkamp
4b8e803aa9
plans 2026-04-07 16:05:21 -04:00
Bryan Helmkamp
518e27260d
plans 2026-04-07 16:05:11 -04:00
Bryan Helmkamp
9ca0113f66
Merge remote-tracking branch 'origin/main'
# Conflicts:
#	apps/fabro-web/app/routes/run-overview.tsx
#	apps/fabro-web/app/routes/workflow-detail.tsx
#	apps/fabro-web/app/routes/workflows.tsx
#	lib/crates/fabro-workflow/src/lifecycle/artifact.rs
#	lib/crates/fabro-workflow/src/pipeline/finalize.rs
2026-04-07 15:24:07 -04:00
Bryan Helmkamp
ba7dc77be0
refactor(billing): unify the LLM billing domain
Replace the overlapping usage and cost model with canonical billing
primitives centered on ModelRef, ModelHandle, TokenCounts, and
BilledModelUsage. This also renames the public API and web surface from
usage to billing, removes compatibility aliases, and normalizes provider
usage adapters onto the shared billing vocabulary.
2026-04-07 14:33:35 -04:00
Bryan Helmkamp
3dcf2ad04b
fix(run): make attach stream terminal-authoritative
Replay persisted run events for attach requests, keep the SSE stream live
only while the run is active, and close on terminal run events instead of
returning 410 for completed runs.

The CLI now treats premature attach EOF as an error, and the affected
integration tests were stabilized around store-backed event ordering and
recovered rewind timelines.
2026-04-07 14:32:23 -04:00
Bryan Helmkamp
d3de014723
plans 2026-04-07 10:30:40 -04:00
Bryan Helmkamp
9fe7268026
chore: remove workflows and steer endpoints
Remove GET /workflows, GET /workflows/{name}, GET /workflows/{name}/runs,
and POST /runs/{id}/steer from the OpenAPI spec, server routes, demo
fixtures, pagination tests, docs navigation, and generated TS client.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 08:25:54 -04:00
Bryan Helmkamp
b96c8d22a2
fix(docs): sync Mintlify docs.json with current OpenAPI spec
Remove endpoint references deleted from the spec (context, files,
sessions) and add the new start endpoint so Mintlify can resolve
all page anchors against the OpenAPI file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 08:08:10 -04:00
Bryan Helmkamp
c9d1b92725
feat(run): harden server-supervised worker lifecycle
Move subprocess workers fully behind the server-owned run store by
switching worker/server coordination to HTTP-backed run events and
control state. Reconcile stale in-flight runs on boot, terminate live
workers during shutdown, and update process titles to reflect server and
worker lifecycle phases.
2026-04-07 07:59:35 -04:00
Bryan Helmkamp
5b63e25b28
chore: remove unused verification, retros, and sessions endpoints
These endpoints had zero CLI callers and served only the web UI demo.
Verification and retros were `not_implemented` stubs in real mode;
sessions had an in-memory implementation but no CLI usage. Removing
them shrinks the API surface and eliminates ~9,000 lines of dead code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:52:51 -04:00
Bryan Helmkamp
4352508f55
Merge remote-tracking branch 'origin/main' 2026-04-06 16:57:45 -04:00
Bryan Helmkamp
839d0187d1
docs 2026-04-06 16:57:34 -04:00
Bryan Helmkamp
097c86024f
chore: remove skill install command and GET /runs/{id}/verification endpoint
Dead feature cleanup: `skill install` was hidden/experimental and never
graduated; the run verification endpoint was only implemented in demo mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:38:03 -04:00
Bryan Helmkamp
62f6f9b100
feat(system): add server-backed system commands 2026-04-06 16:10:06 -04:00
Bryan Helmkamp
a740b2d395
refactor(settings): make settings command server-backed
Default `fabro settings` now resolves effective runtime settings against the
selected server, while `--local` preserves local-only inspection. This also
extracts shared settings resolution logic so CLI output, manifest preparation,
and the `/api/v1/settings` contract stay aligned.
2026-04-06 13:34:28 -04:00
Bryan Helmkamp
48a5d4cc58
plan 2026-04-06 10:43:41 -04:00
Bryan Helmkamp
f444fc783a
plans 2026-04-06 10:17:30 -04:00
Bryan Helmkamp
35593d0a11
refactor(storage): unify scratch paths and key schema
- centralize FABRO_HOME and storage path resolution in fabro-config
- rename store types, extract ArtifactStore, and simplify run key layout
- switch run scratch to scratch/, remove RuntimeState, and refresh docs/clients
2026-04-06 10:13:37 -04:00
Bryan Helmkamp
beec4c8dff
refactor(cli): make run-adjacent commands server-only 2026-04-06 06:12:16 -04:00
Bryan Helmkamp
36ff3c5377
refactor(config): unify machine config as settings.toml
Consolidate CLI and server machine defaults under settings.toml,
including loader renames, writer preservation fixes, same-machine
manifest handling, and docs/test updates for the new config model.
2026-04-05 23:55:28 -04:00
Bryan Helmkamp
aa15e7a1be
chore: remove stale SQLite references after retirement
SQLite was retired as the server metadata store in 1ed3861e.
Remove dead sqlx workspace dep, better-sqlite3 trustedDependencies,
and update docs that still referenced SQLite persistence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 21:29:24 -04:00
Bryan Helmkamp
bd70f805be
refactor(server): submit runs and preflight via manifests 2026-04-05 20:07:10 -04:00