Commit graph

40 commits

Author SHA1 Message Date
Bryan Helmkamp
90b373c366
fix(test): eliminate recovery, cancel, and label flakes
Stabilize the recovery scenario around rebuilt metadata timing and node
ordinals, make in-process run cancellation converge on a cancelled
reason, and keep the label assertion unit test out of the shared
TestContext session lifecycle.
2026-04-08 10:13:17 -04:00
Bryan Helmkamp
e37d20bd6f
test fix 2026-04-08 07:55:32 -04:00
Bryan Helmkamp
00d2a1e5b2
Merge remote-tracking branch 'origin/main' 2026-04-07 20:45:12 -04:00
Bryan Helmkamp
4bcd8f7647
refactor(scratch): remove stale scratch file refs
Drop scratch-only compatibility paths and legacy test scaffolding now that
SlateDB-backed state is authoritative. This removes scratch file fallbacks,
updates docs and UI labels, and moves tests onto durable store-backed helpers.
2026-04-07 20:40:06 -04:00
Bryan Helmkamp
c5f68d4eda
fix(server): skip stale worker cleanup after resume
Guard server worker cleanup against superseded subprocesses so rewind and
resume flows do not append a synthetic failure from an older worker. Update
CLI snapshots for the current interview events and give the shared test
session lock more time to cover daemon startup and shutdown.
2026-04-07 20:11:17 -04:00
Bryan Helmkamp
0d48f2c255
fix(artifacts): harden object-backed upload rollout
Tighten the worker upload path so object-backed runs only fail when an
artifact upload is actually attempted without a token, and update CLI
snapshots for the new artifact storage metadata.

Fold in the workspace test and clippy fixes needed to verify the final
artifact upload implementation cleanly across Rust and web targets.
2026-04-07 17:45:49 -04:00
Bryan Helmkamp
a2ab0afc86
perf(nextest): keep shared test server state in memory
Default test daemons now opt into an in-memory object store and test
helpers carry explicit run ids instead of rediscovering runs from
shared state.

This also disables the disk-backed store dump integration tests until
store dump is routed through the server's live store handles.
2026-04-07 11:39:40 -04:00
Bryan Helmkamp
099362cd42
fix(nextest): let shared test server reap run workers
Give the harness longer to stop the shared test server than the server
itself uses to shut down active run workers. This prevents session cleanup
from SIGKILLing the server before it can terminate worker process groups,
which was leaving orphaned `fabro <run> running` subprocesses behind.
2026-04-07 09:52:12 -04:00
Bryan Helmkamp
9d9ebd529f
test(nextest): share one session server by default
Eagerly start one shared test server per nextest session and point default
TestContext commands at that session socket instead of leaking per-test
daemons keyed by FABRO_STORAGE_DIR. Add isolated_server() for tests that
need an explicit separate daemon, and tighten the ps filtering test so it
still proves the contract without timing out under full-suite load.
2026-04-07 09:44:34 -04:00
Bryan Helmkamp
cb58b18385
refactor(cli): separate local socket and storage defaults
Keep local server targeting based on explicit server targets instead of
implicitly deriving a socket from storage_dir. This makes ~/.fabro/fabro.sock
the default local socket again, keeps storage under ~/.fabro/storage, threads
FABRO_CONFIG through server autostart paths, and updates the CLI test harness
for the new split.
2026-04-06 11:57:14 -04:00
Bryan Helmkamp
35593d0a11
refactor(storage): unify scratch paths and key schema
- centralize FABRO_HOME and storage path resolution in fabro-config
- rename store types, extract ArtifactStore, and simplify run key layout
- switch run scratch to scratch/, remove RuntimeState, and refresh docs/clients
2026-04-06 10:13:37 -04:00
Bryan Helmkamp
260460aebf
refactor(cli): remove global execution mode 2026-04-05 15:20:00 -04:00
Bryan Helmkamp
0bdcb4df77
Merge remote-tracking branch 'origin/main' 2026-04-05 14:44:35 -04:00
Bryan Helmkamp
b2ef792a85
refactor(cli): make model commands server-canonical 2026-04-05 14:09:24 -04:00
Bryan Helmkamp
74cad460b0
fix(test): stop leaked server processes and add test names to temp dirs
Replace subprocess-based server stop (fabro server stop) with direct
SIGTERM/SIGKILL via fabro_proc, eliminating silent failures under
nextest parallelism that left orphaned daemon processes. Use test
function name as temp dir prefix (.ft-<name>-) so leaked processes
are identifiable by test, truncated to 16 chars for Unix socket
path limits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 14:07:23 -04:00
Bryan Helmkamp
bfd359dbce
test: clean up config-owned fabro daemons 2026-04-05 12:59:20 -04:00
Bryan Helmkamp
d644ca1e95
test: standardize no-proxy localhost HTTP clients 2026-04-05 12:17:29 -04:00
Bryan Helmkamp
fa7de26188
fix(cli): restore green server-backed test checkpoint 2026-04-05 03:41:10 -04:00
Bryan Helmkamp
2f45976e39
fix(cli): remove remaining local test store access 2026-04-05 03:34:26 -04:00
Bryan Helmkamp
9597a115b7
fix(test): reduce shared-daemon cli overhead 2026-04-05 00:56:08 -04:00
Bryan Helmkamp
2af2ff0068
refactor(cli): route run state through the shared server daemon
Move durable run access and execution control onto the server-backed client,
canonicalize run APIs under /api/v1/runs, and switch CLI integration tests
to a shared test daemon/storage model with shared-state-safe assertions.
2026-04-05 00:15:33 -04:00
Bryan Helmkamp
b5f5162c3a
Rename SessionConfig to SessionOptions and McpServerConfig to McpServerSettings
Aligns naming with the convention that "Config" is for file-level configuration
while "Options" and "Settings" describe runtime parameters. Also applies
rustfmt formatting fixes in web_auth.rs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 06:47:42 -07:00
Bryan Helmkamp
a6e83f551e
Expand OpenAI twin coverage across integration tests
Add shared twin scenario helpers and use them to cover OpenAI-backed
CLI, agent parity, workflow, and exec integration paths. This brings the
worktree implementation back into the main checkout as a single commit.
2026-04-01 11:03:34 -04:00
Bryan Helmkamp
fea64a105b
Integrate twin-github for fabro-github tests
Add the stripped twin-github test server to the workspace, wire it through
fabro-test, and cover fabro-github's real HTTP auth and pull-request flows
with twin-backed integration tests. This also refactors the GitHub helper
entry points to take explicit base URLs so tests and callers share the same
request path.
2026-04-01 09:48:33 -04:00
Bryan Helmkamp
6383e7037b
Add twin test mode for OpenAI E2E tests
Integrate twin-openai (fake OpenAI server) into the workspace and wire
it into the e2e_test macro so OpenAI tests can run without real API
credentials. The twin server starts in-process via OnceLock on first use
and provides per-test isolation through bearer-token namespacing.

Changes:
- Add Twin as default TestMode, replacing Off (gating now via #[ignore])
- Extend #[e2e_test] macro with `twin` requirement for twin-only,
  live-only, and dual-mode (twin + live) test gating
- Add e2e_openai!() macro returning (base_url, api_key)
- Convert openai_complete and openai_gpt_5_3_codex_complete to dual-mode
- Add new openai_server_error twin-only test with scripted 500 error
- Standardize axum 0.8 as workspace dependency across all crates
- Relax twin-openai ResponsesRequest to accept unknown fields via flatten

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:36:48 -04:00
Bryan Helmkamp
b5e5b91d51
Default TestContext::command() working directory to temp_dir
Tests were flaky because command() inherited the real repo as the
working directory. When the repo was clean, detached runs attempted
git worktree creation against it, sometimes failing and injecting
extra warning lines into snapshots.

Now command() defaults to the non-git temp_dir, eliminating this
class of flakiness. Tests needing a specific directory override with
.current_dir(). Also canonicalizes fixture paths and adds a
[FIXTURES] snapshot filter via test_context!() macro.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:36:48 -04:00
Bryan Helmkamp
b9bdc81028
Fix workspace clippy issues in fabro-test 2026-03-31 13:33:20 -04:00
Bryan Helmkamp
fb6f0eae1e
Centralize E2E test env var handling 2026-03-31 09:30:38 -04:00
Bryan Helmkamp
d064b6c510
Stabilize merged workspace validation 2026-03-30 18:57:06 -04:00
Bryan Helmkamp
0313a1f4ee
Merge remote-tracking branch 'origin/main'
# Conflicts:
#	lib/crates/fabro-cli/tests/it/cmd/attach.rs
#	lib/crates/fabro-cli/tests/it/cmd/create.rs
#	lib/crates/fabro-cli/tests/it/cmd/resume.rs
#	lib/crates/fabro-cli/tests/it/cmd/start.rs
2026-03-30 18:26:57 -04:00
Bryan Helmkamp
c8a398d527
Add CLI integration test coverage 2026-03-30 18:23:24 -04:00
Bryan Helmkamp
967b419339
Rename sandbox subcommand tests to follow namespace convention
Move ssh.rs, cp.rs, preview.rs to sandbox_ssh.rs, sandbox_cp.rs,
sandbox_preview.rs to match the naming convention used by other
namespaced tests (e.g. pr_close.rs, system_prune.rs). Use
context.command() + args instead of one-off helpers. Remove redundant
config_show.rs (duplicate of config.rs).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 16:17:39 -04:00
Bryan Helmkamp
0346edd112
Refactor run CLI tests to use snapshots 2026-03-30 16:17:38 -04:00
Bryan Helmkamp
615636c4e7
Move cp, ssh, preview under fabro sandbox namespace
These commands all operate on a run's sandbox environment. Grouping them
under `fabro sandbox` makes the mental model clear and avoids confusion
with `fabro asset cp`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 12:00:37 -04:00
Bryan Helmkamp
69510cd016
Fix warnings and stabilize dry-run snapshots 2026-03-30 11:51:25 -04:00
Bryan Helmkamp
a102557d32
Rename config show command to settings 2026-03-30 11:41:18 -04:00
Bryan Helmkamp
3dee953cd2
Add git_init() helper to TestContext
Initializes a git repo in temp_dir. Replaces the local init_git_repo()
helper in repo.rs tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:40:31 -04:00
Bryan Helmkamp
fa43f8f780
Add write_temp/write_home helpers to TestContext
Add convenience methods that write a file under temp_dir or home_dir,
auto-creating parent directories. Returns &Self for chaining.

Apply write_home in config.rs fixture helpers and standalone tests,
replacing manual create_dir_all + write boilerplate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:40:31 -04:00
Bryan Helmkamp
d1a969510e
Move cli.rs and scenario.rs tests into per-subcommand cmd/ files
Redistribute all tests from the monolithic cli.rs (2190 lines) and the
standalone tests from scenario.rs into their respective cmd/ files,
completing the migration to the one-file-per-subcommand structure.

- Delete cli.rs entirely; move tests to cmd/{run,config,llm,exec,doctor,serve}.rs
- Move scenario.rs standalone tests to cmd/{repo,secret,workflow,doctor}.rs
- Slim scenario.rs to only 6 parametrized E2E workflow scenarios + run lifecycle
- Create new cmd/serve.rs and cmd/workflow.rs modules
- Remove duplicate tests already covered by snapshot tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:37:24 -04:00
Bryan Helmkamp
d4a91eb29d
Replace trycmd tests with fabro-test crate and insta snapshots
Adopt uv's testing pattern: a shared `fabro-test` crate with TestContext
and fabro_snapshot! macro, plus one test file per subcommand under
tests/it/cmd/. This replaces the trycmd-based tests which were hard to
read and didn't compose well with programmatic assertions.

- Create lib/crates/fabro-test with TestContext, run_and_format,
  apply_filters, INSTA_FILTERS, and test_context!/fabro_snapshot! macros
- Add 42 snapshot tests across 16 subcommand files
- Delete trycmd.rs and all tests/cmd/ trycmd files
- Remove trycmd dependency, add fabro-test dev-dependency

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:28:41 -04:00