Commit graph

12 commits

Author SHA1 Message Date
Bryan Helmkamp
3eef9ee928
Model Test Bounded Concurrency Implementation Plan (#204)
This change makes bulk `fabro model test` run configured model checks
concurrently instead of serially. A new `--jobs/-j` flag (defaulting to
4, minimum 1) controls the concurrency bound; the single-model path
(`--model <MODEL>`) is unaffected. Under the hood, the serial `for` loop
over configured models is replaced with a
`futures::stream::buffer_unordered(jobs)` pipeline that clones the
shared-state `Client` per request. Completed results carry their
original list index and are sorted before rendering, so final stdout
table rows and JSON output remain in listing order regardless of which
requests finish first.

Three new integration tests verify the concurrency behavior using an
inline Axum harness with a `ConcurrencyGate` barrier. The gate holds all
in-flight requests until the expected number arrive simultaneously, then
releases them, letting tests assert `max_in_flight` exactly rather than
relying on timing. The ordering test goes further by assigning
reverse-listing response delays so the last-listed model always finishes
first; if the index sort were dropped, the JSON result order would
invert and the assertion would fail. A 15-second gate timeout ensures a
regression to serial execution surfaces as a clear `max_in_flight == 1`
failure rather than a hung test.

Existing behavior is fully preserved: unconfigured models are still
skipped without a POST, a configured model returning `skip` after
listing is still a failure, `--deep` uses the same `--jobs` value, and
`--jobs 1` reproduces the previous serial behavior for users hitting
provider rate limits.

### Fabro Details

<details>
<summary>Ran 9 stages in 30m 52s for $19.61</summary>

| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 1s | – | 0 |
| preflight_compile | 2m 8s | – | 0 |
| preflight_lint | 2m 21s | – | 0 |
| implement | 8m 56s | $3.87 | 0 |
| simplify_opus | 7m 55s | $1.43 | 0 |
| simplify_gpt | 6m 58s | $14.32 | 0 |
| verify | 1m 49s | – | 0 |
| fmt | 2s | – | 0 |
| **Total** | **30m 52s** | **$19.61** | **0** |

</details>

<details>
<summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15
edges)</summary>

```dot
digraph ImplementPlan {
    graph [
        goal="Implement and simplify",
        model_stylesheet="
            * { model: claude-opus-4-7; }
        "
    ]
    rankdir=LR

    start [shape=Mdiamond, label="Start"]
    exit  [shape=Msquare, label="Exit"]

    toolchain         [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
    preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
    preflight_lint    [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0]
    fix_lints         [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
    implement         [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
    simplify_opus     [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
    simplify_gpt      [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"]
    verify            [label="Verify", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1 && cargo dev docs refresh 2>&1 && cargo dev docs check 2>&1", goal_gate=true, retry_target="fixup"]
    fixup             [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings, test failures, and generated docs errors.", max_visits=3]
    fmt               [label="Format", shape=parallelogram, script="cargo +nightly-2026-04-14 fmt --all 2>&1", max_retries=0]

    start -> toolchain
    toolchain -> preflight_compile [condition="outcome=succeeded"]
    toolchain -> exit
    preflight_compile -> preflight_lint [condition="outcome=succeeded"]
    preflight_compile -> exit
    preflight_lint -> implement [condition="outcome=succeeded"]
    preflight_lint -> fix_lints
    fix_lints -> preflight_lint
    implement -> simplify_opus -> simplify_gpt -> verify
    verify -> fmt   [condition="outcome=succeeded"]
    verify -> fixup
    fixup -> verify
    fmt -> exit
}

```

</details>

⚒️ Generated with [Fabro](https://fabro.sh)

---------

Co-authored-by: Fabro <noreply@fabro.sh>
2026-05-04 13:46:44 -04:00
Bryan Helmkamp
cea1fa739d
refactor(run-projection): use stage vocabulary 2026-05-01 19:56:22 -04:00
Bryan Helmkamp
1b30a2803a
fix(cli): remove dev-token env auth override 2026-05-01 15:04:01 -04:00
Bryan Helmkamp
b96647314c
docs(deployment): split into local vs self-hosted, drop Render/Fly/DO
Reframe the deployment docs around the actual product story: Fabro
runs as a server, and the only deployment question is where that
server runs (laptop vs self-hosted Docker). Drop the Render, Fly.io,
and DigitalOcean guides and their config files; keep Railway as the
managed shortcut.

- New: administration/deployment.mdx (overview, two-mode framing)
- New: administration/self-host-docker.mdx (compose-first how-to)
- Move: administration/deploy-server.mdx -> reference/server-operations.mdx
  (it was operational reference, not deploy guidance)
- Delete: deploy-render.mdx, deploy-fly-io.mdx, deploy-digital-ocean.mdx
- Delete: render.yaml, fly.toml, railway.toml, Dockerfile.deploy
- docker-compose.yaml: load .env if present so users can drive the
  stack from a single env file end-to-end
- Update internal links and the docs-test that pinned the old path
2026-05-01 09:26:35 -04:00
Bryan Helmkamp
157ec03a18
docs: sync public docs to recent runtime changes
Reflect Docker as the default sandbox provider, add `skip_clone` for
clone-based providers, document the `[run.sandbox.docker]` config
table, and update tutorial command lines from `files-internal/...` to
`docs/internal/...`. Bump the docs skill watermark to the latest synced
commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 08:35:53 -04:00
Bryan Helmkamp
d65c1fa635
refactor(types): remove stage status compatibility 2026-04-30 06:48:47 -04:00
Bryan Helmkamp
f16391485b
refactor(workflow): update stage outcome semantics 2026-04-30 06:06:51 -04:00
Bryan Helmkamp
24b8f1ec04
fix(docs): use MDX comment syntax for generated fences
Mintlify parses pages as MDX and rejects HTML-style `<!-- ... -->`
comments, which broke the docs deployment on cli.mdx with a parse
error. Switch the generator fences (and the matching markers in the
two reference pages and the dev test fixtures) to `{/* ... */}` so
Mintlify can parse them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 01:27:43 -04:00
Bryan Helmkamp
f676fa779b
fix(ci): clean up clippy warnings and refresh CLI docs
Drop async from validate::run after the preflight refactor removed all
awaits, replace absolute paths and a one-liner helper in
manifest_validation, swap a redundant to_path_buf for clone in a test,
and regenerate cli.mdx so docs check stays green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 01:03:16 -04:00
Bryan Helmkamp
ab9b28875b
fix: close sandbox-native metadata gaps
Ensure local runs use the worktree checkpoint path by default, expose source and sandbox paths in API/web surfaces, and remove dead fork/rewind push controls. Update docs for clone-based sandboxes and durable checkpoint timelines.
2026-04-28 08:05:18 -07:00
Bryan Helmkamp
cdd46b4fa8
Make git metadata sandbox-native 2026-04-27 21:43:15 -07:00
Bryan Helmkamp
283eab181f
refactor(docs): split docs/ into public/ and internal/
Invert the docs convention so the Mintlify-published site lives under
docs/public/ and internal artifacts (strategy docs, brainstorms, plans,
etc.) sit at docs/ root or docs/internal/. Tools that default to writing
into docs/ now land in the catch-all instead of leaking into the
published tree.

- Move Mintlify content (administration/, agents/, api-reference/,
  changelog/, core-concepts/, examples/, execution/, getting-started/,
  human-tools/, integrations/, languages/, reference/, tutorials/,
  workflows/, images/, logo/, docs.json, favicon.svg, dot-highlight.js)
  into docs/public/.
- Collapse docs-internal/ into docs/internal/.
- Update Rust path references (fabro-api/build.rs, fabro-server,
  fabro-dev), TypeScript generator arg, CI path filters, clippy.toml
  reasons, AGENTS.md/CLAUDE.md, and README.md image refs.

Mintlify dashboard project root must be updated to docs/public/ in a
follow-up. .mintignore move/trim and .claude/skills/ updates land in a
separate commit.
2026-04-27 07:21:13 -07:00