Bryan Helmkamp 3eef9ee928 Model Test Bounded Concurrency Implementation Plan (#204) ... This change makes bulk `fabro model test` run configured model checks concurrently instead of serially. A new `--jobs/-j` flag (defaulting to 4, minimum 1) controls the concurrency bound; the single-model path (`--model <MODEL>`) is unaffected. Under the hood, the serial `for` loop over configured models is replaced with a `futures::stream::buffer_unordered(jobs)` pipeline that clones the shared-state `Client` per request. Completed results carry their original list index and are sorted before rendering, so final stdout table rows and JSON output remain in listing order regardless of which requests finish first. Three new integration tests verify the concurrency behavior using an inline Axum harness with a `ConcurrencyGate` barrier. The gate holds all in-flight requests until the expected number arrive simultaneously, then releases them, letting tests assert `max_in_flight` exactly rather than relying on timing. The ordering test goes further by assigning reverse-listing response delays so the last-listed model always finishes first; if the index sort were dropped, the JSON result order would invert and the assertion would fail. A 15-second gate timeout ensures a regression to serial execution surfaces as a clear `max_in_flight == 1` failure rather than a hung test. Existing behavior is fully preserved: unconfigured models are still skipped without a POST, a configured model returning `skip` after listing is still a failure, `--deep` uses the same `--jobs` value, and `--jobs 1` reproduces the previous serial behavior for users hitting provider rate limits. ### Fabro Details <details> <summary>Ran 9 stages in 30m 52s for $19.61</summary> | Stage | Duration | Cost | Retries | |---|---|---|---| | start | 0s | – | 0 | | toolchain | 1s | – | 0 | | preflight_compile | 2m 8s | – | 0 | | preflight_lint | 2m 21s | – | 0 | | implement | 8m 56s | $3.87 | 0 | | simplify_opus | 7m 55s | $1.43 | 0 | | simplify_gpt | 6m 58s | $14.32 | 0 | | verify | 1m 49s | – | 0 | | fmt | 2s | – | 0 | | **Total** | **30m 52s** | **$19.61** | **0** | </details> <details> <summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15 edges)</summary> ```dot digraph ImplementPlan { graph [ goal="Implement and simplify", model_stylesheet=" * { model: claude-opus-4-7; } " ] rankdir=LR start [shape=Mdiamond, label="Start"] exit [shape=Msquare, label="Exit"] toolchain [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0] preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0] preflight_lint [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0] fix_lints [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3] implement [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."] simplify_opus [label="Simplify (Opus)", prompt="@prompts/simplify.md"] simplify_gpt [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"] verify [label="Verify", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1 && cargo dev docs refresh 2>&1 && cargo dev docs check 2>&1", goal_gate=true, retry_target="fixup"] fixup [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings, test failures, and generated docs errors.", max_visits=3] fmt [label="Format", shape=parallelogram, script="cargo +nightly-2026-04-14 fmt --all 2>&1", max_retries=0] start -> toolchain toolchain -> preflight_compile [condition="outcome=succeeded"] toolchain -> exit preflight_compile -> preflight_lint [condition="outcome=succeeded"] preflight_compile -> exit preflight_lint -> implement [condition="outcome=succeeded"] preflight_lint -> fix_lints fix_lints -> preflight_lint implement -> simplify_opus -> simplify_gpt -> verify verify -> fmt [condition="outcome=succeeded"] verify -> fixup fixup -> verify fmt -> exit } ``` </details> ⚒️ Generated with [Fabro](https://fabro.sh) --------- Co-authored-by: Fabro <noreply@fabro.sh>		2026-05-04 13:46:44 -04:00
..
crates	Model Test Bounded Concurrency Implementation Plan (#204)	2026-05-04 13:46:44 -04:00
packages/fabro-api-client	fix(billing): render non-LLM run stages	2026-05-03 22:49:33 -04:00