mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-09-05 08:10:39 +00:00
3eef9ee928
This change makes bulk `fabro model test` run configured model checks
concurrently instead of serially. A new `--jobs/-j` flag (defaulting to
4, minimum 1) controls the concurrency bound; the single-model path
(`--model <MODEL>`) is unaffected. Under the hood, the serial `for` loop
over configured models is replaced with a
`futures::stream::buffer_unordered(jobs)` pipeline that clones the
shared-state `Client` per request. Completed results carry their
original list index and are sorted before rendering, so final stdout
table rows and JSON output remain in listing order regardless of which
requests finish first.
Three new integration tests verify the concurrency behavior using an
inline Axum harness with a `ConcurrencyGate` barrier. The gate holds all
in-flight requests until the expected number arrive simultaneously, then
releases them, letting tests assert `max_in_flight` exactly rather than
relying on timing. The ordering test goes further by assigning
reverse-listing response delays so the last-listed model always finishes
first; if the index sort were dropped, the JSON result order would
invert and the assertion would fail. A 15-second gate timeout ensures a
regression to serial execution surfaces as a clear `max_in_flight == 1`
failure rather than a hung test.
Existing behavior is fully preserved: unconfigured models are still
skipped without a POST, a configured model returning `skip` after
listing is still a failure, `--deep` uses the same `--jobs` value, and
`--jobs 1` reproduces the previous serial behavior for users hitting
provider rate limits.
### Fabro Details
<details>
<summary>Ran 9 stages in 30m 52s for $19.61</summary>
| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 1s | – | 0 |
| preflight_compile | 2m 8s | – | 0 |
| preflight_lint | 2m 21s | – | 0 |
| implement | 8m 56s | $3.87 | 0 |
| simplify_opus | 7m 55s | $1.43 | 0 |
| simplify_gpt | 6m 58s | $14.32 | 0 |
| verify | 1m 49s | – | 0 |
| fmt | 2s | – | 0 |
| **Total** | **30m 52s** | **$19.61** | **0** |
</details>
<details>
<summary>Ran <code>ImplementPlan.fabro</code> (12 nodes and 15
edges)</summary>
```dot
digraph ImplementPlan {
graph [
goal="Implement and simplify",
model_stylesheet="
* { model: claude-opus-4-7; }
"
]
rankdir=LR
start [shape=Mdiamond, label="Start"]
exit [shape=Msquare, label="Exit"]
toolchain [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
preflight_lint [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0]
fix_lints [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
implement [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
simplify_opus [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
simplify_gpt [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"]
verify [label="Verify", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1 && cargo dev docs refresh 2>&1 && cargo dev docs check 2>&1", goal_gate=true, retry_target="fixup"]
fixup [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings, test failures, and generated docs errors.", max_visits=3]
fmt [label="Format", shape=parallelogram, script="cargo +nightly-2026-04-14 fmt --all 2>&1", max_retries=0]
start -> toolchain
toolchain -> preflight_compile [condition="outcome=succeeded"]
toolchain -> exit
preflight_compile -> preflight_lint [condition="outcome=succeeded"]
preflight_compile -> exit
preflight_lint -> implement [condition="outcome=succeeded"]
preflight_lint -> fix_lints
fix_lints -> preflight_lint
implement -> simplify_opus -> simplify_gpt -> verify
verify -> fmt [condition="outcome=succeeded"]
verify -> fixup
fixup -> verify
fmt -> exit
}
```
</details>
⚒️ Generated with [Fabro](https://fabro.sh)
---------
Co-authored-by: Fabro <noreply@fabro.sh>
|
||
|---|---|---|
| .. | ||
| fabro-agent | ||
| fabro-api | ||
| fabro-auth | ||
| fabro-checkpoint | ||
| fabro-cli | ||
| fabro-client | ||
| fabro-config | ||
| fabro-core | ||
| fabro-dev | ||
| fabro-devcontainer | ||
| fabro-dump | ||
| fabro-github | ||
| fabro-graphviz | ||
| fabro-hooks | ||
| fabro-http | ||
| fabro-install | ||
| fabro-interview | ||
| fabro-llm | ||
| fabro-macros | ||
| fabro-mcp | ||
| fabro-model | ||
| fabro-oauth | ||
| fabro-options-metadata | ||
| fabro-proc | ||
| fabro-redact | ||
| fabro-retro | ||
| fabro-sandbox | ||
| fabro-server | ||
| fabro-slack | ||
| fabro-spa | ||
| fabro-static | ||
| fabro-store | ||
| fabro-telemetry | ||
| fabro-template | ||
| fabro-test | ||
| fabro-tracker | ||
| fabro-types | ||
| fabro-util | ||
| fabro-validate | ||
| fabro-vault | ||
| fabro-workflow | ||