mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
bd837fc0f0
## Summary
Adds `POST /api/v1/providers/test` so API, CLI, and UI callers can check
LLM provider health without parsing `/health/diagnostics`. The endpoint
tests every configured provider once using the catalog probe model and
returns typed, per-provider results with an aggregate summary — all at
HTTP 200, with provider failures expressed in the body.
This PR also adds `scheduler_slots_used` to `SystemRunCounts` to
distinguish runs occupying concurrency slots from all "active" runs
(e.g. runs blocked waiting for human input count as active but do not
hold a scheduler slot).
### What changed and why
**Provider probe logic** (`diagnostics.rs`)
The inline probe loop inside `check_llm_providers` was extracted into
`test_llm_providers` / `probe_single_provider`, which both the new
endpoint and the existing diagnostics check now share. The extraction
preserves the diagnostics output format: `diagnostic_detail` (a
`#[serde(skip)]` field) carries the richer context string used for the
`LLM Providers` section, while `error_message` carries the redacted,
public-facing error.
Key decisions:
- `ProviderProbeStatus` is `ok | error` only — no `skip`, because v1
only iterates configured providers.
- `model_id` is nullable so auth/registration failures (where no probe
was sent) can be expressed cleanly.
- API key values appearing in upstream error responses are passed
through `redact_string` before being stored in `error_message`.
**Route** (`handler/models.rs`)
`.route("/providers/test", post(test_providers))` added alongside
`/providers`, protected by the same `RequiredUser` extractor.
**`scheduler_slots_used`** (`handler/system.rs`, `server.rs`)
The status predicate (`Starting | Running | Blocked | Paused`) was
already duplicated between the scheduler loop and `get_system_info`.
It's now a named function `counts_toward_scheduler_capacity`, used in
both places and in the new `SystemRunCounts` field. The web UI
monitoring panel was updated to display "slots used" instead of
"active."
**Generated clients**
OpenAPI spec updated; Rust and TypeScript clients regenerated. New
TypeScript types: `ProviderTestList`, `ProviderTestResult`,
`ProviderTestStatus`, `ProviderTestSummary`.
### Plan Summary
- Add `testProviders` OpenAPI operation and `ProviderTestList` /
supporting schemas to `fabro-api.yaml`.
- Extract shared `test_llm_providers` from `check_llm_providers` in
`diagnostics.rs`; keep diagnostics output identical.
- Wire `POST /providers/test` handler in `handler/models.rs`.
- Add `scheduler_slots_used` to `SystemRunCounts` and extract
`counts_toward_scheduler_capacity` predicate.
- Regenerate Rust and TypeScript API clients.
- Add integration tests covering: no providers, successful probe, auth
failure (no upstream call), registration failure, mixed catalog order,
and API key non-leakage.
### Fabro Details
<details>
<summary>Ran 8 stages in 53m 33s for $40.83</summary>
| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 1s | – | 0 |
| preflight_compile | 2m 8s | – | 0 |
| preflight_lint | 2m 19s | – | 0 |
| implement | 23m 56s | $30.73 | 0 |
| simplify_opus | 12m 53s | $5.80 | 0 |
| simplify_gpt | 2m 40s | $4.30 | 0 |
| verify | 9m 5s | – | 0 |
| **Total** | **53m 33s** | **$40.83** | **0** |
</details>
<details>
<summary>Ran <code>ImplementPlan.fabro</code> (11 nodes and 14
edges)</summary>
```dot
digraph ImplementPlan {
graph [
goal="Implement and simplify",
model_stylesheet="
* { model: claude-opus-4-7; }
"
]
rankdir=LR
start [shape=Mdiamond, label="Start"]
exit [shape=Msquare, label="Exit"]
toolchain [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
preflight_lint [label="Preflight Lint", shape=parallelogram, script="cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1", max_retries=0]
fix_lints [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
implement [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD.", model="gpt-55", reasoning_effort="xhigh"]
simplify_opus [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
simplify_gpt [label="Simplify (GPT-55)", prompt="@prompts/simplify.md", model="gpt-55"]
verify [label="Verify", shape=parallelogram, script="git fetch origin main 2>&1 && git merge --no-edit --no-stat origin/main 2>&1 && cargo +nightly-2026-04-14 fmt --all 2>&1 && cargo dev docs refresh 2>&1 && cargo +nightly-2026-04-14 fmt --check --all 2>&1 && { command -v rg >/dev/null 2>&1 || { echo 'rg is required for verify'; exit 127; }; } && ! rg -n 'AuthMode::Disabled|RunAuthMethod|RunSubjectProvenance|\bActorRef\b|\bActorKind\b|AuthenticatedSubject|AuthenticatedService|AuthorizeRunScoped|AuthorizeRunBlob|AuthorizeStageArtifact|AuthorizeCommandLog|auth_method\s*==\s*\"disabled\"' lib/crates apps lib/packages docs/public/api-reference/fabro-api.yaml 2>&1 && cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings 2>&1 && cargo nextest run --workspace --status-level slow --profile ci 2>&1 && cargo dev docs check 2>&1 && bun install --frozen-lockfile 2>&1 && (cd apps/fabro-web && bun run typecheck) 2>&1 && (cd apps/fabro-web && bun run test) 2>&1 && (cd lib/packages/fabro-api-client && bun run typecheck) 2>&1 && cargo dev build -- -p fabro-cli --release 2>&1", goal_gate=true, retry_target="fixup"]
fixup [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all format, clippy, Rust test, docs, TypeScript typecheck/test, and build failures.", max_visits=3]
start -> toolchain
toolchain -> preflight_compile [condition="outcome=succeeded"]
toolchain -> exit
preflight_compile -> preflight_lint [condition="outcome=succeeded"]
preflight_compile -> exit
preflight_lint -> implement [condition="outcome=succeeded"]
preflight_lint -> fix_lints
fix_lints -> preflight_lint
implement -> simplify_opus -> simplify_gpt -> verify
verify -> exit [condition="outcome=succeeded"]
verify -> fixup
fixup -> verify
}
```
</details>
⚒️ Generated with [Fabro](https://fabro.sh)
---------
Co-authored-by: Fabro <noreply@fabro.sh>
|
||
|---|---|---|
| .. | ||
| client-sdks.mdx | ||
| fabro-api.yaml | ||
| overview.mdx | ||