scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-09-06 08:18:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
fabro/lib/crates/fabro-cli/tests/cmd/model/bare.trycmd
brynary-fabro[bot] 4b0a4f773e Add node-level model validation + missing catalog aliases (#110) 
This PR addresses two related issues that caused `fabro run` to fail at
runtime with an unhelpful LLM error when using model names like
`gpt-54`: the model catalog was missing hyphenated aliases, and the
validator had no rule to catch unknown model names on workflow nodes.

The catalog changes add `gpt-54`, `gpt-54-pro`, and `gpt-54-mini` as
recognized aliases alongside the existing `gpt54`-style variants for the
`gpt-5.4` model family. This makes the hyphenated form a valid input
that resolves correctly at both validation and runtime, and the updated
`fabro model list` output and insta snapshots reflect the expanded alias
columns. Three new alias-resolution unit tests confirm each mapping.

On the validation side, a new `NodeModelKnownRule` mirrors the existing
`StylesheetModelKnownRule` but iterates over graph nodes instead of
stylesheet declarations, emitting `Severity::Warning` diagnostics with
the relevant `node_id` when a node's `model` or `provider` attribute
doesn't match anything in the catalog. To avoid duplication, the inline
validation logic in `StylesheetModelKnownRule` was refactored into
shared `check_model_known` and `check_provider_known` helpers that both
rules now call. Five unit tests cover the valid-model, unknown-model,
alias, unknown-provider, and no-attribute cases. An unrelated dead-code
helper in `git.rs` was also removed as part of the cleanup.

### Fabro Details

<details>
<summary>Ran 11 stages in 12m 36s for $2.85</summary>

| Stage | Duration | Cost | Retries |
|---|---|---|---|
| start | 0s | – | 0 |
| toolchain | 0s | – | 0 |
| preflight_compile | 58s | – | 0 |
| preflight_lint | 10s | – | 0 |
| implement | 2m 56s | $1.13 | 0 |
| simplify_opus | 4m 12s | $1.13 | 0 |
| simplify_gpt | 0s | – | 0 |
| verify | 10s | – | 0 |
| fixup | 2m 27s | $0.59 | 0 |
| verify | 10s | – | 0 |
| fmt | 0s | – | 0 |
| **Total** | **12m 36s** | **$2.85** | **0** |

</details>

<details>
<summary>Ran <code>ImplementAndSimplify.fabro</code> (12 nodes and 15
edges)</summary>

```dot
digraph ImplementAndSimplify {
    graph [
        goal="Implement and simplify",
        model_stylesheet="
            * { backend: api; model: claude-opus-4-6;}
        "
    ]
    rankdir=LR

    start [shape=Mdiamond, label="Start"]
    exit  [shape=Msquare, label="Exit"]

    toolchain         [label="Toolchain", shape=parallelogram, script="command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", max_retries=0]
    preflight_compile [label="Preflight Compile", shape=parallelogram, script="cargo check -q --workspace 2>&1", max_retries=0]
    preflight_lint    [label="Preflight Lint", shape=parallelogram, script="cargo clippy -q --workspace -- -D warnings 2>&1", max_retries=0]
    fix_lints         [label="Fix Lints", prompt="The preflight lint step failed. Read the build output from context and fix all clippy lint warnings.", max_visits=3]
    implement         [label="Implement", prompt="Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD."]
    simplify_opus     [label="Simplify (Opus)", prompt="@prompts/simplify.md"]
    simplify_gpt      [label="Simplify (GPT-54)", prompt="@prompts/simplify.md", model="gpt-54"]
    verify            [label="Verify", shape=parallelogram, script="cargo clippy -q --workspace -- -D warnings 2>&1 && cargo nextest run --cargo-quiet --workspace --status-level fail 2>&1", goal_gate=true, retry_target="fixup"]
    fixup             [label="Fixup", prompt="The verify step failed. Read the build output from context and fix all clippy lint warnings and test failures.", max_visits=3]
    fmt               [label="Format", shape=parallelogram, script="cargo fmt --all 2>&1", max_retries=0]

    start -> toolchain
    toolchain -> preflight_compile [condition="outcome=success"]
    toolchain -> exit
    preflight_compile -> preflight_lint [condition="outcome=success"]
    preflight_compile -> exit
    preflight_lint -> implement [condition="outcome=success"]
    preflight_lint -> fix_lints
    fix_lints -> preflight_lint
    implement -> simplify_opus -> simplify_gpt -> verify
    verify -> fmt   [condition="outcome=success"]
    verify -> fixup
    fixup -> verify
    fmt -> exit
}

```

</details>

⚒️ Generated with [Fabro](https://fabro.sh)

---------

Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Bryan Helmkamp <bryan@brynary.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 22:48:08 -04:00

5.3 KiB
Raw Blame History

```console
$ fabro model
MODEL PROVIDER ALIASES CONTEXT COST SPEED
claude-opus-4-6 anthropic opus, claude-opus 1m $15.0 / $75.0 25 tok/s
claude-sonnet-4-5 anthropic 200k $3.0 / $15.0 50 tok/s
claude-sonnet-4-6 anthropic sonnet, claude-sonnet 200k $3.0 / $15.0 50 tok/s
claude-haiku-4-5 anthropic haiku, claude-haiku 200k $0.8 / $4.0 100 tok/s
gpt-5.2 openai gpt5 1m $1.8 / $14.0 65 tok/s
gpt-5-mini openai gpt5-mini 1m $0.2 / $2.0 70 tok/s
gpt-5.2-codex openai 1m $1.8 / $14.0 100 tok/s
gpt-5.3-codex openai codex 1m $1.8 / $14.0 100 tok/s
gpt-5.3-codex-spark openai codex-spark 131k - / - 1000 tok/s
gpt-5.4 openai gpt54, gpt-54 1m $2.5 / $15.0 70 tok/s
gpt-5.4-pro openai gpt54-pro, gpt-54-pro 1m $30.0 / $180.0 20 tok/s
gpt-5.4-mini openai gpt54-mini, gpt-54-mini 400k $0.8 / $4.5 140 tok/s
gemini-3.1-pro-preview gemini gemini-pro 1m $2.0 / $12.0 85 tok/s
gemini-3.1-pro-preview-customtools gemini gemini-customtools 1m $2.0 / $12.0 85 tok/s
gemini-3-flash-preview gemini gemini-flash 1m $0.5 / $3.0 150 tok/s
gemini-3.1-flash-lite-preview gemini gemini-flash-lite 1m $0.2 / $1.5 200 tok/s
kimi-k2.5 kimi kimi 262k $0.6 / $3.0 50 tok/s
glm-4.7 zai glm, glm4 203k $0.6 / $2.2 100 tok/s
minimax-m2.5 minimax minimax 197k $0.3 / $1.2 45 tok/s
mercury-2 inception mercury 131k $0.2 / $0.8 1000 tok/s
 
```