mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-10-09 03:20:56 +00:00
b9497517a5
## Summary Adds catalog-level `agent_profile` overrides so custom providers and individual models can choose Anthropic, OpenAI, or Gemini agent behavior independently from their adapter default. The effective precedence is model override, then provider override, then adapter metadata. ## What Changed - Added typed provider/model `agent_profile` settings in `fabro-config` and `fabro-model`, with serde/strum support for `anthropic`, `openai`, and `gemini`. - Centralized effective profile resolution in the catalog, including provider alias canonicalization and a guard against unrelated model overrides leaking across providers. - Updated run startup, API sessions, CLI/ACP backends, prompt project-memory discovery, and standalone agent startup to use the resolved catalog profile. - Documented provider-level and model-level `agent_profile` configuration in the public model and user configuration docs. No OpenAPI or model-list response shape changes are included. ## Validation - `cargo nextest run -p fabro-model -p fabro-config -p fabro-workflow -p fabro-agent` passed: 1826 passed, 125 skipped. - `cargo +nightly-2026-04-14 fmt --check --all` passed. - `cargo +nightly-2026-04-14 clippy -p fabro-model -p fabro-config -p fabro-workflow -p fabro-agent --all-targets -- -D warnings` passed. - `git diff --check` passed. - `cargo nextest list -p fabro-dev` confirmed there is no docs-options reference test target to run. ## Post-Deploy Monitoring & Validation Watch workflow and agent-session logs for provider/model resolution errors, unexpected project-memory file selection, or CLI/ACP launch command mismatches on custom catalog providers. Healthy signal: custom provider/model runs start normally and use the intended profile-specific behavior. Failure trigger: repeated `Provider ... is not configured` errors, missing expected project memory, or profile-specific agent startup failures after configuring `agent_profile`. Mitigation is to remove the override from config or revert this PR. Validation window: first deploy cycle after merge; owner: release/on-call engineer. --- [](https://github.com/EveryInc/compound-engineering-plugin) 🤖 Generated with GPT-5 via [Codex](https://openai.com/codex)
243 lines
9 KiB
Text
243 lines
9 KiB
Text
---
|
|
title: "Models"
|
|
description: "How Fabro routes tasks to LLM models and providers"
|
|
---
|
|
|
|
No single model is best at everything. Fabro lets you assign the right model to each workflow step — cheap, fast models for boilerplate, frontier models for hard reasoning, and a different provider for cross-critique so the reviewer brings fresh eyes. When a provider goes down, Fabro can fail over automatically.
|
|
|
|
<Frame>
|
|
<img src="/images/ensemble-workflow.svg" alt="Ensemble workflow: fan out to Opus and Gemini Pro, merge, then synthesize" />
|
|
</Frame>
|
|
|
|
## Model catalog
|
|
|
|
| Model | Provider | Aliases | Context | Cost (in/out per Mtok) | Speed |
|
|
|---|---|---|---|---|---|
|
|
| `claude-opus-4-7` | anthropic | `opus`, `claude-opus` | 1M | $5.00 / $25.00 | 25 tok/s |
|
|
| `claude-opus-4-6` | anthropic | | 1M | $5.00 / $25.00 | 25 tok/s |
|
|
| `claude-sonnet-4-6` | anthropic | `sonnet`, `claude-sonnet` | 200K | $3.00 / $15.00 | 50 tok/s |
|
|
| `claude-sonnet-4-5` | anthropic | | 200K | $3.00 / $15.00 | 50 tok/s |
|
|
| `claude-haiku-4-5` | anthropic | `haiku`, `claude-haiku` | 200K | $0.80 / $4.00 | 100 tok/s |
|
|
| `gpt-5.2` | openai | `gpt5` | 1M | $1.80 / $14.00 | 65 tok/s |
|
|
| `gpt-5-mini` | openai | `gpt5-mini` | 1M | $0.20 / $2.00 | 70 tok/s |
|
|
| `gpt-5.2-codex` | openai | | 1M | $1.80 / $14.00 | 100 tok/s |
|
|
| `gpt-5.3-codex` | openai | `codex` | 1M | $1.80 / $14.00 | 100 tok/s |
|
|
| `gpt-5.3-codex-spark` | openai | `codex-spark` | 128K | n/a | 1000 tok/s |
|
|
| `gpt-5.4` | openai | `gpt54` | 1M | $2.50 / $15.00 | 70 tok/s |
|
|
| `gpt-5.5` | openai | `gpt55` | 1M | $5.00 / $30.00 | 70 tok/s |
|
|
| `gpt-5.5-pro` | openai | `gpt55-pro` | 1M | $30.00 / $180.00 | 20 tok/s |
|
|
| `gpt-5.4-mini` | openai | `gpt54-mini` | 400K | $0.75 / $4.50 | 140 tok/s |
|
|
| `gpt-5.4-pro` | openai | `gpt54-pro` | 1M | $30.00 / $180.00 | 20 tok/s |
|
|
| `gemini-3.1-pro-preview` | gemini | `gemini-pro` | 1M | $2.00 / $12.00 | 85 tok/s |
|
|
| `gemini-3.1-pro-preview-customtools` | gemini | `gemini-customtools` | 1M | $2.00 / $12.00 | 85 tok/s |
|
|
| `gemini-3-flash-preview` | gemini | `gemini-flash` | 1M | $0.50 / $3.00 | 150 tok/s |
|
|
| `gemini-3.1-flash-lite-preview` | gemini | `gemini-flash-lite` | 1M | $0.20 / $1.50 | 200 tok/s |
|
|
| `kimi-k2.5` | kimi | `kimi` | 262K | $0.60 / $3.00 | 50 tok/s |
|
|
| `glm-4.7` | zai | `glm`, `glm4` | 203K | $0.60 / $2.20 | 100 tok/s |
|
|
| `minimax-m2.5` | minimax | `minimax` | 197K | $0.30 / $1.20 | 45 tok/s |
|
|
| `mercury-2` | inception | `mercury` | 131K | $0.20 / $0.80 | 1000 tok/s |
|
|
|
|
Each provider requires its own API key set via environment variable (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`). See the [Quick Start](/getting-started/quick-start) for setup.
|
|
|
|
## Configuring providers and models
|
|
|
|
Fabro's catalog starts with the built-in providers and models, then merges any `[llm]` entries from settings. Provider and model IDs are strings, so a server or project can add an OpenAI-compatible provider without a Fabro release.
|
|
|
|
```toml title="settings.toml"
|
|
[llm.providers.proxy]
|
|
display_name = "Acme Gateway"
|
|
adapter = "openai_compatible"
|
|
agent_profile = "openai"
|
|
base_url = "https://llm-gateway.example.com/v1"
|
|
credentials = ["env:ACME_GATEWAY_API_KEY"]
|
|
aliases = ["gateway"]
|
|
|
|
[llm.providers.proxy.extra_headers]
|
|
x-portkey-api-key = { env = "PORTKEY_API_KEY" }
|
|
x-portkey-config = { literal = "@bedrock-prod" }
|
|
|
|
[llm.models."team-code-large"]
|
|
provider = "proxy"
|
|
api_id = "provider-wire-model-name"
|
|
agent_profile = "anthropic"
|
|
display_name = "Team Code Large"
|
|
family = "team-code"
|
|
default = true
|
|
aliases = ["team-code"]
|
|
estimated_output_tps = 80
|
|
|
|
[llm.models."team-code-large".limits]
|
|
context_window = 200000
|
|
max_output = 32000
|
|
|
|
[llm.models."team-code-large".features]
|
|
tools = true
|
|
reasoning = true
|
|
reasoning_effort = "levels"
|
|
prompt_cache = true
|
|
effort = true
|
|
|
|
[llm.models."team-code-large".controls]
|
|
reasoning_effort = ["low", "medium", "high"]
|
|
speed = ["fast"]
|
|
|
|
[llm.models."team-code-large".costs]
|
|
input_cost_per_mtok = 1.50
|
|
output_cost_per_mtok = 8.00
|
|
cache_input_cost_per_mtok = 0.30
|
|
|
|
[llm.models."team-code-large".costs.speed.fast]
|
|
input_cost_per_mtok = 3.00
|
|
output_cost_per_mtok = 16.00
|
|
cache_input_cost_per_mtok = 0.60
|
|
```
|
|
|
|
For [LiteLLM](/integrations/litellm), Fabro ships a disabled provider entry. Enable it in settings and declare the models your proxy exposes:
|
|
|
|
```toml title="settings.toml"
|
|
[llm.providers.litellm]
|
|
enabled = true
|
|
base_url = "http://localhost:4000/v1"
|
|
|
|
[llm.models."litellm-gpt-5"]
|
|
provider = "litellm"
|
|
api_id = "gpt-5"
|
|
display_name = "LiteLLM GPT-5"
|
|
family = "litellm"
|
|
default = true
|
|
|
|
[llm.models."litellm-gpt-5".limits]
|
|
context_window = 128000
|
|
max_output = 8192
|
|
|
|
[llm.models."litellm-gpt-5".features]
|
|
tools = true
|
|
vision = false
|
|
reasoning = false
|
|
```
|
|
|
|
`api_id` is the model name sent to the provider API. Omit it when the Fabro model ID and provider model ID are the same.
|
|
|
|
Header values and credentials are typed references, not raw secrets. Use `env:<NAME>` or `credential:<id>` for provider credentials, and `{ env = "NAME" }`, `{ credential = "id" }`, or `{ literal = "value" }` for extra headers.
|
|
|
|
`agent_profile` controls profile-specific behavior such as project-memory filenames, CLI/ACP command selection, and native session routing. Valid values are `anthropic`, `openai`, and `gemini`; model-level values override provider-level values, which override the adapter default.
|
|
|
|
<Note>
|
|
Provider fields in configuration, APIs, and model routing are provider ID strings. Built-in names like `anthropic`, `openai`, and `gemini` still work, but custom IDs like `proxy` work anywhere a provider ID is accepted.
|
|
</Note>
|
|
|
|
### Ollama
|
|
|
|
Fabro ships an Ollama provider definition that is disabled by default. Enable it in settings when you want Fabro to route through a local Ollama server:
|
|
|
|
```toml title="settings.toml"
|
|
[llm.providers.ollama]
|
|
enabled = true
|
|
```
|
|
|
|
Enabling the provider alone does not expose any models — until #267 adds auto-discovery, add explicit `[llm.models.<id>]` blocks for each Ollama model you have pulled locally. Ollama's OpenAI-compatible endpoint accepts any bearer token, so local users can set `OLLAMA_API_KEY=ollama`.
|
|
|
|
## Default models
|
|
|
|
When no model or provider is specified, Fabro checks configured provider credentials and chooses the first configured provider by catalog priority. If no provider credentials are configured, it uses the catalog's global default model. Each provider has a default model:
|
|
|
|
| Provider | Default model |
|
|
|---|---|
|
|
| `anthropic` | `claude-opus-4-7` |
|
|
| `openai` | `gpt-5.5` |
|
|
| `gemini` | `gemini-3.1-pro-preview` |
|
|
| `kimi` | `kimi-k2.5` |
|
|
| `zai` | `glm-4.7` |
|
|
| `minimax` | `minimax-m2.5` |
|
|
| `inception` | `mercury` |
|
|
|
|
## Using models in workflows
|
|
|
|
Assign models to workflow nodes using [model stylesheets](/workflows/stylesheets), which use a CSS-like syntax:
|
|
|
|
```dot title="example.fabro"
|
|
digraph Example {
|
|
graph [
|
|
model_stylesheet="
|
|
* { model: claude-haiku-4-5; }
|
|
.coding { model: claude-sonnet-4-5; reasoning_effort: high; }
|
|
#review { model: gemini-3.1-pro-preview; }
|
|
"
|
|
]
|
|
|
|
spec [label="Write Spec"]
|
|
implement [label="Implement", class="coding"]
|
|
review [label="Review"]
|
|
}
|
|
```
|
|
|
|
This routes the spec node to Haiku (the default), implementation to Sonnet, and review to Gemini Pro.
|
|
|
|
## Overriding the default model
|
|
|
|
Model stylesheets set per-node models inside the workflow graph, but you can also override the default model for an entire run. This is useful for quick experimentation or when you want to swap models without editing the Graphviz file.
|
|
|
|
### CLI flags
|
|
|
|
Pass `--model` and optionally `--provider` to `fabro run`:
|
|
|
|
```bash
|
|
fabro run docs/internal/demo/01-hello.fabro --model claude-opus-4-6
|
|
fabro run docs/internal/demo/04-pipeline.fabro --model gemini-3.1-pro-preview
|
|
```
|
|
|
|
These flags set the default model for all nodes that don't have an explicit model assigned via a stylesheet. The provider is automatically inferred from the model catalog — you only need `--provider` for models not in the catalog or to force a specific provider.
|
|
|
|
### Run config TOML
|
|
|
|
For repeatable runs, set the model in a run config file:
|
|
|
|
```toml title="run.toml"
|
|
_version = 1
|
|
|
|
[workflow]
|
|
graph = "implement.fabro"
|
|
|
|
[run]
|
|
goal = "Implement the feature"
|
|
|
|
[run.model]
|
|
name = "claude-sonnet-4-5"
|
|
fallbacks = ["gemini", "openai"]
|
|
```
|
|
|
|
Then launch with:
|
|
|
|
```bash
|
|
fabro run run.toml
|
|
```
|
|
|
|
The `fallbacks` array is optional. Each entry may be a bare provider token (like `"gemini"`), a bare model alias (like `"gpt-5.4"`), or a qualified `"provider/model"` reference. Fabro tries them in order when the primary provider is unavailable.
|
|
|
|
<Note>
|
|
The precedence order is: node-level stylesheet > run config TOML > CLI flags > server defaults. More specific settings always win.
|
|
</Note>
|
|
|
|
## CLI commands
|
|
|
|
### List models
|
|
|
|
View all available models, or filter by provider:
|
|
|
|
```bash
|
|
fabro model list
|
|
fabro model list --provider anthropic
|
|
fabro model list --query codex
|
|
```
|
|
|
|
### Test models
|
|
|
|
Verify that your API keys are working by sending a test prompt to each configured provider:
|
|
|
|
```bash
|
|
fabro model test
|
|
fabro model test --model claude-sonnet-4-5
|
|
fabro model test --provider openai
|
|
```
|
|
|
|
This is useful for confirming connectivity after setup or when adding a new provider key.
|