feat: add opt-in LiteLLM TOML provider (#269)

## Summary
- Add a disabled built-in `litellm` provider fragment backed by the
OpenAI-compatible adapter and local proxy defaults.
- Document how to enable LiteLLM in `settings.toml`, configure
credentials, and declare explicit LiteLLM-routed models.
- Register the LiteLLM integration page and cross-link it from the model
and settings docs.

## Validation
- `cargo test -p fabro-model`
- `cargo test -p fabro-config`
- `jq empty docs/public/docs.json`
- `rg -n 'aliases = \["openai_compatible",
"openai-compatible"\]|llm\.discovery|FABRO_LITELLM|litellm_api_key_env|x-litellm-'
lib/crates/fabro-model/src/catalog/providers/litellm.toml
docs/public/integrations/litellm.mdx
docs/public/core-concepts/models.mdx
docs/public/reference/user-configuration.mdx` returned no matches

---------

Co-authored-by: Mark Ferraz <mferraz@netwoven.com>
This commit is contained in:
Bryan Helmkamp 2026-05-16 05:36:31 -07:00 • committed by GitHub
parent 9768651b52
commit d52a2ccbe2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 187 additions and 0 deletions

View file

@ -90,6 +90,30 @@ output_cost_per_mtok = 16.00
cache_input_cost_per_mtok = 0.60
```
For [LiteLLM](/integrations/litellm), Fabro ships a disabled provider entry. Enable it in settings and declare the models your proxy exposes:
```toml title="settings.toml"
[llm.providers.litellm]
enabled = true
base_url = "http://localhost:4000/v1"
[llm.models."litellm-gpt-5"]
provider = "litellm"
api_id = "gpt-5"
display_name = "LiteLLM GPT-5"
family = "litellm"
default = true
[llm.models."litellm-gpt-5".limits]
context_window = 128000
max_output = 8192
[llm.models."litellm-gpt-5".features]
tools = true
vision = false
reasoning = false
```
`api_id` is the model name sent to the provider API. Omit it when the Fabro model ID and provider model ID are the same.
Header values and credentials are typed references, not raw secrets. Use `env:<NAME>` or `credential:<id>` for provider credentials, and `{ env = "NAME" }`, `{ credential = "id" }`, or `{ literal = "value" }` for extra headers.

View file

@ -93,6 +93,7 @@
"pages": [
"integrations/github",
"integrations/daytona",
"integrations/litellm",
"integrations/slack",
"integrations/brave-search"
]

View file

@ -0,0 +1,133 @@
---
title: "LiteLLM"
description: "Route Fabro models through a LiteLLM proxy"
---
[LiteLLM](https://docs.litellm.ai/) can run as an OpenAI-compatible proxy in front of many model providers. Fabro includes a disabled `litellm` provider entry so you can opt in from `settings.toml` without changing Fabro code.
## Prerequisites
- A running LiteLLM proxy reachable from the Fabro process
- At least one LiteLLM model name you want Fabro to route to
- A LiteLLM key or placeholder key available to Fabro
Fabro's built-in LiteLLM provider points at `http://localhost:4000/v1`. Change `base_url` if your proxy is hosted elsewhere.
## Enable the provider
Add the provider override and one or more model entries to `~/.fabro/settings.toml`:
```toml title="settings.toml"
_version = 1
[llm.providers.litellm]
enabled = true
base_url = "http://localhost:4000/v1"
[llm.models."litellm-gpt-5"]
provider = "litellm"
api_id = "gpt-5"
display_name = "LiteLLM GPT-5"
family = "litellm"
default = true
[llm.models."litellm-gpt-5".limits]
context_window = 128000
max_output = 8192
[llm.models."litellm-gpt-5".features]
tools = true
vision = false
reasoning = false
```
`api_id` is the model name Fabro sends to LiteLLM. It should match a model name configured in your LiteLLM proxy.
## Configure credentials
The LiteLLM provider checks `credential:litellm` first, then `LITELLM_API_KEY` from the Fabro process environment.
For a server-owned secret:
```bash
fabro secret set litellm sk-proxy-key
```
For a process environment variable:
```bash
export LITELLM_API_KEY=sk-proxy-key
```
If your local LiteLLM proxy does not enforce authentication, use a placeholder value such as `anything`; the OpenAI-compatible client still needs a credential value.
## Use LiteLLM models
Once the provider is enabled and at least one model is declared, use the Fabro model ID like any other catalog model:
```bash
fabro model list --provider litellm
fabro model test --model litellm-gpt-5
fabro run workflow.fabro --model litellm-gpt-5
```
In workflow stylesheets:
```dot title="workflow.fabro"
digraph Example {
graph [
model_stylesheet="
* { model: litellm-gpt-5; }
"
]
start [shape=Mdiamond, label="Start"]
work [label="Work", prompt="Use the configured LiteLLM model."]
exit [shape=Msquare, label="Exit"]
start -> work -> exit
}
```
## Declaring more models
Declare each LiteLLM-routed model explicitly so Fabro knows its provider, context window, tool support, and routing defaults:
```toml title="settings.toml"
[llm.models."litellm-fast"]
provider = "litellm"
api_id = "fast-model"
display_name = "LiteLLM Fast"
family = "litellm"
aliases = ["fast"]
[llm.models."litellm-fast".limits]
context_window = 64000
max_output = 4096
[llm.models."litellm-fast".features]
tools = true
vision = false
reasoning = false
```
Only one model for a provider should set `default = true`.
## Troubleshooting
**"No API key configured"** — Set `credential:litellm` with `fabro secret set litellm ...` or export `LITELLM_API_KEY` in the Fabro process environment.
**Connection refused** — Confirm the LiteLLM proxy is running and that `base_url` is reachable from the Fabro process. For Docker deployments, `localhost` means the Fabro container unless you point it at a host or service name.
**Unknown model from LiteLLM** — Check that the model's `api_id` matches the model name configured in LiteLLM, then run `fabro model test --model <fabro-model-id>`.
## Further reading
<Columns cols={2}>
<Card title="Models" icon="microchip" href="/core-concepts/models">
How Fabro routes model IDs, providers, and fallbacks.
</Card>
<Card title="Settings Configuration" icon="gear" href="/reference/user-configuration">
Full reference for `[llm.providers.<id>]` and `[llm.models.<id>]`.
</Card>
</Columns>

View file

@ -0,0 +1,29 @@
[providers.litellm]
display_name = "LiteLLM"
adapter = "openai_compatible"
base_url = "http://localhost:4000/v1"
credentials = ["credential:litellm", "env:LITELLM_API_KEY"]
priority = 50
enabled = false
# To enable LiteLLM, add entries like these to settings.toml:
#
# [llm.providers.litellm]
# enabled = true
# base_url = "http://localhost:4000/v1"
#
# [llm.models."litellm-gpt-5"]
# provider = "litellm"
# api_id = "gpt-5"
# display_name = "LiteLLM GPT-5"
# family = "litellm"
# default = true
#
# [llm.models."litellm-gpt-5".limits]
# context_window = 128000
# max_output = 8192
#
# [llm.models."litellm-gpt-5".features]
# tools = true
# vision = false
# reasoning = false