mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-09-27 01:21:25 +00:00
cd74013d06
## Summary Compared with `origin/main`, this PR splits credential storage and credential references into explicit types. Vault secrets now distinguish `token`, `oauth`, and `file` payloads, while runtime/model configuration points to credentials through explicit `env:<NAME>` and `vault:<NAME>` source refs. ## Changes - Replaces the old `environment`/`credential` secret schema vocabulary with `token`/`oauth`/`file` across OpenAPI, Rust API tests, generated TypeScript models, CLI/docs references, and the changelog. - Updates auth resolution, refresh, provider strategies, workflow LLM handling, server diagnostics, install flows, run manifests, and secret handlers to consume typed vault entries and explicit credential sources. - Updates provider catalog TOMLs and config parsing so provider auth and extra headers use `vault` refs instead of ambiguous `credential` refs. - Updates CLI install/login/run/secret paths and integration tests to write and read the new credential shapes. - Removes the temporary legacy vault migration and empty-vault fallback, then centralizes provider vault secret-name lookup and Codex API credential shaping. ## Verification - `cargo +nightly-2026-04-14 fmt --all` - `cargo +nightly-2026-04-14 clippy -p fabro-auth -p fabro-model -p fabro-config -p fabro-vault -p fabro-server -p fabro-cli --all-targets -- -D warnings` - `ulimit -n 4096 && cargo nextest run -p fabro-auth -p fabro-model -p fabro-config -p fabro-vault -p fabro-server -p fabro-cli` (`1938` passed, `35` skipped)
133 lines
3.8 KiB
Text
133 lines
3.8 KiB
Text
---
|
|
title: "LiteLLM"
|
|
description: "Route Fabro models through a LiteLLM proxy"
|
|
---
|
|
|
|
[LiteLLM](https://docs.litellm.ai/) can run as an OpenAI-compatible proxy in front of many model providers. Fabro includes a disabled `litellm` provider entry so you can opt in from `settings.toml` without changing Fabro code.
|
|
|
|
## Prerequisites
|
|
|
|
- A running LiteLLM proxy reachable from the Fabro process
|
|
- At least one LiteLLM model name you want Fabro to route to
|
|
- A LiteLLM key or placeholder key available to Fabro
|
|
|
|
Fabro's built-in LiteLLM provider points at `http://localhost:4000/v1`. Change `base_url` if your proxy is hosted elsewhere.
|
|
|
|
## Enable the provider
|
|
|
|
Add the provider override and one or more model entries to `~/.fabro/settings.toml`:
|
|
|
|
```toml title="settings.toml"
|
|
_version = 1
|
|
|
|
[llm.providers.litellm]
|
|
enabled = true
|
|
base_url = "http://localhost:4000/v1"
|
|
|
|
[llm.models."litellm-gpt-5"]
|
|
provider = "litellm"
|
|
api_id = "gpt-5"
|
|
display_name = "LiteLLM GPT-5"
|
|
family = "litellm"
|
|
default = true
|
|
|
|
[llm.models."litellm-gpt-5".limits]
|
|
context_window = 128000
|
|
max_output = 8192
|
|
|
|
[llm.models."litellm-gpt-5".features]
|
|
tools = true
|
|
vision = false
|
|
reasoning = false
|
|
```
|
|
|
|
`api_id` is the model name Fabro sends to LiteLLM. It should match a model name configured in your LiteLLM proxy.
|
|
|
|
## Configure credentials
|
|
|
|
The LiteLLM provider checks `LITELLM_API_KEY` from the Fabro process environment first, then the `vault:LITELLM_API_KEY` server secret.
|
|
|
|
For a server-owned secret:
|
|
|
|
```bash
|
|
fabro secret set LITELLM_API_KEY sk-proxy-key
|
|
```
|
|
|
|
For a process environment variable:
|
|
|
|
```bash
|
|
export LITELLM_API_KEY=sk-proxy-key
|
|
```
|
|
|
|
If your local LiteLLM proxy does not enforce authentication, use a placeholder value such as `anything`; the OpenAI-compatible client still needs a credential value.
|
|
|
|
## Use LiteLLM models
|
|
|
|
Once the provider is enabled and at least one model is declared, use the Fabro model ID like any other catalog model:
|
|
|
|
```bash
|
|
fabro model list --provider litellm
|
|
fabro model test --model litellm-gpt-5
|
|
fabro run workflow.fabro --model litellm-gpt-5
|
|
```
|
|
|
|
In workflow stylesheets:
|
|
|
|
```dot title="workflow.fabro"
|
|
digraph Example {
|
|
graph [
|
|
model_stylesheet="
|
|
* { model: litellm-gpt-5; }
|
|
"
|
|
]
|
|
|
|
start [shape=Mdiamond, label="Start"]
|
|
work [label="Work", prompt="Use the configured LiteLLM model."]
|
|
exit [shape=Msquare, label="Exit"]
|
|
|
|
start -> work -> exit
|
|
}
|
|
```
|
|
|
|
## Declaring more models
|
|
|
|
Declare each LiteLLM-routed model explicitly so Fabro knows its provider, context window, tool support, and routing defaults:
|
|
|
|
```toml title="settings.toml"
|
|
[llm.models."litellm-fast"]
|
|
provider = "litellm"
|
|
api_id = "fast-model"
|
|
display_name = "LiteLLM Fast"
|
|
family = "litellm"
|
|
aliases = ["fast"]
|
|
|
|
[llm.models."litellm-fast".limits]
|
|
context_window = 64000
|
|
max_output = 4096
|
|
|
|
[llm.models."litellm-fast".features]
|
|
tools = true
|
|
vision = false
|
|
reasoning = false
|
|
```
|
|
|
|
Only one model for a provider should set `default = true`.
|
|
|
|
## Troubleshooting
|
|
|
|
**"No API key configured"** — Set `vault:LITELLM_API_KEY` with `fabro secret set LITELLM_API_KEY ...` or export `LITELLM_API_KEY` in the Fabro process environment.
|
|
|
|
**Connection refused** — Confirm the LiteLLM proxy is running and that `base_url` is reachable from the Fabro process. For Docker deployments, `localhost` means the Fabro container unless you point it at a host or service name.
|
|
|
|
**Unknown model from LiteLLM** — Check that the model's `api_id` matches the model name configured in LiteLLM, then run `fabro model test --model <fabro-model-id>`.
|
|
|
|
## Further reading
|
|
|
|
<Columns cols={2}>
|
|
<Card title="Models" icon="microchip" href="/core-concepts/models">
|
|
How Fabro routes model IDs, providers, and fallbacks.
|
|
</Card>
|
|
<Card title="Settings Configuration" icon="gear" href="/reference/user-configuration">
|
|
Full reference for `[llm.providers.<id>]` and `[llm.models.<id>]`.
|
|
</Card>
|
|
</Columns>
|