mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-09-05 08:10:39 +00:00
1b6189ee32
Some checks are pending
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run
TypeScript / Build (push) Waiting to run
## Summary Finish phase 9 of the configurable LLM provider/model work by aligning public docs, release notes, and guardrails with the implementation already landed in phases 0-8. - documents settings-driven providers/models, OpenAI-compatible gateway examples, typed `extra_headers`, model `api_id`, controls, and per-speed costs - adds the 2026-05-13 changelog entry and provider string migration note - updates the internal phase plan ledger to reflect current implementation status - adds a workspace policy test blocking direct production `Catalog::builtin()` usage outside catalog owner/test code - clarifies `Provider` as a built-in compatibility enum while open-ended identity is `ProviderId` ## Verification - `cargo nextest run -p fabro-dev --features dev --test it policy` - `cargo dev docs check` - `cargo nextest run -p fabro-model -p fabro-config -p fabro-auth -p fabro-llm` - `cargo build --workspace` - `cargo nextest run --workspace` (5717 passed, 182 skipped, nextest reported 1 leaky test) - `cargo +nightly-2026-04-14 fmt --check --all` - `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings` - `git diff --check`
35 lines
2.3 KiB
Markdown
35 lines
2.3 KiB
Markdown
# LLM Client Resolution
|
|
|
|
This document defines how Fabro resolves LLM credentials and constructs `fabro-llm` clients.
|
|
|
|
## Core Rules
|
|
|
|
- `fabro_auth::CredentialSource` is the credential authority.
|
|
- Long-lived runtime contexts store `Arc<dyn CredentialSource>` and `Arc<Catalog>`, not `Client`.
|
|
- Call `fabro_llm::client::Client::from_source(&source, catalog).await?` at the point of use.
|
|
- Standalone setup and tests that use default settings build a default `Arc<Catalog>` locally, then pass it explicitly.
|
|
- `GenerateParams::new(model, client)` always receives an explicit `Arc<Client>`.
|
|
- When a caller needs diagnostics in runtime request-serving paths, call `source.resolve(catalog)` directly and consume both `credentials` and `auth_issues`.
|
|
- `EnvCredentialSource` is the env-backed source for env-only or no-vault contexts.
|
|
- `VaultCredentialSource` is the normal source for vault-backed runtime contexts.
|
|
|
|
## Why
|
|
|
|
- Rebuilding a client from the source at point of use preserves OAuth refresh behavior on long-running processes.
|
|
- Holding the source on contexts avoids process-global installs and cross-context leakage.
|
|
- Threading the catalog into credential resolution keeps custom providers, aliases, header-only providers, and API model IDs consistent across auth, client registration, and request translation.
|
|
- Requiring an explicit client on `GenerateParams` makes the old silent fallback bug unrepresentable.
|
|
|
|
## Application
|
|
|
|
- Workflow state lives on `RunServices.llm_source` and `RunServices.catalog`.
|
|
- Server state lives on `AppState.llm_source` and `AppState.catalog()`.
|
|
- Hooks and other long-lived executors receive a source plus catalog and derive clients when they actually generate.
|
|
- One-shot CLI commands may resolve a source locally, build a default settings catalog if they do not load runtime catalog settings, then derive a client once for that operation.
|
|
|
|
## Enforcement
|
|
|
|
- Do not add new `Client::from_env`-style shortcuts in production paths.
|
|
- Do not cache a long-lived `Client` where OAuth refresh or storage-dir rebinding matters.
|
|
- Do not construct LLM clients or resolve credentials without an explicit `Arc<Catalog>` or `&Catalog`.
|
|
- Mirror [server-secrets-strategy.md](server-secrets-strategy.md): production credential resolution should be explicit about where secrets come from and how they flow into subprocesses.
|