scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-08-28 05:27:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
fabro/docs/internal/llm-client-resolution.md
Bryan Helmkamp 1b6189ee32
Some checks are pending
Rust / Format (push) Waiting to run
Details
Rust / Clippy (push) Waiting to run
Details
Rust / Generated Docs (push) Waiting to run
Details
Rust / Test (Linux) (push) Waiting to run
Details
Rust / Test (macOS) (push) Waiting to run
Details
TypeScript / Typecheck (push) Waiting to run
Details
TypeScript / Test (push) Waiting to run
Details
TypeScript / Build (push) Waiting to run
Details
docs(llm): finish configurable provider cleanup (#260) 
## Summary

Finish phase 9 of the configurable LLM provider/model work by aligning
public docs, release notes, and guardrails with the implementation
already landed in phases 0-8.

- documents settings-driven providers/models, OpenAI-compatible gateway
examples, typed `extra_headers`, model `api_id`, controls, and per-speed
costs
- adds the 2026-05-13 changelog entry and provider string migration note
- updates the internal phase plan ledger to reflect current
implementation status
- adds a workspace policy test blocking direct production
`Catalog::builtin()` usage outside catalog owner/test code
- clarifies `Provider` as a built-in compatibility enum while open-ended
identity is `ProviderId`

## Verification

- `cargo nextest run -p fabro-dev --features dev --test it policy`
- `cargo dev docs check`
- `cargo nextest run -p fabro-model -p fabro-config -p fabro-auth -p
fabro-llm`
- `cargo build --workspace`
- `cargo nextest run --workspace` (5717 passed, 182 skipped, nextest
reported 1 leaky test)
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D
warnings`
- `git diff --check`
2026-05-13 17:17:54 -04:00

2.3 KiB
Raw Permalink Blame History

LLM Client Resolution

This document defines how Fabro resolves LLM credentials and constructs fabro-llm clients.

Core Rules

  • fabro_auth::CredentialSource is the credential authority.
  • Long-lived runtime contexts store Arc<dyn CredentialSource> and Arc<Catalog>, not Client.
  • Call fabro_llm::client::Client::from_source(&source, catalog).await? at the point of use.
  • Standalone setup and tests that use default settings build a default Arc<Catalog> locally, then pass it explicitly.
  • GenerateParams::new(model, client) always receives an explicit Arc<Client>.
  • When a caller needs diagnostics in runtime request-serving paths, call source.resolve(catalog) directly and consume both credentials and auth_issues.
  • EnvCredentialSource is the env-backed source for env-only or no-vault contexts.
  • VaultCredentialSource is the normal source for vault-backed runtime contexts.

Why

  • Rebuilding a client from the source at point of use preserves OAuth refresh behavior on long-running processes.
  • Holding the source on contexts avoids process-global installs and cross-context leakage.
  • Threading the catalog into credential resolution keeps custom providers, aliases, header-only providers, and API model IDs consistent across auth, client registration, and request translation.
  • Requiring an explicit client on GenerateParams makes the old silent fallback bug unrepresentable.

Application

  • Workflow state lives on RunServices.llm_source and RunServices.catalog.
  • Server state lives on AppState.llm_source and AppState.catalog().
  • Hooks and other long-lived executors receive a source plus catalog and derive clients when they actually generate.
  • One-shot CLI commands may resolve a source locally, build a default settings catalog if they do not load runtime catalog settings, then derive a client once for that operation.

Enforcement

  • Do not add new Client::from_env-style shortcuts in production paths.
  • Do not cache a long-lived Client where OAuth refresh or storage-dir rebinding matters.
  • Do not construct LLM clients or resolve credentials without an explicit Arc<Catalog> or &Catalog.
  • Mirror server-secrets-strategy.md: production credential resolution should be explicit about where secrets come from and how they flow into subprocesses.