scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-10-06 02:48:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
fabro/docs/internal/llm-client-resolution.md
Bryan Helmkamp 6dfc96d3fd
Resolve provider secrets through lithos conventional credentials 
lithos-llm now owns which named secrets each provider reads and how they
shape into its auth scheme, including a derived `<PROVIDER>_API_KEY` for
operator-defined providers. Fabro's job shrinks to supplying the store:
`VaultCredentialSource` hands lithos a lookup that reads the process
environment, then the vault, under the same conventional names.

What Fabro still adds on top: the Codex OAuth credential in the vault,
refreshed and persisted when it expires; `{{ secrets.NAME }}` tokens in a
provider's `default_headers`, resolved against the vault and re-sent as
credential headers; and OpenAI organization and project headers from the
environment.

Deleted with the `metadata.fabro.credentials` list: `CredentialRef`,
`CredentialResolver`, `EnvCredentialSource` (now
`VaultCredentialSource::environment_only`), and the `env_var_names` /
`expected_vault_secret_name` helpers, replaced by `secret_names` and
`expected_secret_name` over the lithos table. `openai-codex` joins the
first-party provider id constants.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-09 23:00:08 -06:00

2.3 KiB
Raw Blame History

LLM Client Resolution

This document defines how Fabro resolves LLM credentials and constructs fabro-llm clients.

Core Rules

  • fabro_auth::CredentialSource is the credential authority.
  • Long-lived runtime contexts store Arc<dyn CredentialSource> and Arc<Catalog>, not Client.
  • Call fabro_llm::client::Client::from_source(&source, catalog).await? at the point of use.
  • Standalone setup and tests that use default settings build a default Arc<Catalog> locally, then pass it explicitly.
  • GenerateParams::new(model, client) always receives an explicit Arc<Client>.
  • When a caller needs diagnostics in runtime request-serving paths, call source.resolve(catalog) directly and consume both credentials and auth_issues.
  • VaultCredentialSource is the normal source for vault-backed runtime contexts; VaultCredentialSource::environment_only() serves env-only or no-vault contexts.

Why

  • Rebuilding a client from the source at point of use preserves OAuth refresh behavior on long-running processes.
  • Holding the source on contexts avoids process-global installs and cross-context leakage.
  • Threading the catalog into credential resolution keeps custom providers, aliases, header-only providers, and API model IDs consistent across auth, client registration, and request translation.
  • Requiring an explicit client on GenerateParams makes the old silent fallback bug unrepresentable.

Application

  • Workflow state lives on RunServices.llm_source and RunServices.catalog.
  • Server state lives on AppState.llm_source and AppState.catalog().
  • Hooks and other long-lived executors receive a source plus catalog and derive clients when they actually generate.
  • One-shot CLI commands may resolve a source locally, build a default settings catalog if they do not load runtime catalog settings, then derive a client once for that operation.

Enforcement

  • Do not add new Client::from_env-style shortcuts in production paths.
  • Do not cache a long-lived Client where OAuth refresh or storage-dir rebinding matters.
  • Do not construct LLM clients or resolve credentials without an explicit Arc<Catalog> or &Catalog.
  • Mirror server-secrets-strategy.md: production credential resolution should be explicit about where secrets come from and how they flow into subprocesses.