fix(cli): use server catalog for provider login (#529)
Some checks are pending
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Build (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run

## Summary

`fabro provider login --server ... --provider openrouter` now asks the
selected Fabro server for provider metadata before reading, validating,
and storing API keys, so server-enabled providers are accepted even when
the local CLI catalog does not know them.

This adds a server-side credential test endpoint that validates
submitted API keys against the server's effective catalog without
persisting them, then keeps saving the resulting secret to the selected
target server. OpenAI Codex device login remains client-side for the
browser/device flow, with the resulting OAuth credential stored on the
selected server.

The OpenRouter docs and model docs are updated to use the current
`--provider openrouter` login syntax and clarify that remote deployments
need the server host settings updated.

## Testing

- `cargo nextest run -p fabro-client -p fabro-server -p fabro-cli
provider`
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy -p fabro-client -p fabro-server -p
fabro-cli --all-targets -- -D warnings`
- `rg -n "provider login openrouter|fabro provider login [a-z]"
docs/public lib/crates/fabro-cli/tests lib/crates/fabro-cli/src -g
'*.md' -g '*.mdx' -g '*.rs'`

---

[![Compound
Engineering](https://img.shields.io/badge/Compound_Engineering-6366f1)](https://github.com/EveryInc/compound-engineering-plugin)
🤖 Generated with GPT-5 (context compacted, extended thinking) via
[Codex](https://openai.com/codex)
This commit is contained in:
Bryan Helmkamp 2026-06-26 07:46:38 -05:00 • committed by GitHub
parent 6529f120de
commit 2307468bc6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 494 additions and 47 deletions

View file

@ -5526,6 +5526,60 @@ paths:
schema:
$ref: "#/components/schemas/ProviderList"
/api/v1/providers/{provider}/credentials/test:
post:
operationId: testProviderCredentials
tags: [Models]
summary: Test Provider Credentials
description: Validates an LLM provider API key against the server's effective catalog without persisting it.
parameters:
- name: provider
in: path
required: true
schema:
type: string
description: The provider identifier.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ProviderCredentialTestRequest"
responses:
"200":
description: Credentials validated successfully
content:
application/json:
schema:
$ref: "#/components/schemas/ProviderCredentialTestResponse"
"400":
description: Invalid provider credential request
headers:
x-request-id:
$ref: "#/components/headers/XRequestId"
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorResponse"
"404":
description: Provider not found
headers:
x-request-id:
$ref: "#/components/headers/XRequestId"
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorResponse"
"422":
description: Credential validation failed
headers:
x-request-id:
$ref: "#/components/headers/XRequestId"
content:
application/json:
schema:
$ref: "#/components/schemas/ErrorResponse"
/api/v1/providers/test:
post:
operationId: testProviders
@ -7931,6 +7985,25 @@ components:
items:
$ref: "#/components/schemas/Provider"
ProviderCredentialTestRequest:
description: API key to validate against an LLM provider without persisting it.
type: object
required:
- api_key
properties:
api_key:
type: string
ProviderCredentialTestResponse:
description: Successful response from provider credential validation.
type: object
required:
- ok
properties:
ok:
type: boolean
example: true
ProviderTestList:
description: Results from testing all configured LLM providers.
type: object

View file

@ -134,7 +134,7 @@ Provider fields in configuration, APIs, and model routing are provider ID string
### OpenRouter
Fabro ships an [OpenRouter](/integrations/openrouter) provider definition with a curated model catalog, disabled by default. Enable it in settings and store an API key with `fabro provider login openrouter`:
Fabro ships an [OpenRouter](/integrations/openrouter) provider definition with a curated model catalog, disabled by default. Enable it in settings and store an API key with `fabro provider login --provider openrouter`:
```toml title="settings.toml"
[llm.providers.openrouter]

View file

@ -7,12 +7,12 @@ description: "Route Fabro models through OpenRouter's multi-provider gateway"
## Prerequisites
- An [OpenRouter account](https://openrouter.ai/) with credit
- An [OpenRouter account](https://openrouter.ai/) with credit for paid models
- An API key from [openrouter.ai/keys](https://openrouter.ai/keys)
## Enable the provider
Add the provider override to `~/.fabro/settings.toml`:
Fabro runs execute through a Fabro server. Add the provider override to the settings file used by that server. For a local server, this is usually `~/.fabro/settings.toml`; for a remote deployment, update the server host's Fabro settings.
```toml title="settings.toml"
_version = 1
@ -23,15 +23,20 @@ enabled = true
## Configure credentials
For server-backed runs, store the key in the Fabro server vault:
Store the key in the target Fabro server vault:
```bash
fabro provider login openrouter
# or
fabro provider login --provider openrouter
# For a non-default remote server:
fabro provider login --server https://your-fabro.example --provider openrouter
# Or set the vault token directly:
fabro secret set OPENROUTER_API_KEY sk-or-v1-...
fabro secret --server https://your-fabro.example set OPENROUTER_API_KEY sk-or-v1-...
```
Standalone local SDK/CLI runs can use an env-backed credential source explicitly:
Direct SDK usage outside a Fabro server can use an env-backed credential source explicitly:
```bash
export OPENROUTER_API_KEY=sk-or-v1-...
@ -58,6 +63,7 @@ Any other OpenRouter model can be added as a settings model entry with `provider
```toml title="settings.toml"
[llm.models."meta-llama/llama-4-maverick"]
provider = "openrouter"
api_id = "meta-llama/llama-4-maverick"
display_name = "Llama 4 Maverick"
family = "llama-4"
@ -78,6 +84,13 @@ fabro model test --model anthropic/claude-sonnet-4-6
fabro run workflow.fabro --model deepseek/deepseek-v4-flash
```
When targeting a non-default remote server, pass the same `--server` value to verification commands:
```bash
fabro model list --server https://your-fabro.example --provider openrouter
fabro model test --server https://your-fabro.example --model anthropic/claude-sonnet-4-6
```
In workflow stylesheets:
```dot title="workflow.fabro"
@ -130,9 +143,11 @@ Fabro does not send OpenRouter's optional attribution headers (`HTTP-Referer`, `
## Troubleshooting
**"No API key configured"** — For server-backed runs, set the key with `fabro provider login openrouter` or `fabro secret set OPENROUTER_API_KEY ...`. For standalone local usage, export `OPENROUTER_API_KEY` in the invoking shell.
**"No API key configured"** — Set the key on the target server with `fabro provider login --provider openrouter` or `fabro secret set OPENROUTER_API_KEY ...`. For direct SDK usage outside a Fabro server, export `OPENROUTER_API_KEY` in the invoking shell.
**402 / insufficient credits** — OpenRouter requires prepaid credit; check your balance at [openrouter.ai/credits](https://openrouter.ai/credits).
**"provider 'openrouter' is not configured in the server model catalog"** — Confirm the server host's `settings.toml` has `[llm.providers.openrouter]` with `enabled = true`. Fabro live-reloads `settings.toml` within a few seconds; after that, `fabro model list --provider openrouter` against the same server should show the enabled catalog.
**402 / insufficient credits** — Paid OpenRouter models require prepaid credit; check your balance at [openrouter.ai/credits](https://openrouter.ai/credits).
**Unknown model** — Confirm the model's `api_id` matches an OpenRouter slug exactly (including the vendor prefix), then run `fabro model test --model <fabro-model-id>`.

View file

@ -1,10 +1,14 @@
use anyhow::{Context, Result};
use fabro_api::types;
use fabro_auth::{LoginResult, OPENAI_CODEX_VAULT_SECRET_NAME};
use fabro_auth::{AuthContextRequest, AuthMethod, LoginResult, OPENAI_CODEX_VAULT_SECRET_NAME};
use fabro_model::ProviderId;
use fabro_util::printer::Printer;
use fabro_util::terminal::Styles;
use tokio::task::spawn_blocking;
use crate::args::ProviderLoginArgs;
use crate::command_context::CommandContext;
use crate::server_client;
use crate::shared::provider_auth;
pub(super) async fn login_command(
@ -16,41 +20,49 @@ pub(super) async fn login_command(
let s = Styles::detect_stderr();
let ctx = base_ctx.with_target(&args.target)?;
let server = ctx.server().await?;
let result = if args.api_key_stdin {
provider_auth::authenticate_provider_with_api_key_source_and_catalog(
let (name, value, type_) = if args.api_key_stdin {
let (name, key) = login_with_server_api_key(
&server,
args.provider,
provider_auth::ApiKeySource::Stdin,
&s,
printer,
ctx.catalog()?,
)
.await?
.await?;
(name, key, types::SecretType::Token)
} else {
provider_auth::authenticate_provider_with_catalog(
args.provider,
&s,
printer,
ctx.catalog()?,
)
.await?
};
let (name, value, type_) = match result {
LoginResult::ApiKey { provider, key } => {
let name = ctx
.catalog()?
.provider_vault_secret_name(&provider)
.with_context(|| {
format!("provider '{provider}' does not define a vault credential path")
})?
.to_string();
(name, key, types::SecretType::Token)
match provider_auth::pick_auth_method(&args.provider).await? {
AuthMethod::ApiKey => {
let (name, key) = login_with_server_api_key(
&server,
args.provider,
provider_auth::ApiKeySource::Prompt,
&s,
printer,
)
.await?;
(name, key, types::SecretType::Token)
}
method @ AuthMethod::CodexDevice(_) => {
let result = provider_auth::authenticate_provider_with_method(
args.provider,
method,
&s,
printer,
)
.await?;
match result {
LoginResult::OAuth { credential, .. } => (
OPENAI_CODEX_VAULT_SECRET_NAME.to_string(),
serde_json::to_string(&credential)?,
types::SecretType::Oauth,
),
LoginResult::ApiKey { .. } => {
unreachable!("Codex device authentication cannot produce an API key result")
}
}
}
}
LoginResult::OAuth { credential, .. } => (
OPENAI_CODEX_VAULT_SECRET_NAME.to_string(),
serde_json::to_string(&credential)?,
types::SecretType::Oauth,
),
};
server
@ -64,3 +76,73 @@ pub(super) async fn login_command(
fabro_util::printerr!(printer, " {} Saved {}", s.green.apply_to("✔"), name);
Ok(())
}
async fn login_with_server_api_key(
server: &server_client::Client,
requested_provider: ProviderId,
source: provider_auth::ApiKeySource,
s: &Styles,
printer: Printer,
) -> Result<(String, String)> {
let provider = server_provider(server, &requested_provider).await?;
let secret_name = provider.expected_secret_name.clone().with_context(|| {
format!(
"provider '{}' does not define a vault credential path",
provider.id
)
})?;
let request = AuthContextRequest::ApiKey {
provider_id: provider.id.clone(),
display_name: provider.display_name.clone(),
env_var_names: vec![secret_name.clone()],
api_key_url: provider.api_key_url.clone(),
};
provider_auth::present_to_user(&request, s, printer);
loop {
let key = provider_auth::read_api_key_from_source(&source, &secret_name).await?;
fabro_util::printerr!(printer, " {}", s.dim.apply_to("Validating API key..."));
match server.test_provider_credentials(&provider.id, &key).await {
Ok(()) => {
fabro_util::printerr!(printer, " {} API key is valid", s.green.apply_to("✔"));
return Ok((secret_name, key));
}
Err(err) => {
fabro_util::printerr!(printer, " [error] API key validation failed: {err}");
if matches!(source, provider_auth::ApiKeySource::Prompt) {
let retry = spawn_blocking(|| {
provider_auth::prompt_confirm("Try again with a different key?", true)
})
.await??;
if !retry {
return Ok((secret_name, key));
}
} else {
return Err(err).context("API key validation failed");
}
}
}
}
}
async fn server_provider(
server: &server_client::Client,
requested_provider: &ProviderId,
) -> Result<types::Provider> {
server
.list_providers()
.await?
.into_iter()
.find(|provider| provider_matches(provider, requested_provider))
.with_context(|| {
format!("provider '{requested_provider}' is not configured in the server model catalog")
})
}
fn provider_matches(provider: &types::Provider, requested_provider: &ProviderId) -> bool {
provider.id == *requested_provider
|| provider
.aliases
.iter()
.any(|alias| alias == requested_provider.as_str())
}

View file

@ -142,7 +142,10 @@ fn read_api_key_from_env_var(name: &str) -> Result<String> {
.with_context(|| format!("environment variable {name} did not contain an API key"))
}
async fn read_api_key_from_source(source: &ApiKeySource, prompt: &str) -> Result<String> {
pub(crate) async fn read_api_key_from_source(
source: &ApiKeySource,
prompt: &str,
) -> Result<String> {
match source {
ApiKeySource::Prompt => {
let prompt = prompt.to_string();

View file

@ -1,4 +1,6 @@
use fabro_test::{fabro_snapshot, test_context};
use httpmock::prelude::*;
use serde_json::json;
#[test]
fn help() {
@ -40,3 +42,76 @@ fn provider_login_rejects_json() {
let stderr = String::from_utf8(output.stderr).unwrap();
assert!(stderr.contains("--json is not supported for this command"));
}
#[test]
fn api_key_login_uses_server_provider_catalog_for_remote_target() {
let context = test_context!();
let server = MockServer::start();
let providers = server.mock(|when, then| {
when.method(GET).path("/api/v1/providers");
then.status(200)
.header("Content-Type", "application/json")
.json_body(json!({
"data": [{
"id": "openrouter",
"display_name": "OpenRouter",
"adapter": "openai_compatible",
"base_url": "https://openrouter.ai/api/v1",
"api_key_url": "https://openrouter.ai/keys",
"priority": 25,
"aliases": [],
"model_count": 1,
"default_model": "openrouter/test",
"configured": false,
"expected_secret_name": "OPENROUTER_API_KEY"
}]
}));
});
let validation = server.mock(|when, then| {
when.method(POST)
.path("/api/v1/providers/openrouter/credentials/test")
.json_body(json!({ "api_key": "sk-or-v1-test" }));
then.status(200)
.header("Content-Type", "application/json")
.json_body(json!({ "ok": true }));
});
let secret = server.mock(|when, then| {
when.method(POST)
.path("/api/v1/secrets")
.json_body_includes(
r#"{
"name": "OPENROUTER_API_KEY",
"value": "sk-or-v1-test",
"type": "token"
}"#,
);
then.status(200)
.header("Content-Type", "application/json")
.json_body(json!({
"name": "OPENROUTER_API_KEY",
"type": "token",
"created_at": "2026-06-25T00:00:00Z",
"updated_at": "2026-06-25T00:00:00Z"
}));
});
context
.command()
.args([
"provider",
"login",
"--server",
&server.url(""),
"--provider",
"openrouter",
"--api-key-stdin",
"--no-upgrade-check",
])
.write_stdin("sk-or-v1-test\n")
.assert()
.success();
providers.assert();
validation.assert();
secret.assert();
}

View file

@ -804,6 +804,32 @@ impl Client {
Ok(models)
}
pub async fn list_providers(&self) -> Result<Vec<types::Provider>> {
let response = self
.send_api(|client| async move { client.list_providers().send().await })
.await?;
convert_type::<_, Vec<types::Provider>>(response.into_inner().data)
}
pub async fn test_provider_credentials(
&self,
provider: &ProviderId,
api_key: &str,
) -> Result<()> {
self.send_api(|client| async move {
client
.test_provider_credentials()
.provider(provider.to_string())
.body(types::ProviderCredentialTestRequest {
api_key: api_key.to_string(),
})
.send()
.await
})
.await?;
Ok(())
}
pub async fn test_model(
&self,
id: &str,
@ -2250,6 +2276,29 @@ mod tests {
assert!(models.is_empty());
}
#[tokio::test]
async fn test_provider_credentials_posts_api_key() {
let server = MockServer::start_async().await;
let mock = server
.mock_async(|when, then| {
when.method(POST)
.path("/api/v1/providers/openrouter/credentials/test")
.json_body(json!({ "api_key": "sk-test" }));
then.status(200)
.header("Content-Type", "application/json")
.json_body(json!({ "ok": true }));
})
.await;
let client = Client::new_no_proxy(&server.url("")).unwrap();
client
.test_provider_credentials(&ProviderId::new("openrouter"), "sk-test")
.await
.unwrap();
mock.assert_async().await;
}
async fn oauth_client(
server: &MockServer,
) -> (tempfile::TempDir, Client, AuthStore, ServerTarget) {

View file

@ -36,12 +36,13 @@ pub use fabro_api::types::{
IntegrationConnectionStatus, IntegrationProvider, IntegrationStatus, LinkRunPullRequestRequest,
MergeRunPullRequestRequest, MergeRunPullRequestResponse, ModelReference, PaginatedEventList,
PaginatedRunList, PaginationMeta, PreflightResponse, PreviewUrlRequest, PreviewUrlResponse,
Provider, ProviderList, PruneRunEntry, PruneRunsRequest, PruneRunsResponse,
RenderWorkflowGraphDirection, RenderWorkflowGraphRequest, RewindRequest, RewindResponse, Run,
RunArtifactEntry, RunArtifactListResponse, RunBilling, RunBillingStage, RunBillingTotals,
RunError, RunManifest, RunStage, SandboxDetails, SandboxFileEntry, SandboxFileListResponse,
SandboxService, SandboxServiceListResponse, SshAccessRequest, SshAccessResponse, StageHandler,
StageState, StartRunRequest, SubmitAnswerRequest, SystemCpuResourceScope, SystemCpuResources,
Provider, ProviderCredentialTestRequest, ProviderCredentialTestResponse, ProviderList,
PruneRunEntry, PruneRunsRequest, PruneRunsResponse, RenderWorkflowGraphDirection,
RenderWorkflowGraphRequest, RewindRequest, RewindResponse, Run, RunArtifactEntry,
RunArtifactListResponse, RunBilling, RunBillingStage, RunBillingTotals, RunError, RunManifest,
RunStage, SandboxDetails, SandboxFileEntry, SandboxFileListResponse, SandboxService,
SandboxServiceListResponse, SshAccessRequest, SshAccessResponse, StageHandler, StageState,
StartRunRequest, SubmitAnswerRequest, SystemCpuResourceScope, SystemCpuResources,
SystemDiskResourceScope, SystemDiskResources, SystemInfoResponse, SystemIntegrationStatus,
SystemIntegrationsResponse, SystemMemoryResourceScope, SystemMemoryResources,
SystemRepairRunIssue, SystemRepairRunsResponse, SystemResourcesResponse, SystemRunCounts,

View file

@ -1,9 +1,15 @@
use std::sync::Arc;
use fabro_auth::ApiCredential;
use fabro_llm::client::Client as LlmClient;
use fabro_llm::model_test::{ModelTestStatus, run_basic_model_probe};
use fabro_redact::redact_string;
use super::super::{
ApiError, AppState, FromStr, HashSet, IntoResponse, Json, MAX_PAGE_OFFSET, ModelTestMode, Path,
ProviderId, ProviderList, Query, RequiredUser, Response, Router, State, StatusCode,
auth_issue_message, default_page_limit, error, get, post, run_model_test,
ProviderCredentialTestRequest, ProviderCredentialTestResponse, ProviderId, ProviderList, Query,
RequiredUser, Response, Router, State, StatusCode, auth_issue_message, default_page_limit,
error, get, post, run_model_test,
};
use crate::diagnostics;
@ -12,6 +18,10 @@ pub(super) fn routes() -> Router<Arc<AppState>> {
.route("/models", get(list_models))
.route("/models/{id}/test", post(test_model))
.route("/providers", get(list_providers))
.route(
"/providers/{provider}/credentials/test",
post(test_provider_credentials),
)
.route("/providers/test", post(test_providers))
}
@ -95,6 +105,72 @@ async fn list_providers(_auth: RequiredUser, State(state): State<Arc<AppState>>)
(StatusCode::OK, Json(ProviderList { data })).into_response()
}
async fn test_provider_credentials(
_auth: RequiredUser,
State(state): State<Arc<AppState>>,
Path(provider): Path<String>,
Json(body): Json<ProviderCredentialTestRequest>,
) -> Response {
if body.api_key.trim().is_empty() {
return ApiError::bad_request("api_key is required").into_response();
}
let requested_provider = ProviderId::new(provider);
let catalog = state.catalog();
let Some(catalog_provider) = catalog.provider(&requested_provider) else {
return ApiError::not_found(format!("Provider not found: {requested_provider}"))
.into_response();
};
if catalog_provider.auth.is_none() {
return ApiError::bad_request(format!(
"provider '{}' does not define an API-key credential path",
catalog_provider.id,
))
.into_response();
}
let provider_id = catalog_provider.id.clone();
let credential =
match ApiCredential::from_api_key(provider_id.clone(), body.api_key, catalog.as_ref()) {
Ok(credential) => credential,
Err(err) => {
return ApiError::bad_request(err.to_string()).into_response();
}
};
let client = match LlmClient::from_credentials(vec![credential], Arc::clone(&catalog)).await {
Ok(client) => Arc::new(client),
Err(err) => {
error!(provider = %provider_id, error = ?err, "Failed to create LLM client for provider credential validation");
return ApiError::new(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create LLM client: {err}"),
)
.into_response();
}
};
let Some(model) = catalog.probe_for_provider(&provider_id) else {
return ApiError::bad_request(format!(
"provider '{provider_id}' does not define a probe model"
))
.into_response();
};
let outcome = run_basic_model_probe(&model.id, &provider_id, client).await;
match outcome.status {
ModelTestStatus::Ok => (
StatusCode::OK,
Json(ProviderCredentialTestResponse { ok: true }),
)
.into_response(),
ModelTestStatus::Error => {
let message = outcome
.error_message
.unwrap_or_else(|| "provider credential validation failed".to_string());
ApiError::new(StatusCode::UNPROCESSABLE_ENTITY, redact_string(&message)).into_response()
}
}
}
async fn test_providers(_auth: RequiredUser, State(state): State<Arc<AppState>>) -> Response {
match diagnostics::test_llm_providers(&state).await {
Ok(report) => (StatusCode::OK, Json(report)).into_response(),

View file

@ -6321,6 +6321,79 @@ async fn test_model_invalid_mode_returns_400() {
assert_status!(response, StatusCode::BAD_REQUEST).await;
}
#[tokio::test]
async fn test_provider_credentials_uses_app_state_catalog() {
let upstream = MockServer::start();
let completion = upstream.mock(|when, then| {
when.method(POST)
.path("/chat/completions")
.header("authorization", "Bearer sk-test");
then.status(200)
.header("content-type", "application/json")
.json_body(json!({
"id": "chatcmpl_test",
"object": "chat.completion",
"created": 1_700_000_000,
"model": "test-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "OK"},
"finish_reason": "stop"
}],
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
}));
});
let llm_catalog_settings: LlmCatalogSettings = toml::from_str(&format!(
r#"
[providers.acme]
display_name = "Acme"
adapter = "openai_compatible"
agent_profile = "openai"
base_url = "{}"
priority = 120
[providers.acme.auth]
credentials = ["vault:ACME_API_KEY"]
[models."acme-probe"]
provider = "acme"
api_id = "test-model"
display_name = "Acme Probe"
family = "acme"
default = true
probe = true
[models."acme-probe".limits]
context_window = 128000
[models."acme-probe".features]
tools = false
vision = false
reasoning = false
"#,
upstream.base_url()
))
.expect("catalog fixture should parse");
let state = TestAppStateBuilder::new()
.runtime_settings(default_test_server_settings(), RunLayer::default())
.max_concurrent_runs(5)
.llm_catalog_settings(llm_catalog_settings)
.build();
let app = crate::test_support::build_test_router(state);
let req = Request::builder()
.method("POST")
.uri(api("/providers/acme/credentials/test"))
.header("content-type", "application/json")
.body(Body::from(json!({ "api_key": "sk-test" }).to_string()))
.unwrap();
let response = app.oneshot(req).await.unwrap();
let body = response_json!(response, StatusCode::OK).await;
assert_eq!(body["ok"], true);
completion.assert();
}
#[tokio::test]
async fn list_models_filters_by_provider() {
let app = test_app_with();