mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-10-05 02:41:45 +00:00
feat: Amazon Bedrock as an opt-in built-in provider
The catalog entry ships disabled (the Ollama/OpenRouter pattern) with a curated multi-vendor lineup over Converse: Claude (Anthropic cache billing via per-model billing_policy), gpt-oss, Nova 2 Lite, Llama 4 Maverick, Mistral Large 3 / Devstral 2, DeepSeek V3.2, Qwen3 Coder Next, Kimi K2.5, GLM 5, MiniMax M2.5, and Nemotron 3 — cross-region inference profile ids where on-demand access requires them. Credential order puts an explicit AWS_BEARER_TOKEN_BEDROCK first with the always-resolving aws_sigv4 chain as fallback; shipping disabled is what resolves the original review concern about aws_sigv4 making Bedrock look configured for everyone. Deliberately absent rows, each needing a non-Converse surface: GPT-5.5/5.4 (Responses-only on bedrock-mantle), Claude Mythos 5 (Messages-only), Claude Fable 5 (Bedrock pins sampling params the Converse route does not gate yet). Plus the provider package: AWS_BEARER_TOKEN_BEDROCK env/secret registry, both upstream gitleaks Bedrock key rules (ABSK long-term, bedrock-api-key short-term), live e2e tests for both auth modes, integration docs, and the aws_sigv4 credential form in the generated config reference. Co-authored-by: depopry <ryan.neal@depop.com> Co-authored-by: Scott Werner <scott@sublayer.com>
This commit is contained in:
Scott Werner
parent
9ef794691a
commit
5a7ecf57e1
17 changed files with 881 additions and 41 deletions
|
|
@ -141,6 +141,16 @@ Fabro ships an [OpenRouter](/integrations/openrouter) provider definition with a
|
|||
enabled = true
|
||||
```
|
||||
|
||||
### Amazon Bedrock
|
||||
|
||||
Fabro ships an [Amazon Bedrock](/integrations/bedrock) provider definition with a curated multi-vendor catalog over Bedrock's Converse API, disabled by default. Enable it and authenticate with a Bedrock API key or AWS SigV4 credentials:
|
||||
|
||||
```toml title="settings.toml"
|
||||
[llm.providers.bedrock]
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
```
|
||||
|
||||
### Ollama
|
||||
|
||||
Fabro ships an Ollama provider definition that is disabled by default. Enable it in settings when you want Fabro to route through a local Ollama server:
|
||||
|
|
|
|||
|
|
@ -95,6 +95,7 @@
|
|||
"integrations/github",
|
||||
"integrations/daytona",
|
||||
"integrations/litellm",
|
||||
"integrations/bedrock",
|
||||
"integrations/openrouter",
|
||||
"integrations/slack",
|
||||
"integrations/brave-search"
|
||||
|
|
|
|||
132
docs/public/integrations/bedrock.mdx
Normal file
132
docs/public/integrations/bedrock.mdx
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
---
|
||||
title: "Amazon Bedrock"
|
||||
description: "Route Fabro models through Amazon Bedrock with SigV4 or API-key auth"
|
||||
---
|
||||
|
||||
[Amazon Bedrock](https://aws.amazon.com/bedrock/) hosts Anthropic, Amazon, Meta, Mistral, DeepSeek, Qwen, Moonshot, Z.AI, MiniMax, NVIDIA, and OpenAI open-weight models behind one AWS endpoint. Fabro ships a disabled `bedrock` provider entry with a curated model catalog over Bedrock's unified Converse API, so you can opt in from `settings.toml` without changing Fabro code.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An AWS account with [Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) granted for the models you want
|
||||
- Either a [Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) or working AWS credentials (environment keys, profile, IMDS, IRSA, SSO)
|
||||
|
||||
## Enable the provider
|
||||
|
||||
Add the provider override to `~/.fabro/settings.toml`:
|
||||
|
||||
```toml title="settings.toml"
|
||||
_version = 1
|
||||
|
||||
[llm.providers.bedrock]
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
```
|
||||
|
||||
The SigV4 signing region is derived from `base_url` — change it to your Region's endpoint (`https://bedrock-runtime.<region>.amazonaws.com`, FIPS and China endpoints included).
|
||||
|
||||
## Configure credentials
|
||||
|
||||
Two auth modes, tried in order:
|
||||
|
||||
**Bedrock API key** (simplest): store the key and Fabro sends it as a bearer token.
|
||||
|
||||
```bash
|
||||
fabro secret set AWS_BEARER_TOKEN_BEDROCK bedrock-api-key-...
|
||||
# or for standalone local runs
|
||||
export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
|
||||
```
|
||||
|
||||
**AWS SigV4** (IAM-scoped): with no API key configured, Fabro signs each request using the AWS default credential chain — environment keys, shared profile, EC2/ECS instance roles, IRSA/web identity, SSO. Expiring session credentials refresh automatically. The catalog declares this as the `aws_sigv4` credential source:
|
||||
|
||||
```toml
|
||||
[llm.providers.bedrock.auth]
|
||||
credentials = ["env:AWS_BEARER_TOKEN_BEDROCK", "aws_sigv4"]
|
||||
```
|
||||
|
||||
## Included models
|
||||
|
||||
The built-in catalog curates Converse-capable models, using cross-region inference profile ids (`us.`/`global.` prefixes) where on-demand access requires them:
|
||||
|
||||
| Fabro model ID | Notes |
|
||||
| --- | --- |
|
||||
| `us.anthropic.claude-sonnet-4-6` | Provider default; Anthropic cache billing |
|
||||
| `us.anthropic.claude-opus-4-8` | Anthropic cache billing |
|
||||
| `us.anthropic.claude-haiku-4-5` | Provider small default |
|
||||
| `us.anthropic.claude-fable-5` | Frontier; sampling params pinned by Bedrock (Fabro drops `temperature`/`top_p` automatically); requires the account-level `provider_data_share` opt-in |
|
||||
| `openai.gpt-oss-120b`, `openai.gpt-oss-20b` | OpenAI open-weights |
|
||||
| `amazon.nova-2-lite` | Vision |
|
||||
| `meta.llama4-maverick` | Vision |
|
||||
| `mistral.mistral-large-3`, `mistral.devstral-2` | |
|
||||
| `deepseek.v3-2`, `qwen.qwen3-coder-next` | |
|
||||
| `moonshotai.kimi-k2.5`, `zai.glm-5` | |
|
||||
| `minimax.minimax-m2.5`, `nvidia.nemotron-3-super` | |
|
||||
|
||||
Any other Converse-capable Bedrock model can be added as a settings model entry with `provider = "bedrock"` and the Bedrock model or inference-profile id as `api_id`.
|
||||
|
||||
Not included on this provider: Claude Mythos 5 (Anthropic-Messages-only on `bedrock-mantle`, limited preview). OpenAI's frontier models live on the companion `bedrock-openai` provider below.
|
||||
|
||||
## OpenAI frontier models (GPT-5.5 / GPT-5.4)
|
||||
|
||||
GPT-5.5 and GPT-5.4 on Bedrock are served only by the `bedrock-mantle` endpoint's OpenAI Responses API — a different surface than Converse. Fabro ships a companion `bedrock-openai` provider for them: the same AWS account and `AWS_BEARER_TOKEN_BEDROCK` key, pointed at the mantle endpoint over the OpenAI dialect.
|
||||
|
||||
```toml title="settings.toml"
|
||||
[llm.providers.bedrock-openai]
|
||||
enabled = true
|
||||
# regional: change to https://bedrock-mantle.<region>.api.aws/openai/v1
|
||||
```
|
||||
|
||||
```bash
|
||||
fabro model test --model openai.gpt-5.5
|
||||
```
|
||||
|
||||
Auth on this provider is Bedrock-API-key only (mantle SigV4 uses a different signing name than the runtime endpoint). Fabro always sends `store: false`, so nothing is retained under mantle's default 30-day response storage.
|
||||
|
||||
## Use Bedrock models
|
||||
|
||||
```bash
|
||||
fabro model list --provider bedrock
|
||||
fabro model test --model us.anthropic.claude-sonnet-4-6
|
||||
fabro run workflow.fabro --model deepseek.v3-2
|
||||
```
|
||||
|
||||
## Prompt caching
|
||||
|
||||
Claude models cache automatically when the catalog row declares `prompt_cache`: Fabro places Converse `cachePoint` blocks after the system prompt, the tool definitions, and the conversation prefix — the same placement as the direct Anthropic provider. Cache reads and writes price Anthropic-style via the per-model `billing_policy`.
|
||||
|
||||
## Converse extensions
|
||||
|
||||
Bedrock-specific request fields pass through verbatim via `provider_options.bedrock` on API/SDK requests — the keys merge into the top level of the Converse envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "us.anthropic.claude-sonnet-4-6",
|
||||
"provider_options": {
|
||||
"bedrock": {
|
||||
"additionalModelRequestFields": { "top_k": 200 },
|
||||
"guardrailConfig": { "guardrailIdentifier": "gr-abc", "guardrailVersion": "1" },
|
||||
"serviceTier": { "type": "flex" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"no AWS credentials provider found"** — Neither an API key nor any AWS chain source resolved. Set `AWS_BEARER_TOKEN_BEDROCK`, or configure standard AWS credentials.
|
||||
|
||||
**`AccessDeniedException` / 403** — The IAM principal lacks `bedrock:InvokeModel*` for the model, or model access has not been granted in the Bedrock console for your Region.
|
||||
|
||||
**`ValidationException` mentioning on-demand throughput** — The model requires an inference-profile id; use the `us.`/`global.`-prefixed id from the catalog rather than the bare model id.
|
||||
|
||||
**`ThrottlingException`** — Account-level Bedrock quota; consider cross-region inference profiles or a quota increase.
|
||||
|
||||
## Further reading
|
||||
|
||||
<Columns cols={2}>
|
||||
<Card title="Models" icon="microchip" href="/core-concepts/models">
|
||||
How Fabro routes model IDs, providers, and fallbacks.
|
||||
</Card>
|
||||
<Card title="Settings Configuration" icon="gear" href="/reference/user-configuration">
|
||||
Full reference for `[llm.providers.<id>]` and `[llm.models.<id>]`.
|
||||
</Card>
|
||||
</Columns>
|
||||
|
|
@ -198,7 +198,7 @@ x-team-secret = { vault = "gateway_team_secret" }
|
|||
| `billing_policy` | `"openai"` \| `"anthropic"` \| `"gemini"` \| `"none"` | derived from `adapter` | Provider-owned billing algorithm for usage estimates. Override for exceptional providers such as local no-billing runtimes. |
|
||||
| `base_url` | string | built-in value or adapter runtime default | Provider API base URL. Required for most custom OpenAI-compatible providers. |
|
||||
| `auth` | table | omitted | API-key auth config. Omit the table entirely for providers that need no API key; any `extra_headers` are still attached. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>` and `env:<NAME>`. Literal secret strings are rejected. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>`, `env:<NAME>`, and `aws_sigv4` (sign requests from the AWS default credential chain — Bedrock). Literal secret strings are rejected. |
|
||||
| `auth.header` | `"bearer"` or `{ custom = "Header-Name" }` | `"bearer"` | Primary API-key header policy. Omit when the provider uses a standard bearer token. |
|
||||
| `extra_headers` | table | `{}` | Additional headers attached to provider requests. Values must be typed refs: `{ literal = "..." }`, `{ env = "NAME" }`, or `{ vault = "NAME" }`. |
|
||||
| `priority` | integer | `0` | Higher-priority configured providers win default selection; ties use canonical provider ID. |
|
||||
|
|
|
|||
|
|
@ -1016,8 +1016,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1026,7 +1026,7 @@ mod tests {
|
|||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let args = AgentArgs {
|
||||
prompt: "test".to_string(),
|
||||
provider: Some("bedrock".to_string()),
|
||||
provider: Some("acme-aws".to_string()),
|
||||
model: None,
|
||||
permissions: None,
|
||||
auto_approve: false,
|
||||
|
|
@ -1037,7 +1037,7 @@ mod tests {
|
|||
};
|
||||
|
||||
let provider_id = parse_provider(&args).unwrap();
|
||||
assert_eq!(provider_id, ProviderId::new("bedrock"));
|
||||
assert_eq!(provider_id, ProviderId::new("acme-aws"));
|
||||
assert_eq!(
|
||||
profile_kind_for_provider(&catalog, &provider_id, None).unwrap(),
|
||||
AgentProfileKind::OpenAi
|
||||
|
|
@ -1049,8 +1049,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1058,9 +1058,9 @@ mod tests {
|
|||
});
|
||||
settings
|
||||
.models
|
||||
.insert("bedrock-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("bedrock".to_string()),
|
||||
display_name: Some("Bedrock Claude".to_string()),
|
||||
.insert("acme-aws-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("acme-aws".to_string()),
|
||||
display_name: Some("Acme AWS Claude".to_string()),
|
||||
family: Some("claude".to_string()),
|
||||
default: Some(true),
|
||||
limits: Some(SettingsModelLimits {
|
||||
|
|
@ -1081,7 +1081,7 @@ mod tests {
|
|||
let args = AgentArgs {
|
||||
prompt: "test".to_string(),
|
||||
provider: None,
|
||||
model: Some("bedrock-claude".to_string()),
|
||||
model: Some("acme-aws-claude".to_string()),
|
||||
permissions: None,
|
||||
auto_approve: false,
|
||||
debug: false,
|
||||
|
|
@ -1092,7 +1092,7 @@ mod tests {
|
|||
|
||||
assert_eq!(
|
||||
resolve_provider_id(&catalog, &args).unwrap(),
|
||||
ProviderId::new("bedrock")
|
||||
ProviderId::new("acme-aws")
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -1101,8 +1101,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1124,7 +1124,7 @@ mod tests {
|
|||
|
||||
assert_eq!(
|
||||
resolve_provider_id(&catalog, &args).unwrap(),
|
||||
ProviderId::new("bedrock")
|
||||
ProviderId::new("acme-aws")
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -1133,8 +1133,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1142,9 +1142,9 @@ mod tests {
|
|||
});
|
||||
settings
|
||||
.models
|
||||
.insert("bedrock-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("bedrock".to_string()),
|
||||
display_name: Some("Bedrock Claude".to_string()),
|
||||
.insert("acme-aws-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("acme-aws".to_string()),
|
||||
display_name: Some("Acme AWS Claude".to_string()),
|
||||
family: Some("claude".to_string()),
|
||||
default: Some(true),
|
||||
agent_profile: Some(AgentProfileKind::Anthropic),
|
||||
|
|
@ -1167,8 +1167,8 @@ mod tests {
|
|||
assert_eq!(
|
||||
profile_kind_for_provider(
|
||||
&catalog,
|
||||
&ProviderId::new("bedrock"),
|
||||
Some("bedrock-claude")
|
||||
&ProviderId::new("acme-aws"),
|
||||
Some("acme-aws-claude")
|
||||
)
|
||||
.unwrap(),
|
||||
AgentProfileKind::Anthropic
|
||||
|
|
@ -1180,20 +1180,20 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
..ProviderCatalogSettings::default()
|
||||
});
|
||||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let provider_id = ProviderId::new("bedrock");
|
||||
let provider_id = ProviderId::new("acme-aws");
|
||||
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "bedrock-claude-sonnet-4-6");
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "acme-aws-claude-sonnet-4-6");
|
||||
|
||||
assert_eq!(model_id.provider(), &provider_id);
|
||||
assert_eq!(model_id.model_id(), "bedrock-claude-sonnet-4-6");
|
||||
assert_eq!(model_id.model_id(), "acme-aws-claude-sonnet-4-6");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -1201,20 +1201,20 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::Anthropic),
|
||||
..ProviderCatalogSettings::default()
|
||||
});
|
||||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let provider_id = ProviderId::new("bedrock");
|
||||
let provider_id = ProviderId::new("acme-aws");
|
||||
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "bedrock-claude-sonnet-4-6");
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "acme-aws-claude-sonnet-4-6");
|
||||
|
||||
assert_eq!(model_id.provider(), &provider_id);
|
||||
assert_eq!(model_id.model_id(), "bedrock-claude-sonnet-4-6");
|
||||
assert_eq!(model_id.model_id(), "acme-aws-claude-sonnet-4-6");
|
||||
}
|
||||
|
||||
// subagent tool registration tests
|
||||
|
|
|
|||
|
|
@ -610,6 +610,7 @@ mod tests {
|
|||
r#"
|
||||
[providers.bedrock]
|
||||
adapter = "bedrock"
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.eu-west-1.amazonaws.com"
|
||||
|
||||
[providers.bedrock.auth]
|
||||
|
|
|
|||
|
|
@ -333,7 +333,7 @@ fn exec_accepts_configured_custom_provider_from_settings() {
|
|||
let context = test_context!();
|
||||
context.write_home(
|
||||
".fabro/settings.toml",
|
||||
"_version = 1\n\n[llm.providers.bedrock]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.bedrock.auth]\ncredentials = [\"env:BEDROCK_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"bedrock\"\nname = \"bedrock-claude-sonnet-4-6\"\n",
|
||||
"_version = 1\n\n[llm.providers.acme-aws]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.acme-aws.auth]\ncredentials = [\"env:ACME_AWS_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"acme-aws\"\nname = \"acme-claude-sonnet-4-6\"\n",
|
||||
);
|
||||
|
||||
let mut cmd = context.exec_cmd();
|
||||
|
|
@ -350,7 +350,7 @@ fn exec_accepts_configured_custom_provider_from_settings() {
|
|||
exit_code: 1
|
||||
----- stdout -----
|
||||
----- stderr -----
|
||||
× LLM credentials not configured for provider 'bedrock'
|
||||
× LLM credentials not configured for provider 'acme-aws'
|
||||
");
|
||||
}
|
||||
|
||||
|
|
@ -398,7 +398,7 @@ fn exec_server_target_accepts_configured_custom_provider_from_settings() {
|
|||
let context = test_context!();
|
||||
context.write_home(
|
||||
".fabro/settings.toml",
|
||||
"_version = 1\n\n[llm.providers.bedrock]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.bedrock.auth]\ncredentials = [\"env:BEDROCK_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"bedrock\"\nname = \"bedrock-claude-sonnet-4-6\"\n",
|
||||
"_version = 1\n\n[llm.providers.acme-aws]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.acme-aws.auth]\ncredentials = [\"env:ACME_AWS_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"acme-aws\"\nname = \"acme-claude-sonnet-4-6\"\n",
|
||||
);
|
||||
let server = MockServer::start();
|
||||
server.mock(|when, then| {
|
||||
|
|
@ -425,7 +425,7 @@ fn exec_server_target_accepts_configured_custom_provider_from_settings() {
|
|||
"expected remote server failure marker, got: {stderr}"
|
||||
);
|
||||
assert!(
|
||||
!stderr.contains("unknown provider: bedrock"),
|
||||
!stderr.contains("unknown provider: acme-aws"),
|
||||
"exec should resolve custom providers from settings for remote transport: {stderr}"
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -244,7 +244,7 @@ x-team-secret = { vault = "gateway_team_secret" }
|
|||
| `billing_policy` | `"openai"` \| `"anthropic"` \| `"gemini"` \| `"none"` | derived from `adapter` | Provider-owned billing algorithm for usage estimates. Override for exceptional providers such as local no-billing runtimes. |
|
||||
| `base_url` | string | built-in value or adapter runtime default | Provider API base URL. Required for most custom OpenAI-compatible providers. |
|
||||
| `auth` | table | omitted | API-key auth config. Omit the table entirely for providers that need no API key; any `extra_headers` are still attached. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>` and `env:<NAME>`. Literal secret strings are rejected. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>`, `env:<NAME>`, and `aws_sigv4` (sign requests from the AWS default credential chain — Bedrock). Literal secret strings are rejected. |
|
||||
| `auth.header` | `"bearer"` or `{ custom = "Header-Name" }` | `"bearer"` | Primary API-key header policy. Omit when the provider uses a standard bearer token. |
|
||||
| `extra_headers` | table | `{}` | Additional headers attached to provider requests. Values must be typed refs: `{ literal = "..." }`, `{ env = "NAME" }`, or `{ vault = "NAME" }`. |
|
||||
| `priority` | integer | `0` | Higher-priority configured providers win default selection; ties use canonical provider ID. |
|
||||
|
|
|
|||
|
|
@ -150,7 +150,11 @@ pub(super) fn map_stop_reason(reason: Option<&str>) -> FinishReason {
|
|||
None | Some("end_turn" | "stop_sequence") => FinishReason::Stop,
|
||||
Some("max_tokens" | "model_context_window_exceeded") => FinishReason::Length,
|
||||
Some("tool_use") => FinishReason::ToolCalls,
|
||||
Some("guardrail_intervened" | "content_filtered") => FinishReason::ContentFilter,
|
||||
// `refusal` is the Claude 5 blocking-classifier stop, passed through
|
||||
// by Bedrock for Fable-class models.
|
||||
Some("guardrail_intervened" | "content_filtered" | "refusal") => {
|
||||
FinishReason::ContentFilter
|
||||
}
|
||||
Some(other) => FinishReason::Other(other.to_string()),
|
||||
}
|
||||
}
|
||||
|
|
@ -193,6 +197,10 @@ mod tests {
|
|||
map_stop_reason(Some("content_filtered")),
|
||||
FinishReason::ContentFilter
|
||||
);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("refusal")),
|
||||
FinishReason::ContentFilter
|
||||
);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("malformed_tool_use")),
|
||||
FinishReason::Other("malformed_tool_use".to_string())
|
||||
|
|
|
|||
|
|
@ -45,14 +45,25 @@ pub(super) fn encode(ctx: &CodecCtx<'_>, stream: bool) -> Result<EncodedRequest,
|
|||
}
|
||||
body.insert("messages".to_string(), Value::Array(messages));
|
||||
|
||||
// Models with `sampling_params = false` reject classic sampling knobs
|
||||
// (Claude Fable 5 pins temperature on Bedrock too).
|
||||
let (temperature, top_p) = if ctx
|
||||
.model
|
||||
.is_none_or(fabro_model::Model::supports_sampling_params)
|
||||
{
|
||||
(request.temperature, request.top_p)
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
|
||||
let mut inference = Map::new();
|
||||
if let Some(max_tokens) = request.max_tokens {
|
||||
inference.insert("maxTokens".to_string(), json!(max_tokens));
|
||||
}
|
||||
if let Some(temperature) = request.temperature {
|
||||
if let Some(temperature) = temperature {
|
||||
inference.insert("temperature".to_string(), json!(temperature));
|
||||
}
|
||||
if let Some(top_p) = request.top_p {
|
||||
if let Some(top_p) = top_p {
|
||||
inference.insert("topP".to_string(), json!(top_p));
|
||||
}
|
||||
if let Some(stop) = &request.stop_sequences {
|
||||
|
|
@ -303,6 +314,8 @@ fn merge_provider_options(body: &mut Value, provider_options: Option<&Value>, pr
|
|||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use fabro_model::Catalog;
|
||||
use fabro_model::catalog::LlmCatalogSettings;
|
||||
use serde_json::json;
|
||||
|
||||
use super::*;
|
||||
|
|
@ -476,6 +489,52 @@ mod tests {
|
|||
assert!(encode(&ctx, false).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sampling_params_false_drops_temperature_and_top_p() {
|
||||
let settings: LlmCatalogSettings = toml::from_str(
|
||||
r#"
|
||||
[providers.bedrock]
|
||||
adapter = "bedrock"
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
|
||||
[models."pinned-model"]
|
||||
provider = "bedrock"
|
||||
display_name = "Pinned"
|
||||
family = "claude-5"
|
||||
default = true
|
||||
|
||||
[models."pinned-model".limits]
|
||||
context_window = 100000
|
||||
|
||||
[models."pinned-model".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
sampling_params = false
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let catalog = Catalog::from_settings(&settings).unwrap();
|
||||
|
||||
let mut request = base_request("pinned-model");
|
||||
request.top_p = Some(0.9);
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request: &request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "pinned-model",
|
||||
model: catalog.get("pinned-model"),
|
||||
params: ¶ms,
|
||||
};
|
||||
let encoded = encode(&ctx, false).unwrap();
|
||||
|
||||
let inference = &encoded.body["inferenceConfig"];
|
||||
assert!(inference.get("temperature").is_none());
|
||||
assert!(inference.get("topP").is_none());
|
||||
assert_eq!(inference["maxTokens"], 256);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_points_follow_the_anthropic_placement() {
|
||||
let mut messages = vec![
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ use std::sync::Arc;
|
|||
use fabro_llm::error::ProviderErrorKind;
|
||||
use fabro_llm::provider::ProviderAdapter;
|
||||
use fabro_llm::providers::{
|
||||
AnthropicAdapter, GeminiAdapter, OpenAiAdapter, OpenAiCompatibleAdapter,
|
||||
AnthropicAdapter, BedrockAdapter, GeminiAdapter, OpenAiAdapter, OpenAiCompatibleAdapter,
|
||||
};
|
||||
use fabro_llm::types::{CostSource, FinishReason, Message, Request};
|
||||
use fabro_model::Catalog;
|
||||
|
|
@ -176,6 +176,69 @@ async fn gemini_complete() {
|
|||
assert_eq!(response.provider, "gemini");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_BEARER_TOKEN_BEDROCK"))]
|
||||
async fn bedrock_complete_with_api_key() {
|
||||
let token = std::env::var(EnvVars::AWS_BEARER_TOKEN_BEDROCK)
|
||||
.expect("AWS_BEARER_TOKEN_BEDROCK must be set");
|
||||
let adapter =
|
||||
BedrockAdapter::new_api_key(token, "https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
.unwrap()
|
||||
.with_name("bedrock");
|
||||
// Amazon Nova: first-party, no Anthropic-approval gate and no third-party
|
||||
// marketplace subscription, so this runs on any Bedrock-enabled account.
|
||||
let request = make_request("us.amazon.nova-2-lite-v1:0");
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert!(response.usage.output_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_ACCESS_KEY_ID"))]
|
||||
async fn bedrock_complete_with_sigv4() {
|
||||
let adapter = BedrockAdapter::new_sigv4("https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
.unwrap()
|
||||
.with_name("bedrock");
|
||||
// First-party Nova — see bedrock_complete_with_api_key for why.
|
||||
let request = make_request("us.amazon.nova-2-lite-v1:0");
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_BEARER_TOKEN_BEDROCK"))]
|
||||
async fn bedrock_openai_frontier_complete() {
|
||||
let token = std::env::var(EnvVars::AWS_BEARER_TOKEN_BEDROCK)
|
||||
.expect("AWS_BEARER_TOKEN_BEDROCK must be set");
|
||||
// GPT-5.x on Bedrock is the bedrock-mantle Responses surface: the plain
|
||||
// openai adapter pointed at the mantle endpoint with the Bedrock key as
|
||||
// the bearer token.
|
||||
let adapter = OpenAiAdapter::new(token)
|
||||
.with_base_url("https://bedrock-mantle.us-east-1.api.aws/openai/v1")
|
||||
.with_name("bedrock-openai");
|
||||
let request = Request {
|
||||
temperature: None,
|
||||
..make_request("openai.gpt-5.5")
|
||||
};
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock-openai");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("OPENROUTER_API_KEY"))]
|
||||
async fn openrouter_complete() {
|
||||
let api_key =
|
||||
|
|
|
|||
|
|
@ -1982,6 +1982,112 @@ reasoning = false
|
|||
assert_eq!(model.provider, ProviderId::new("acme"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_bedrock_provider_is_opt_in() {
|
||||
let bedrock = ProviderId::new("bedrock");
|
||||
let builtin = Catalog::builtin();
|
||||
|
||||
assert!(builtin.provider(&bedrock).is_none());
|
||||
assert!(builtin.list(Some(&bedrock)).is_empty());
|
||||
|
||||
let catalog = Catalog::from_builtin_with_overrides(&minimal_settings(
|
||||
r"
|
||||
[providers.bedrock]
|
||||
enabled = true
|
||||
",
|
||||
))
|
||||
.expect("enabled Bedrock override should build from the built-in provider settings");
|
||||
|
||||
let provider = catalog
|
||||
.provider(&bedrock)
|
||||
.expect("enabled Bedrock provider should be present");
|
||||
assert_eq!(provider.adapter, AdapterKind::Bedrock);
|
||||
assert_eq!(provider.codec, CodecKind::BedrockConverse);
|
||||
assert_eq!(
|
||||
provider.base_url.as_deref(),
|
||||
Some("https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
);
|
||||
// Bearer key first, SigV4 chain as the fallback.
|
||||
assert_eq!(provider.auth.as_ref().unwrap().credentials, vec![
|
||||
CredentialRef::Env("AWS_BEARER_TOKEN_BEDROCK".to_string()),
|
||||
CredentialRef::AwsSigv4,
|
||||
]);
|
||||
|
||||
// Claude rows bill Anthropic-style; open-weights rows override the
|
||||
// provider's Anthropic defaults the other way.
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-sonnet-4-6")
|
||||
.unwrap()
|
||||
.billing_policy,
|
||||
BillingPolicy::Anthropic
|
||||
);
|
||||
assert_eq!(
|
||||
catalog.model_settings("zai.glm-5").unwrap().billing_policy,
|
||||
BillingPolicy::OpenAi
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-haiku-4-5")
|
||||
.unwrap()
|
||||
.api_id,
|
||||
"us.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.default_for_provider(&bedrock)
|
||||
.map(|model| model.id.as_str()),
|
||||
Some("us.anthropic.claude-sonnet-4-6")
|
||||
);
|
||||
// Fable 5 ships with sampling params pinned off (the Converse
|
||||
// encoder drops temperature/top_p for it).
|
||||
let fable = catalog
|
||||
.get("us.anthropic.claude-fable-5")
|
||||
.expect("fable row should be present");
|
||||
assert!(!fable.features.sampling_params);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-fable-5")
|
||||
.unwrap()
|
||||
.billing_policy,
|
||||
BillingPolicy::Anthropic
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_bedrock_openai_provider_is_opt_in() {
|
||||
let provider_id = ProviderId::new("bedrock-openai");
|
||||
let builtin = Catalog::builtin();
|
||||
|
||||
assert!(builtin.provider(&provider_id).is_none());
|
||||
|
||||
let catalog = Catalog::from_builtin_with_overrides(&minimal_settings(
|
||||
r"
|
||||
[providers.bedrock-openai]
|
||||
enabled = true
|
||||
",
|
||||
))
|
||||
.expect("enabled bedrock-openai override should build");
|
||||
|
||||
let provider = catalog
|
||||
.provider(&provider_id)
|
||||
.expect("enabled bedrock-openai provider should be present");
|
||||
// OpenAI frontier on Bedrock rides the existing openai_responses
|
||||
// dialect against the bedrock-mantle endpoint — pure configuration.
|
||||
assert_eq!(provider.adapter, AdapterKind::OpenAi);
|
||||
assert_eq!(provider.codec, CodecKind::OpenAiResponses);
|
||||
assert_eq!(
|
||||
provider.base_url.as_deref(),
|
||||
Some("https://bedrock-mantle.us-east-1.api.aws/openai/v1")
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.default_for_provider(&provider_id)
|
||||
.map(|model| model.id.as_str()),
|
||||
Some("openai.gpt-5.5")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_openrouter_provider_is_opt_in() {
|
||||
let openrouter = ProviderId::new("openrouter");
|
||||
|
|
|
|||
|
|
@ -0,0 +1,70 @@
|
|||
[providers.bedrock-openai]
|
||||
display_name = "Amazon Bedrock (OpenAI frontier)"
|
||||
adapter = "openai"
|
||||
api_key_url = "https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html"
|
||||
base_url = "https://bedrock-mantle.us-east-1.api.aws/openai/v1"
|
||||
priority = 19
|
||||
enabled = false
|
||||
|
||||
[providers.bedrock-openai.auth]
|
||||
credentials = ["env:AWS_BEARER_TOKEN_BEDROCK"]
|
||||
|
||||
# OpenAI's frontier models on Bedrock (GPT-5.5/5.4) are served ONLY by the
|
||||
# bedrock-mantle endpoint's OpenAI Responses API — they are not reachable
|
||||
# through Converse or InvokeModel on bedrock-runtime. That surface speaks
|
||||
# the openai_responses dialect with a Bedrock API key as the bearer token,
|
||||
# so this companion provider row is pure configuration over the existing
|
||||
# openai adapter: same AWS account and key as the `bedrock` provider, a
|
||||
# different endpoint and wire dialect.
|
||||
#
|
||||
# Notes:
|
||||
# - Auth is Bedrock-API-key only on this row (SigV4 on mantle uses the
|
||||
# `bedrock-mantle` signing name, which the openai adapter does not do).
|
||||
# - bedrock-mantle is regional (13 regions); change base_url to
|
||||
# `https://bedrock-mantle.<region>.api.aws/openai/v1` as needed.
|
||||
# - Responses state: Fabro always sends `store: false`, so nothing is
|
||||
# retained under mantle's default 30-day Project retention.
|
||||
#
|
||||
# To enable, add to ~/.fabro/settings.toml:
|
||||
#
|
||||
# [llm.providers.bedrock-openai]
|
||||
# enabled = true
|
||||
|
||||
[models."openai.gpt-5.5"]
|
||||
provider = "bedrock-openai"
|
||||
display_name = "GPT-5.5 (Bedrock)"
|
||||
family = "gpt-5"
|
||||
default = true
|
||||
|
||||
[models."openai.gpt-5.5".limits]
|
||||
context_window = 272000
|
||||
max_output = 128000
|
||||
|
||||
[models."openai.gpt-5.5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
reasoning_effort = "levels"
|
||||
|
||||
[models."openai.gpt-5.5".costs]
|
||||
input_cost_per_mtok = 5.5
|
||||
output_cost_per_mtok = 33.0
|
||||
|
||||
[models."openai.gpt-5.4"]
|
||||
provider = "bedrock-openai"
|
||||
display_name = "GPT-5.4 (Bedrock)"
|
||||
family = "gpt-5"
|
||||
|
||||
[models."openai.gpt-5.4".limits]
|
||||
context_window = 272000
|
||||
max_output = 128000
|
||||
|
||||
[models."openai.gpt-5.4".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
reasoning_effort = "levels"
|
||||
|
||||
[models."openai.gpt-5.4".costs]
|
||||
input_cost_per_mtok = 2.75
|
||||
output_cost_per_mtok = 16.5
|
||||
372
lib/crates/fabro-model/src/catalog/providers/bedrock.toml
Normal file
372
lib/crates/fabro-model/src/catalog/providers/bedrock.toml
Normal file
|
|
@ -0,0 +1,372 @@
|
|||
[providers.bedrock]
|
||||
display_name = "Amazon Bedrock"
|
||||
adapter = "bedrock"
|
||||
api_key_url = "https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html"
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
priority = 20
|
||||
enabled = false
|
||||
|
||||
[providers.bedrock.auth]
|
||||
# An explicit Bedrock API key wins; SigV4 (the AWS default credential
|
||||
# chain, resolved at request time) is the fallback. `aws_sigv4` always
|
||||
# resolves, which is why this provider ships disabled: enabling it is the
|
||||
# operator's statement that AWS credentials are expected to work.
|
||||
credentials = ["env:AWS_BEARER_TOKEN_BEDROCK", "aws_sigv4"]
|
||||
|
||||
# To enable Bedrock, add the following to ~/.fabro/settings.toml:
|
||||
#
|
||||
# [llm.providers.bedrock]
|
||||
# enabled = true
|
||||
# base_url = "https://bedrock-runtime.<your-region>.amazonaws.com"
|
||||
#
|
||||
# The signing region is derived from the base_url. Authenticate with
|
||||
# either a Bedrock API key (AWS_BEARER_TOKEN_BEDROCK) or any AWS default
|
||||
# credential chain source (env keys, profile, IMDS, IRSA, SSO).
|
||||
#
|
||||
# Model ids use cross-region inference profiles (`us.` / `global.`
|
||||
# prefixes) where on-demand access requires them. Pricing rows are
|
||||
# best-effort estimates from June 2026 list prices.
|
||||
|
||||
# ---------- Anthropic Claude ----------
|
||||
#
|
||||
# Claude bills Anthropic-style cache reads/writes, so these rows override
|
||||
# the provider's billing default. Claude Fable 5 is deliberately absent:
|
||||
# its Bedrock deployment pins sampling parameters (temperature must be
|
||||
# unset) that the Converse route does not gate yet — a named follow-up.
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Sonnet 4.6 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
default = true
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".limits]
|
||||
context_window = 1000000
|
||||
max_output = 64000
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".costs]
|
||||
input_cost_per_mtok = 3.0
|
||||
output_cost_per_mtok = 15.0
|
||||
cache_input_cost_per_mtok = 0.3
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Opus 4.8 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".limits]
|
||||
context_window = 1000000
|
||||
max_output = 128000
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".costs]
|
||||
input_cost_per_mtok = 5.0
|
||||
output_cost_per_mtok = 25.0
|
||||
cache_input_cost_per_mtok = 0.5
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5"]
|
||||
provider = "bedrock"
|
||||
api_id = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
display_name = "Claude Haiku 4.5 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
small_default = true
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".limits]
|
||||
context_window = 200000
|
||||
max_output = 64000
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".costs]
|
||||
input_cost_per_mtok = 1.0
|
||||
output_cost_per_mtok = 5.0
|
||||
cache_input_cost_per_mtok = 0.1
|
||||
|
||||
# ---------- OpenAI open-weights ----------
|
||||
#
|
||||
# GPT-5.5/5.4 are NOT here: on Bedrock they are Responses-API-only on the
|
||||
# bedrock-mantle endpoint (no Converse), a named follow-up route.
|
||||
|
||||
[models."openai.gpt-oss-120b"]
|
||||
provider = "bedrock"
|
||||
api_id = "openai.gpt-oss-120b-1:0"
|
||||
display_name = "GPT-OSS 120B (Bedrock)"
|
||||
family = "gpt-oss"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."openai.gpt-oss-120b".limits]
|
||||
context_window = 128000
|
||||
max_output = 16384
|
||||
|
||||
[models."openai.gpt-oss-120b".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."openai.gpt-oss-120b".costs]
|
||||
input_cost_per_mtok = 0.15
|
||||
output_cost_per_mtok = 0.60
|
||||
|
||||
[models."openai.gpt-oss-20b"]
|
||||
provider = "bedrock"
|
||||
api_id = "openai.gpt-oss-20b-1:0"
|
||||
display_name = "GPT-OSS 20B (Bedrock)"
|
||||
family = "gpt-oss"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."openai.gpt-oss-20b".limits]
|
||||
context_window = 128000
|
||||
max_output = 16384
|
||||
|
||||
[models."openai.gpt-oss-20b".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."openai.gpt-oss-20b".costs]
|
||||
input_cost_per_mtok = 0.07
|
||||
output_cost_per_mtok = 0.30
|
||||
|
||||
# ---------- Amazon Nova ----------
|
||||
|
||||
[models."amazon.nova-2-lite"]
|
||||
provider = "bedrock"
|
||||
api_id = "global.amazon.nova-2-lite-v1:0"
|
||||
display_name = "Nova 2 Lite (Bedrock)"
|
||||
family = "nova-2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."amazon.nova-2-lite".limits]
|
||||
context_window = 1000000
|
||||
max_output = 65536
|
||||
|
||||
[models."amazon.nova-2-lite".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."amazon.nova-2-lite".costs]
|
||||
input_cost_per_mtok = 0.30
|
||||
output_cost_per_mtok = 2.50
|
||||
|
||||
# ---------- Open-weights ----------
|
||||
|
||||
[models."meta.llama4-maverick"]
|
||||
provider = "bedrock"
|
||||
api_id = "us.meta.llama4-maverick-17b-instruct-v1:0"
|
||||
display_name = "Llama 4 Maverick (Bedrock)"
|
||||
family = "llama-4"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."meta.llama4-maverick".limits]
|
||||
context_window = 1000000
|
||||
max_output = 8192
|
||||
|
||||
[models."meta.llama4-maverick".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."mistral.mistral-large-3"]
|
||||
provider = "bedrock"
|
||||
api_id = "mistral.mistral-large-3-675b-instruct"
|
||||
display_name = "Mistral Large 3 (Bedrock)"
|
||||
family = "mistral-large"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."mistral.mistral-large-3".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."mistral.mistral-large-3".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."mistral.mistral-large-3".costs]
|
||||
input_cost_per_mtok = 0.50
|
||||
output_cost_per_mtok = 1.50
|
||||
|
||||
[models."mistral.devstral-2"]
|
||||
provider = "bedrock"
|
||||
api_id = "mistral.devstral-2-123b"
|
||||
display_name = "Devstral 2 (Bedrock)"
|
||||
family = "devstral"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."mistral.devstral-2".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."mistral.devstral-2".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."deepseek.v3-2"]
|
||||
provider = "bedrock"
|
||||
api_id = "deepseek.v3.2"
|
||||
display_name = "DeepSeek V3.2 (Bedrock)"
|
||||
family = "deepseek-v3"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."deepseek.v3-2".limits]
|
||||
context_window = 164000
|
||||
max_output = 8192
|
||||
|
||||
[models."deepseek.v3-2".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."deepseek.v3-2".costs]
|
||||
input_cost_per_mtok = 0.62
|
||||
output_cost_per_mtok = 1.85
|
||||
|
||||
[models."qwen.qwen3-coder-next"]
|
||||
provider = "bedrock"
|
||||
display_name = "Qwen3 Coder Next (Bedrock)"
|
||||
family = "qwen3"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."qwen.qwen3-coder-next".limits]
|
||||
context_window = 256000
|
||||
max_output = 16384
|
||||
|
||||
[models."qwen.qwen3-coder-next".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."moonshotai.kimi-k2.5"]
|
||||
provider = "bedrock"
|
||||
display_name = "Kimi K2.5 (Bedrock)"
|
||||
family = "kimi-k2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."moonshotai.kimi-k2.5".limits]
|
||||
context_window = 262144
|
||||
max_output = 16384
|
||||
|
||||
[models."moonshotai.kimi-k2.5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."moonshotai.kimi-k2.5".costs]
|
||||
input_cost_per_mtok = 0.60
|
||||
output_cost_per_mtok = 3.00
|
||||
|
||||
[models."zai.glm-5"]
|
||||
provider = "bedrock"
|
||||
display_name = "GLM 5 (Bedrock)"
|
||||
family = "glm"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."zai.glm-5".limits]
|
||||
context_window = 200000
|
||||
max_output = 128000
|
||||
|
||||
[models."zai.glm-5".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."zai.glm-5".costs]
|
||||
input_cost_per_mtok = 1.00
|
||||
output_cost_per_mtok = 3.20
|
||||
|
||||
[models."minimax.minimax-m2.5"]
|
||||
provider = "bedrock"
|
||||
display_name = "MiniMax M2.5 (Bedrock)"
|
||||
family = "minimax-m2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."minimax.minimax-m2.5".limits]
|
||||
context_window = 196000
|
||||
max_output = 8192
|
||||
|
||||
[models."minimax.minimax-m2.5".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."minimax.minimax-m2.5".costs]
|
||||
input_cost_per_mtok = 0.30
|
||||
output_cost_per_mtok = 1.20
|
||||
|
||||
[models."nvidia.nemotron-3-super"]
|
||||
provider = "bedrock"
|
||||
api_id = "nvidia.nemotron-super-3-120b"
|
||||
display_name = "Nemotron 3 Super (Bedrock)"
|
||||
family = "nemotron-3"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."nvidia.nemotron-3-super".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."nvidia.nemotron-3-super".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
# Claude Fable 5: adaptive thinking is always on server-side; the row pins
|
||||
# sampling_params = false so the Converse encoder drops temperature/top_p
|
||||
# (Bedrock rejects them for this model). Requires the account-level
|
||||
# provider_data_share opt-in in the Bedrock console. Effort-level mapping
|
||||
# through additionalModelRequestFields is a named follow-up, so
|
||||
# reasoning_effort stays undeclared here (requests carrying one are
|
||||
# rejected up front rather than silently dropped).
|
||||
|
||||
[models."us.anthropic.claude-fable-5"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Fable 5 (Bedrock)"
|
||||
family = "claude-5"
|
||||
billing_policy = "anthropic"
|
||||
|
||||
[models."us.anthropic.claude-fable-5".limits]
|
||||
context_window = 1000000
|
||||
max_output = 128000
|
||||
|
||||
[models."us.anthropic.claude-fable-5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
sampling_params = false
|
||||
|
||||
[models."us.anthropic.claude-fable-5".costs]
|
||||
input_cost_per_mtok = 10.0
|
||||
output_cost_per_mtok = 50.0
|
||||
cache_input_cost_per_mtok = 1.0
|
||||
|
|
@ -2546,6 +2546,20 @@ regex = '''\b(sk-[a-zA-Z0-9]{20}T3BlbkFJ[a-zA-Z0-9]{20})(?:['|\"|\n|\r|\s|\x60|;
|
|||
entropy = 3
|
||||
keywords = ["t3blbkfj"]
|
||||
|
||||
[[rules]]
|
||||
id = "aws-bedrock-long-term-api-key"
|
||||
description = "Found an AWS Bedrock long-term API key, posing a risk of unauthorized model invocation and billing."
|
||||
regex = '''\b(ABSK[A-Za-z0-9+/]{109,269}={0,2})(?:[\x60'\"\s;]|\\[nr]|$)'''
|
||||
entropy = 3
|
||||
keywords = ["absk"]
|
||||
|
||||
[[rules]]
|
||||
id = "aws-bedrock-short-term-api-key"
|
||||
description = "Found an AWS Bedrock short-term API key, posing a risk of unauthorized model invocation."
|
||||
regex = '''\b(bedrock-api-key-YmVkcm9jay5hbWF6b25hd3MuY29t[A-Za-z0-9+/=]{16,})(?:[\x60'\"\s;]|\\[nr]|$)'''
|
||||
entropy = 3
|
||||
keywords = ["bedrock-api-key-"]
|
||||
|
||||
[[rules]]
|
||||
id = "openrouter-api-key"
|
||||
description = "Found an OpenRouter API Key, posing a risk of unauthorized access to LLM provider routing and billing."
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ impl EnvVars {
|
|||
|
||||
// LLM providers and tool integrations
|
||||
pub const ANTHROPIC_API_KEY: &'static str = "ANTHROPIC_API_KEY";
|
||||
pub const AWS_BEARER_TOKEN_BEDROCK: &'static str = "AWS_BEARER_TOKEN_BEDROCK";
|
||||
pub const ANTHROPIC_BASE_URL: &'static str = "ANTHROPIC_BASE_URL";
|
||||
pub const BRAVE_SEARCH_API_KEY: &'static str = "BRAVE_SEARCH_API_KEY";
|
||||
pub const CHATGPT_ACCOUNT_ID: &'static str = "CHATGPT_ACCOUNT_ID";
|
||||
|
|
@ -179,6 +180,7 @@ mod tests {
|
|||
EnvVars::FABRO_WORKER_TOKEN,
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::ANTHROPIC_BASE_URL,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::CHATGPT_ACCOUNT_ID,
|
||||
EnvVars::GEMINI_API_KEY,
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ const BOOTSTRAP_SECRETS: &[&str] = &[
|
|||
|
||||
const OPTIONAL_VAULT_SECRETS: &[&str] = &[
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::FABRO_SLACK_APP_TOKEN,
|
||||
EnvVars::FABRO_SLACK_BOT_TOKEN,
|
||||
|
|
@ -86,6 +87,7 @@ mod tests {
|
|||
EnvVars::DAYTONA_API_KEY,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::GEMINI_API_KEY,
|
||||
EnvVars::INCEPTION_API_KEY,
|
||||
EnvVars::KIMI_API_KEY,
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue