mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
feat(llm): Amazon Bedrock provider — Converse codec, SigV4 + API-key auth (#459)
Adds **Amazon Bedrock** as an opt-in built-in provider, over Bedrock's unified **Converse / ConverseStream** API. One codec serves every Converse-capable family — Claude, Amazon Nova, Meta Llama, Mistral, DeepSeek, Moonshot Kimi, Z.AI GLM, MiniMax, NVIDIA Nemotron, and OpenAI gpt-oss — because AWS translates the envelope to each model's native dialect server-side. Auth is either **AWS SigV4** (the default credential chain — env / profile / IMDS / IRSA / SSO, resolved per request so sessions refresh) or a **Bedrock API key** (`AWS_BEARER_TOKEN_BEDROCK`, bearer). Disabled by default (the Ollama / OpenRouter opt-in pattern). This is the redo of #459's original Claude-only `InvokeModel` adapter, rebuilt on the gateway-refactor seams (#481–#497). @depopry's SigV4 signer, AWS event-stream frame decoder, `BedrockAuth`, the `aws_sigv4` credential grammar, `AdapterKind::Bedrock`, region-from-base_url, and the lean-deps decision are preserved and authored by him on the first two commits; the per-family `BedrockCodec` trait he wrote turned out to be the crate-wide `Codec` seam in miniature, so the refactor promoted exactly that shape. The original Claude-only description is preserved in a comment below. ## What's here - **`AdapterKind::Bedrock` × `CodecKind::BedrockConverse`** on the route, plus the `aws_sigv4` credential source (no static secret — the adapter signs at request time; `fabro-auth` stays AWS-free). *(@depopry)* - **SigV4 signer + AWS event-stream `FrameDecoder`** on the lean AWS stack (no `aws-sdk-bedrockruntime`; transport stays on `fabro-http`). Re-targeted at Converse's direct-JSON stream frames; the signer resolves credentials per request. *(@depopry)* - **`bedrock_converse` codec** — Converse envelope (`system[]`, typed content blocks, `inferenceConfig`, `toolConfig`), prompt caching via `cachePoint`, thinking-signature round-trip through `reasoningContent`, usage mapped onto the disjoint `TokenCounts` buckets, `provider_options.bedrock` passthrough. Plus the adapter shell and an event-stream byte loop beside the transport's shared SSE loop. - **Catalog**: `bedrock.toml` (Claude incl. Fable 5, Nova 2, Llama 4, Mistral, DeepSeek, Kimi, GLM, MiniMax, Nemotron, gpt-oss — cross-region inference-profile ids, per-model `billing_policy` so Claude bills Anthropic-style) and a companion **`bedrock-openai`** provider for GPT-5.5/5.4 over the `bedrock-mantle` Responses endpoint (pure config over the existing `openai_responses` codec, zero new code). - Secrets registry (`AWS_BEARER_TOKEN_BEDROCK`), gitleaks rules for both Bedrock key formats, the `docs/integrations/bedrock` guide, and live e2e tests. ## Live verification (confirmed end-to-end against a real AWS account) Verified on a real Bedrock account (us-east-2, SigV4 + bearer): - **SigV4 + Converse** — multiple families (Claude, Nova, DeepSeek, …) via the full settings → catalog → route → adapter → codec path. - **ConverseStream** — streaming deltas through the workflow engine. - **Multi-turn tool use** — agent loop with tool calls round-tripping (no-arg tools included). - **Multi-model routing** — Claude + DeepSeek pinned in one run through the single Converse codec. - **mantle Responses** — `openai.gpt-5.5` answered via the `bedrock-openai` provider (bearer auth). The exercise caught and fixed several issues that unit tests (static creds, mocked transports) could not — see the follow-up commits below. ## Follow-up fixes from live testing (commits on top of the foundation) 1. **Worker AWS env** — the workflow worker scrubs its env to an allowlist, so SigV4 (which re-resolves from the ambient chain per request) couldn't work through `fabro run`. The AWS credential-chain inputs now cross into the worker. 2. **Vault bearer key** — Bedrock was the only key-based provider missing a `vault:` credential ref, so `fabro secret set AWS_BEARER_TOKEN_BEDROCK` silently didn't feed it. Now resolves env → vault → SigV4. 3. **Converse tool-encoding hardening** — a no-arg tool call's `toolUse.input` is now a `{}` object (Bedrock rejects null), and every tool `inputSchema` gets a top-level `type: "object"` (strict families like DeepSeek reject a typeless schema Claude tolerates). 4. **Nova output cap** — `amazon.nova-2-lite` max_output 65536 → 65535 (Bedrock's per-request limit). Earlier fixes already folded into the foundation commits: the `aws-config` sleep-impl (default chain panicked) and AWS error-body decoding (top-level `message`/`Message`/`__type` → proper messages instead of "Unknown error"). ## Manual testing & setup See `docs/integrations/bedrock` — now documents the non-obvious account setup that live testing surfaced: the per-Region Anthropic use-case approval, `aws-marketplace:Subscribe` for third-party models, the Fable 5 / Mythos-class data-sharing opt-in, and the bearer-vs-SigV4 precedence override for running Converse + mantle side by side. ## Open decision / discussion - **Model-id naming** — Bedrock rows use dotted ids mirroring Bedrock's native inference-profile ids (`us.anthropic.claude-sonnet-4-6`, `openai.gpt-5.5`), which also makes them the wire `api_id`. Third scheme alongside bare ids and OpenRouter's `vendor/model` slashes. No collision risk (enforced at catalog build). Open to a uniform scheme if preferred. - **`BEDROCK_API_KEY` alias** — see the comment thread; the AWS console hands some users `export BEDROCK_API_KEY=` while the SDK-standard var is `AWS_BEARER_TOKEN_BEDROCK`. Question of whether to accept both. ## Deferred (named follow-ups) - **`qwen.qwen3-coder-next`** — omitted pending a verified Bedrock model/inference-profile id (its fabro id isn't a valid Bedrock identifier; needs an explicit `api_id`). Re-add once confirmed via `aws bedrock list-inference-profiles`. - **Claude Mythos 5** — Anthropic-Messages-only on `bedrock-mantle` (limited preview). - **Converse structured output** (`response_format` rejected with a clear error). - **`reasoning_effort` on Converse rows** via `additionalModelRequestFields` (the `bedrock-openai` GPT rows already accept effort levels). - **CountTokens** route (`count_input_tokens` returns `None`). ## Verification `cargo nextest run --workspace`: green except the pre-existing environment-dependent fabro-workflow failures (identical on main). clippy `-D warnings` + pinned-nightly fmt clean. Codec unit tests + adapter httpmock tests + frame-decoder/signer locks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Scott Werner <scott@sublayer.com> Co-authored-by: Scott Werner <stwerner@vt.edu>
This commit is contained in:
Ryan Neal
GitHub
parent
dbf4829b47
commit
d5b2220ed3
37 changed files with 4457 additions and 140 deletions
661
Cargo.lock
generated
661
Cargo.lock
generated
File diff suppressed because it is too large
Load diff
16
Cargo.toml
16
Cargo.toml
|
|
@ -33,6 +33,22 @@ fs2 = "0.4"
|
|||
base64 = "0.22"
|
||||
bytes = "1"
|
||||
tokio-util = "0.7"
|
||||
# AWS building blocks for the native Bedrock adapter. Lean stack: request
|
||||
# signing + credential chain + event-stream decode only. Transport for the
|
||||
# actual Bedrock inference calls stays on fabro-http; the full
|
||||
# aws-sdk-bedrockruntime (and its parallel hyper stack) is not pulled in.
|
||||
# aws-config keeps its DEFAULT features on purpose: `rt-tokio` supplies the
|
||||
# TokioSleep impl the credential chain's retry requires (without it,
|
||||
# resolving the default chain panics with "an async sleep implementation is
|
||||
# required"), and `sso`/`credentials-process` make from_default_chain's
|
||||
# documented SSO/credential-process support real. `rustls` pins the TLS
|
||||
# backend for credential-resolution HTTP.
|
||||
aws-config = { version = "1", features = ["behavior-version-latest", "rustls"] }
|
||||
aws-credential-types = { version = "1", features = ["hardcoded-credentials"] }
|
||||
aws-sigv4 = "1"
|
||||
aws-smithy-eventstream = "0.60"
|
||||
aws-smithy-runtime-api = "1"
|
||||
aws-smithy-types = "1"
|
||||
clap = { version = "4", features = ["derive", "env"] }
|
||||
clap_complete = "4"
|
||||
jsonschema = { version = "0.42", default-features = false }
|
||||
|
|
|
|||
|
|
@ -141,6 +141,16 @@ Fabro ships an [OpenRouter](/integrations/openrouter) provider definition with a
|
|||
enabled = true
|
||||
```
|
||||
|
||||
### Amazon Bedrock
|
||||
|
||||
Fabro ships an [Amazon Bedrock](/integrations/bedrock) provider definition with a curated multi-vendor catalog over Bedrock's Converse API, disabled by default. Enable it and authenticate with a Bedrock API key or AWS SigV4 credentials:
|
||||
|
||||
```toml title="settings.toml"
|
||||
[llm.providers.bedrock]
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
```
|
||||
|
||||
### Ollama
|
||||
|
||||
Fabro ships an Ollama provider definition that is disabled by default. Enable it in settings when you want Fabro to route through a local Ollama server:
|
||||
|
|
|
|||
|
|
@ -95,6 +95,7 @@
|
|||
"integrations/github",
|
||||
"integrations/daytona",
|
||||
"integrations/litellm",
|
||||
"integrations/bedrock",
|
||||
"integrations/openrouter",
|
||||
"integrations/slack",
|
||||
"integrations/brave-search"
|
||||
|
|
|
|||
162
docs/public/integrations/bedrock.mdx
Normal file
162
docs/public/integrations/bedrock.mdx
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
---
|
||||
title: "Amazon Bedrock"
|
||||
description: "Route Fabro models through Amazon Bedrock with SigV4 or API-key auth"
|
||||
---
|
||||
|
||||
[Amazon Bedrock](https://aws.amazon.com/bedrock/) hosts Anthropic, Amazon, Meta, Mistral, DeepSeek, Qwen, Moonshot, Z.AI, MiniMax, NVIDIA, and OpenAI open-weight models behind one AWS endpoint. Fabro ships a disabled `bedrock` provider entry with a curated model catalog over Bedrock's unified Converse API, so you can opt in from `settings.toml` without changing Fabro code.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An AWS account with [Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) granted for the models you want (see [Model access and approvals](#model-access-and-approvals))
|
||||
- Either a [Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) or working AWS credentials (environment keys, profile, IMDS, IRSA, SSO)
|
||||
|
||||
## Model access and approvals
|
||||
|
||||
Access is granted per Region and varies by model family — enabling the provider in Fabro is necessary but not sufficient.
|
||||
|
||||
- **IAM.** Converse and ConverseStream have no dedicated IAM actions; they're authorized by `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream`. A Bedrock API key additionally needs `bedrock:CallWithBearerToken`.
|
||||
- **Anthropic (Claude) models** require a one-time use-case submission in the Bedrock console (**Model access**) before first use, and the grant is **per Region** — approval in `us-east-1` does not cover `us-east-2`. An un-approved Region returns `AccessDeniedException`.
|
||||
- **Third-party models** (OpenAI gpt-oss, DeepSeek, Qwen, Moonshot, Z.AI, MiniMax, NVIDIA) auto-enable on first call, which needs `aws-marketplace:Subscribe` and `aws-marketplace:ViewSubscriptions` on the calling principal. The first call may take a moment while the subscription activates.
|
||||
- **Claude Fable 5 / Mythos-class** models additionally require opting into data sharing via the [Data Retention API](https://docs.aws.amazon.com/bedrock/latest/userguide/data-retention.html) (`provider_data_share`, 30-day retention) **before** they can be invoked. With the account/project on the `default` retention mode, Converse rejects the request with *"data retention mode 'default' is not available for this model."*
|
||||
|
||||
## Enable the provider
|
||||
|
||||
Add the provider override to `~/.fabro/settings.toml`:
|
||||
|
||||
```toml title="settings.toml"
|
||||
_version = 1
|
||||
|
||||
[llm.providers.bedrock]
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
```
|
||||
|
||||
The SigV4 signing region is derived from `base_url` — change it to your Region's endpoint (`https://bedrock-runtime.<region>.amazonaws.com`, FIPS and China endpoints included).
|
||||
|
||||
## Configure credentials
|
||||
|
||||
Two auth modes, tried in order:
|
||||
|
||||
**Bedrock API key** (simplest): store the key and Fabro sends it as a bearer token. The key is read from either `AWS_BEARER_TOKEN_BEDROCK` (AWS's canonical name, also honored by the AWS SDKs and CLI) or `BEDROCK_API_KEY` (Fabro's `<PROVIDER>_API_KEY` convention) — use whichever you prefer.
|
||||
|
||||
```bash
|
||||
fabro secret set AWS_BEARER_TOKEN_BEDROCK bedrock-api-key-...
|
||||
# or, equivalently
|
||||
fabro secret set BEDROCK_API_KEY bedrock-api-key-...
|
||||
# or for standalone local runs
|
||||
export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
|
||||
```
|
||||
|
||||
**AWS SigV4** (IAM-scoped): with no API key configured, Fabro signs each request using the AWS default credential chain — environment keys, shared profile, EC2/ECS instance roles, IRSA/web identity, SSO. Expiring session credentials refresh automatically. The catalog declares this as the `aws_sigv4` credential source:
|
||||
|
||||
```toml
|
||||
[llm.providers.bedrock.auth]
|
||||
credentials = ["env:AWS_BEARER_TOKEN_BEDROCK", "env:BEDROCK_API_KEY", "vault:AWS_BEARER_TOKEN_BEDROCK", "vault:BEDROCK_API_KEY", "aws_sigv4"]
|
||||
```
|
||||
|
||||
The key resolves from the process environment first (either name), then the server vault (`fabro secret set`), then falls back to SigV4 — so on a server, prefer `secret set`. To select a non-default AWS profile for SigV4, set `AWS_PROFILE` (it, and the rest of the AWS credential-chain variables, are passed through to workflow workers).
|
||||
|
||||
<Warning>
|
||||
**Bearer-vs-SigV4 precedence.** Because the bearer key is tried before SigV4, setting `AWS_BEARER_TOKEN_BEDROCK` makes the `bedrock` (Converse) provider authenticate with that key too — not just the `bedrock-openai` mantle provider below. If your key is valid only for mantle (it lacks `bedrock:InvokeModel*` on the runtime), every Converse model then fails with *"Authentication failed."* To run Converse models on SigV4 while using a mantle-only bearer key for GPT-5.x, pin the Converse provider to SigV4 explicitly:
|
||||
|
||||
```toml
|
||||
[llm.providers.bedrock.auth]
|
||||
credentials = ["aws_sigv4"]
|
||||
```
|
||||
</Warning>
|
||||
|
||||
## Included models
|
||||
|
||||
The built-in catalog curates Converse-capable models, using cross-region inference profile ids (`us.`/`global.` prefixes) where on-demand access requires them:
|
||||
|
||||
| Fabro model ID | Notes |
|
||||
| --- | --- |
|
||||
| `us.anthropic.claude-sonnet-4-6` | Provider default; Anthropic cache billing |
|
||||
| `us.anthropic.claude-opus-4-8` | Anthropic cache billing |
|
||||
| `us.anthropic.claude-haiku-4-5` | Provider small default |
|
||||
| `us.anthropic.claude-fable-5` | Frontier; sampling params pinned by Bedrock (Fabro drops `temperature`/`top_p` automatically); requires the account-level `provider_data_share` data-sharing opt-in (see [Model access](#model-access-and-approvals)) |
|
||||
| `openai.gpt-oss-120b`, `openai.gpt-oss-20b` | OpenAI open-weights |
|
||||
| `amazon.nova-2-lite` | Vision |
|
||||
| `meta.llama4-maverick` | Vision |
|
||||
| `mistral.mistral-large-3`, `mistral.devstral-2` | |
|
||||
| `deepseek.v3-2` | |
|
||||
| `moonshotai.kimi-k2.5`, `zai.glm-5` | |
|
||||
| `minimax.minimax-m2.5`, `nvidia.nemotron-3-super` | |
|
||||
|
||||
Any other Converse-capable Bedrock model can be added as a settings model entry with `provider = "bedrock"` and the Bedrock model or inference-profile id as `api_id`.
|
||||
|
||||
Not included on this provider: Claude Mythos 5 (Anthropic-Messages-only on `bedrock-mantle`, limited preview). OpenAI's frontier models live on the companion `bedrock-openai` provider below.
|
||||
|
||||
## OpenAI frontier models (GPT-5.5 / GPT-5.4)
|
||||
|
||||
GPT-5.5 and GPT-5.4 on Bedrock are served only by the `bedrock-mantle` endpoint's OpenAI Responses API — a different surface than Converse. Fabro ships a companion `bedrock-openai` provider for them: the same AWS account and `AWS_BEARER_TOKEN_BEDROCK` key, pointed at the mantle endpoint over the OpenAI dialect.
|
||||
|
||||
```toml title="settings.toml"
|
||||
[llm.providers.bedrock-openai]
|
||||
enabled = true
|
||||
# regional: change to https://bedrock-mantle.<region>.api.aws/openai/v1
|
||||
```
|
||||
|
||||
```bash
|
||||
fabro model test --model openai.gpt-5.5
|
||||
```
|
||||
|
||||
Auth on this provider is Bedrock-API-key only (mantle SigV4 uses a different signing name than the runtime endpoint). Fabro always sends `store: false`, so nothing is retained under mantle's default 30-day response storage.
|
||||
|
||||
## Use Bedrock models
|
||||
|
||||
```bash
|
||||
fabro model list --provider bedrock
|
||||
fabro model test --model us.anthropic.claude-sonnet-4-6
|
||||
fabro run workflow.fabro --model deepseek.v3-2
|
||||
```
|
||||
|
||||
## Prompt caching
|
||||
|
||||
Claude models cache automatically when the catalog row declares `prompt_cache`: Fabro places Converse `cachePoint` blocks after the system prompt, the tool definitions, and the conversation prefix — the same placement as the direct Anthropic provider. Cache reads and writes price Anthropic-style via the per-model `billing_policy`.
|
||||
|
||||
## Converse extensions
|
||||
|
||||
Bedrock-specific request fields pass through verbatim via `provider_options.bedrock` on API/SDK requests — the keys merge into the top level of the Converse envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "us.anthropic.claude-sonnet-4-6",
|
||||
"provider_options": {
|
||||
"bedrock": {
|
||||
"additionalModelRequestFields": { "top_k": 200 },
|
||||
"guardrailConfig": { "guardrailIdentifier": "gr-abc", "guardrailVersion": "1" },
|
||||
"serviceTier": { "type": "flex" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"no AWS credentials provider found"** — Neither an API key nor any AWS chain source resolved. Set `AWS_BEARER_TOKEN_BEDROCK` (or `BEDROCK_API_KEY`), or configure standard AWS credentials.
|
||||
|
||||
**`AccessDeniedException` / 403** — The IAM principal lacks `bedrock:InvokeModel*` for the model, or model access has not been granted in the Bedrock console for your Region (Claude needs the per-Region use-case approval; third-party models need `aws-marketplace:Subscribe`).
|
||||
|
||||
**"Authentication failed: Please make sure your API Key is valid."** — A Bedrock API key was sent but rejected by the runtime. Common cause: a mantle-scoped key used against Converse — see the bearer-vs-SigV4 [warning above](#configure-credentials). Verify the key is valid for `bedrock-runtime` in this Region, or pin Converse to `aws_sigv4`.
|
||||
|
||||
**"data retention mode 'default' is not available for this model"** — Fable 5 / Mythos-class models require opting into data sharing first; see [Model access and approvals](#model-access-and-approvals).
|
||||
|
||||
**"The provided model identifier is invalid"** — The wire id sent to Bedrock isn't a recognized model or inference-profile id. Set an explicit `api_id` (from `aws bedrock list-inference-profiles`) on the model entry.
|
||||
|
||||
**`ValidationException` mentioning on-demand throughput** — The model requires an inference-profile id; use the `us.`/`global.`-prefixed id from the catalog rather than the bare model id.
|
||||
|
||||
**`ValidationException` mentioning maximum tokens** — The requested `max_tokens` exceeds the model's per-request output cap; lower the model's `max_output` to the documented limit.
|
||||
|
||||
**`ThrottlingException`** — Account-level Bedrock quota; consider cross-region inference profiles or a quota increase.
|
||||
|
||||
## Further reading
|
||||
|
||||
<Columns cols={2}>
|
||||
<Card title="Models" icon="microchip" href="/core-concepts/models">
|
||||
How Fabro routes model IDs, providers, and fallbacks.
|
||||
</Card>
|
||||
<Card title="Settings Configuration" icon="gear" href="/reference/user-configuration">
|
||||
Full reference for `[llm.providers.<id>]` and `[llm.models.<id>]`.
|
||||
</Card>
|
||||
</Columns>
|
||||
|
|
@ -198,7 +198,7 @@ x-team-secret = { vault = "gateway_team_secret" }
|
|||
| `billing_policy` | `"openai"` \| `"anthropic"` \| `"gemini"` \| `"none"` | derived from `adapter` | Provider-owned billing algorithm for usage estimates. Override for exceptional providers such as local no-billing runtimes. |
|
||||
| `base_url` | string | built-in value or adapter runtime default | Provider API base URL. Required for most custom OpenAI-compatible providers. |
|
||||
| `auth` | table | omitted | API-key auth config. Omit the table entirely for providers that need no API key; any `extra_headers` are still attached. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>` and `env:<NAME>`. Literal secret strings are rejected. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>`, `env:<NAME>`, and `aws_sigv4` (sign requests from the AWS default credential chain — Bedrock). Literal secret strings are rejected. |
|
||||
| `auth.header` | `"bearer"` or `{ custom = "Header-Name" }` | `"bearer"` | Primary API-key header policy. Omit when the provider uses a standard bearer token. |
|
||||
| `extra_headers` | table | `{}` | Additional headers attached to provider requests. Values must be typed refs: `{ literal = "..." }`, `{ env = "NAME" }`, or `{ vault = "NAME" }`. |
|
||||
| `priority` | integer | `0` | Higher-priority configured providers win default selection; ties use canonical provider ID. |
|
||||
|
|
|
|||
|
|
@ -1016,8 +1016,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1026,7 +1026,7 @@ mod tests {
|
|||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let args = AgentArgs {
|
||||
prompt: "test".to_string(),
|
||||
provider: Some("bedrock".to_string()),
|
||||
provider: Some("acme-aws".to_string()),
|
||||
model: None,
|
||||
permissions: None,
|
||||
auto_approve: false,
|
||||
|
|
@ -1037,7 +1037,7 @@ mod tests {
|
|||
};
|
||||
|
||||
let provider_id = parse_provider(&args).unwrap();
|
||||
assert_eq!(provider_id, ProviderId::new("bedrock"));
|
||||
assert_eq!(provider_id, ProviderId::new("acme-aws"));
|
||||
assert_eq!(
|
||||
profile_kind_for_provider(&catalog, &provider_id, None).unwrap(),
|
||||
AgentProfileKind::OpenAi
|
||||
|
|
@ -1049,8 +1049,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1058,9 +1058,9 @@ mod tests {
|
|||
});
|
||||
settings
|
||||
.models
|
||||
.insert("bedrock-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("bedrock".to_string()),
|
||||
display_name: Some("Bedrock Claude".to_string()),
|
||||
.insert("acme-aws-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("acme-aws".to_string()),
|
||||
display_name: Some("Acme AWS Claude".to_string()),
|
||||
family: Some("claude".to_string()),
|
||||
default: Some(true),
|
||||
limits: Some(SettingsModelLimits {
|
||||
|
|
@ -1081,7 +1081,7 @@ mod tests {
|
|||
let args = AgentArgs {
|
||||
prompt: "test".to_string(),
|
||||
provider: None,
|
||||
model: Some("bedrock-claude".to_string()),
|
||||
model: Some("acme-aws-claude".to_string()),
|
||||
permissions: None,
|
||||
auto_approve: false,
|
||||
debug: false,
|
||||
|
|
@ -1092,7 +1092,7 @@ mod tests {
|
|||
|
||||
assert_eq!(
|
||||
resolve_provider_id(&catalog, &args).unwrap(),
|
||||
ProviderId::new("bedrock")
|
||||
ProviderId::new("acme-aws")
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -1101,8 +1101,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1124,7 +1124,7 @@ mod tests {
|
|||
|
||||
assert_eq!(
|
||||
resolve_provider_id(&catalog, &args).unwrap(),
|
||||
ProviderId::new("bedrock")
|
||||
ProviderId::new("acme-aws")
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -1133,8 +1133,8 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
|
|
@ -1142,9 +1142,9 @@ mod tests {
|
|||
});
|
||||
settings
|
||||
.models
|
||||
.insert("bedrock-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("bedrock".to_string()),
|
||||
display_name: Some("Bedrock Claude".to_string()),
|
||||
.insert("acme-aws-claude".to_string(), ModelCatalogSettings {
|
||||
provider: Some("acme-aws".to_string()),
|
||||
display_name: Some("Acme AWS Claude".to_string()),
|
||||
family: Some("claude".to_string()),
|
||||
default: Some(true),
|
||||
agent_profile: Some(AgentProfileKind::Anthropic),
|
||||
|
|
@ -1167,8 +1167,8 @@ mod tests {
|
|||
assert_eq!(
|
||||
profile_kind_for_provider(
|
||||
&catalog,
|
||||
&ProviderId::new("bedrock"),
|
||||
Some("bedrock-claude")
|
||||
&ProviderId::new("acme-aws"),
|
||||
Some("acme-aws-claude")
|
||||
)
|
||||
.unwrap(),
|
||||
AgentProfileKind::Anthropic
|
||||
|
|
@ -1180,20 +1180,20 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::OpenAi),
|
||||
..ProviderCatalogSettings::default()
|
||||
});
|
||||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let provider_id = ProviderId::new("bedrock");
|
||||
let provider_id = ProviderId::new("acme-aws");
|
||||
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "bedrock-claude-sonnet-4-6");
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "acme-aws-claude-sonnet-4-6");
|
||||
|
||||
assert_eq!(model_id.provider(), &provider_id);
|
||||
assert_eq!(model_id.model_id(), "bedrock-claude-sonnet-4-6");
|
||||
assert_eq!(model_id.model_id(), "acme-aws-claude-sonnet-4-6");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -1201,20 +1201,20 @@ mod tests {
|
|||
let mut settings = LlmCatalogSettings::default();
|
||||
settings
|
||||
.providers
|
||||
.insert("bedrock".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Bedrock".to_string()),
|
||||
.insert("acme-aws".to_string(), ProviderCatalogSettings {
|
||||
display_name: Some("Acme AWS".to_string()),
|
||||
adapter: Some("openai_compatible".to_string()),
|
||||
base_url: Some("https://example.invalid/v1".to_string()),
|
||||
agent_profile: Some(AgentProfileKind::Anthropic),
|
||||
..ProviderCatalogSettings::default()
|
||||
});
|
||||
let catalog = Catalog::from_builtin_with_overrides(&settings).unwrap();
|
||||
let provider_id = ProviderId::new("bedrock");
|
||||
let provider_id = ProviderId::new("acme-aws");
|
||||
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "bedrock-claude-sonnet-4-6");
|
||||
let model_id = summarizer_model_id(&provider_id, &catalog, "acme-aws-claude-sonnet-4-6");
|
||||
|
||||
assert_eq!(model_id.provider(), &provider_id);
|
||||
assert_eq!(model_id.model_id(), "bedrock-claude-sonnet-4-6");
|
||||
assert_eq!(model_id.model_id(), "acme-aws-claude-sonnet-4-6");
|
||||
}
|
||||
|
||||
// subagent tool registration tests
|
||||
|
|
|
|||
|
|
@ -45,7 +45,13 @@ pub struct OAuthConfig {
|
|||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub enum ApiKeyHeader {
|
||||
Bearer(String),
|
||||
Custom { name: String, value: String },
|
||||
Custom {
|
||||
name: String,
|
||||
value: String,
|
||||
},
|
||||
/// No static header: the request is authenticated by AWS SigV4 signing,
|
||||
/// with credentials resolved from the AWS default chain at request time.
|
||||
AwsSigv4,
|
||||
}
|
||||
|
||||
fn redact_for_debug(value: &str) -> String {
|
||||
|
|
@ -69,6 +75,7 @@ impl std::fmt::Debug for ApiKeyHeader {
|
|||
.field("name", name)
|
||||
.field("value", &redact_for_debug(value))
|
||||
.finish(),
|
||||
Self::AwsSigv4 => f.write_str("AwsSigv4"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,6 +28,9 @@ pub(crate) enum ResolvedSecret {
|
|||
credential: Box<OAuthCredential>,
|
||||
vault_name: String,
|
||||
},
|
||||
/// Opaque AWS SigV4 source: no static secret; the adapter signs requests
|
||||
/// using the AWS default credential chain.
|
||||
AwsSigv4,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
|
|
@ -333,6 +336,9 @@ impl CredentialResolver {
|
|||
Err(err) => Err(vault_lookup_error(provider, name, err)),
|
||||
},
|
||||
CredentialRef::Env(name) => Ok((self.env_lookup)(name).map(ResolvedSecret::ApiKey)),
|
||||
// AWS SigV4 is an opaque source: it always "resolves" (the adapter
|
||||
// signs at request time from the AWS chain), no vault/env lookup.
|
||||
CredentialRef::AwsSigv4 => Ok(Some(ResolvedSecret::AwsSigv4)),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -379,23 +385,39 @@ impl CredentialResolver {
|
|||
) -> Result<ApiCredential, ResolveError> {
|
||||
let base_url = Self::provider_base_url_for_catalog(provider_id, catalog);
|
||||
match secret {
|
||||
// Opaque AWS SigV4 source: carry the marker so the adapter signs
|
||||
// with the AWS chain; no static secret resolved here.
|
||||
ResolvedSecret::AwsSigv4 => Ok(ApiCredential {
|
||||
provider: provider_id.clone(),
|
||||
auth_header: Some(ApiKeyHeader::AwsSigv4),
|
||||
extra_headers: self.resolved_extra_headers_for_catalog(
|
||||
vault,
|
||||
provider_id,
|
||||
catalog,
|
||||
)?,
|
||||
base_url,
|
||||
codex_mode: false,
|
||||
org_id: None,
|
||||
project_id: None,
|
||||
}),
|
||||
ResolvedSecret::ApiKey(key) => {
|
||||
let provider = catalog
|
||||
.provider(provider_id)
|
||||
.ok_or_else(|| ResolveError::NotConfigured(provider_id.clone()))?;
|
||||
let auth_header = auth_header_for_catalog_provider(provider, key.clone())?;
|
||||
let mut cred = ApiCredential {
|
||||
provider: provider_id.clone(),
|
||||
auth_header: Some(auth_header),
|
||||
extra_headers: HashMap::new(),
|
||||
base_url: None,
|
||||
codex_mode: false,
|
||||
org_id: None,
|
||||
project_id: None,
|
||||
provider: provider_id.clone(),
|
||||
auth_header: Some(auth_header),
|
||||
extra_headers: self.resolved_extra_headers_for_catalog(
|
||||
vault,
|
||||
provider_id,
|
||||
catalog,
|
||||
)?,
|
||||
base_url,
|
||||
codex_mode: false,
|
||||
org_id: None,
|
||||
project_id: None,
|
||||
};
|
||||
cred.base_url = base_url;
|
||||
cred.extra_headers =
|
||||
self.resolved_extra_headers_for_catalog(vault, provider_id, catalog)?;
|
||||
if provider_id == &ProviderId::openai() {
|
||||
apply_openai_api_env_context(&mut cred, &*self.env_lookup);
|
||||
}
|
||||
|
|
@ -579,6 +601,38 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sigv4_provider_resolves_to_aws_sigv4_credential() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let vault = Vault::load(dir.path().join("secrets.json")).unwrap();
|
||||
// No env credentials configured: SigV4 must still resolve.
|
||||
let resolver = test_resolver(vault, Arc::new(|_| None));
|
||||
let catalog = catalog_with(
|
||||
r#"
|
||||
[providers.bedrock]
|
||||
adapter = "bedrock"
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.eu-west-1.amazonaws.com"
|
||||
|
||||
[providers.bedrock.auth]
|
||||
credentials = ["aws_sigv4"]
|
||||
"#,
|
||||
);
|
||||
|
||||
let resolved = resolver
|
||||
.resolve(
|
||||
ProviderId::from("bedrock"),
|
||||
CredentialUsage::ApiRequest,
|
||||
&catalog,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let ResolvedCredential::Api(api) = resolved;
|
||||
assert_eq!(api.provider, ProviderId::from("bedrock"));
|
||||
assert_eq!(api.auth_header, Some(ApiKeyHeader::AwsSigv4));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolve_returns_not_configured_for_missing_provider() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ impl ApiKeyStrategy {
|
|||
.iter()
|
||||
.filter_map(|credential_ref| match credential_ref {
|
||||
CredentialRef::Env(name) => Some(name.clone()),
|
||||
CredentialRef::Vault(_) => None,
|
||||
CredentialRef::Vault(_) | CredentialRef::AwsSigv4 => None,
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ fn provider_env_var_label(provider: &ProviderId, catalog: &Catalog) -> String {
|
|||
.iter()
|
||||
.filter_map(|credential| match credential {
|
||||
CredentialRef::Env(name) => Some(name.as_str()),
|
||||
CredentialRef::Vault(_) => None,
|
||||
CredentialRef::Vault(_) | CredentialRef::AwsSigv4 => None,
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(" / ")
|
||||
|
|
|
|||
|
|
@ -333,7 +333,7 @@ fn exec_accepts_configured_custom_provider_from_settings() {
|
|||
let context = test_context!();
|
||||
context.write_home(
|
||||
".fabro/settings.toml",
|
||||
"_version = 1\n\n[llm.providers.bedrock]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.bedrock.auth]\ncredentials = [\"env:BEDROCK_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"bedrock\"\nname = \"bedrock-claude-sonnet-4-6\"\n",
|
||||
"_version = 1\n\n[llm.providers.acme-aws]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.acme-aws.auth]\ncredentials = [\"env:ACME_AWS_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"acme-aws\"\nname = \"acme-claude-sonnet-4-6\"\n",
|
||||
);
|
||||
|
||||
let mut cmd = context.exec_cmd();
|
||||
|
|
@ -350,7 +350,7 @@ fn exec_accepts_configured_custom_provider_from_settings() {
|
|||
exit_code: 1
|
||||
----- stdout -----
|
||||
----- stderr -----
|
||||
× LLM credentials not configured for provider 'bedrock'
|
||||
× LLM credentials not configured for provider 'acme-aws'
|
||||
");
|
||||
}
|
||||
|
||||
|
|
@ -398,7 +398,7 @@ fn exec_server_target_accepts_configured_custom_provider_from_settings() {
|
|||
let context = test_context!();
|
||||
context.write_home(
|
||||
".fabro/settings.toml",
|
||||
"_version = 1\n\n[llm.providers.bedrock]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.bedrock.auth]\ncredentials = [\"env:BEDROCK_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"bedrock\"\nname = \"bedrock-claude-sonnet-4-6\"\n",
|
||||
"_version = 1\n\n[llm.providers.acme-aws]\nadapter = \"openai_compatible\"\nagent_profile = \"openai\"\nbase_url = \"https://bedrock.example.invalid/v1\"\n\n[llm.providers.acme-aws.auth]\ncredentials = [\"env:ACME_AWS_API_KEY\"]\n\n[cli.exec.model]\nprovider = \"acme-aws\"\nname = \"acme-claude-sonnet-4-6\"\n",
|
||||
);
|
||||
let server = MockServer::start();
|
||||
server.mock(|when, then| {
|
||||
|
|
@ -425,7 +425,7 @@ fn exec_server_target_accepts_configured_custom_provider_from_settings() {
|
|||
"expected remote server failure marker, got: {stderr}"
|
||||
);
|
||||
assert!(
|
||||
!stderr.contains("unknown provider: bedrock"),
|
||||
!stderr.contains("unknown provider: acme-aws"),
|
||||
"exec should resolve custom providers from settings for remote transport: {stderr}"
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -244,7 +244,7 @@ x-team-secret = { vault = "gateway_team_secret" }
|
|||
| `billing_policy` | `"openai"` \| `"anthropic"` \| `"gemini"` \| `"none"` | derived from `adapter` | Provider-owned billing algorithm for usage estimates. Override for exceptional providers such as local no-billing runtimes. |
|
||||
| `base_url` | string | built-in value or adapter runtime default | Provider API base URL. Required for most custom OpenAI-compatible providers. |
|
||||
| `auth` | table | omitted | API-key auth config. Omit the table entirely for providers that need no API key; any `extra_headers` are still attached. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>` and `env:<NAME>`. Literal secret strings are rejected. |
|
||||
| `auth.credentials` | array<string> | required when `auth` present | Ordered credential refs. Accepted forms are `vault:<NAME>`, `env:<NAME>`, and `aws_sigv4` (sign requests from the AWS default credential chain — Bedrock). Literal secret strings are rejected. |
|
||||
| `auth.header` | `"bearer"` or `{ custom = "Header-Name" }` | `"bearer"` | Primary API-key header policy. Omit when the provider uses a standard bearer token. |
|
||||
| `extra_headers` | table | `{}` | Additional headers attached to provider requests. Values must be typed refs: `{ literal = "..." }`, `{ env = "NAME" }`, or `{ vault = "NAME" }`. |
|
||||
| `priority` | integer | `0` | Higher-priority configured providers win default selection; ties use canonical provider ID. |
|
||||
|
|
|
|||
|
|
@ -32,6 +32,12 @@ base64.workspace = true
|
|||
bytes.workspace = true
|
||||
tokio-util.workspace = true
|
||||
tracing.workspace = true
|
||||
aws-config.workspace = true
|
||||
aws-credential-types.workspace = true
|
||||
aws-sigv4.workspace = true
|
||||
aws-smithy-eventstream.workspace = true
|
||||
aws-smithy-runtime-api.workspace = true
|
||||
aws-smithy-types.workspace = true
|
||||
fabro-http.workspace = true
|
||||
fabro-auth = { path = "../fabro-auth" }
|
||||
fabro-model = { path = "../fabro-model" }
|
||||
|
|
|
|||
|
|
@ -90,7 +90,9 @@ fn apply_primary_auth_header(
|
|||
extra_headers.insert(name, value);
|
||||
None
|
||||
}
|
||||
None => None,
|
||||
// SigV4 is not a static header; only the Bedrock adapter consumes
|
||||
// the marker (it signs at request time).
|
||||
Some(ApiKeyHeader::AwsSigv4) | None => None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -213,6 +215,7 @@ pub fn factory_for(adapter_kind: AdapterKind) -> AdapterFactory {
|
|||
AdapterKind::OpenAi => build_openai,
|
||||
AdapterKind::Gemini => build_gemini,
|
||||
AdapterKind::OpenAiCompatible => build_openai_compatible,
|
||||
AdapterKind::Bedrock => providers::bedrock::build,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
|
||||
use super::SYNTHETIC_TOOL_NAME;
|
||||
use super::decode::{convert_synthetic_tool_to_text, map_finish_reason, refusal_error};
|
||||
use crate::codec::{RawEvent, StreamDecoder};
|
||||
use crate::codec::{RawEvent, StreamDecoder, parse_tool_arguments_or_empty};
|
||||
use crate::error::{Error, ProviderErrorDetail, ProviderErrorKind};
|
||||
use crate::types::{
|
||||
ContentPart, FinishReason, Message, RateLimitInfo, Response, Role, StreamEvent, ThinkingData,
|
||||
|
|
@ -253,8 +253,7 @@ impl SseAccumulator {
|
|||
}
|
||||
Some(ContentBlockKind::ToolUse { id, name }) => {
|
||||
let raw_args = std::mem::take(&mut self.current_tool_args);
|
||||
let arguments =
|
||||
serde_json::from_str(&raw_args).unwrap_or_else(|_| serde_json::json!({}));
|
||||
let arguments = parse_tool_arguments_or_empty(&raw_args);
|
||||
let mut tool_call = ToolCall::new(id, name, arguments);
|
||||
tool_call.raw_arguments = Some(raw_args);
|
||||
self.content_parts
|
||||
|
|
|
|||
296
lib/crates/fabro-llm/src/codec/bedrock_converse/decode.rs
Normal file
296
lib/crates/fabro-llm/src/codec/bedrock_converse/decode.rs
Normal file
|
|
@ -0,0 +1,296 @@
|
|||
//! Response decoding: Converse body → canonical `Response`.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::codec::CodecCtx;
|
||||
use crate::error::{Error, error_from_status_code};
|
||||
use crate::types::{
|
||||
ContentPart, FinishReason, Message, RateLimitInfo, Response, Role, ThinkingData, TokenCounts,
|
||||
ToolCall,
|
||||
};
|
||||
|
||||
/// Map a non-2xx Bedrock runtime response to an `Error`, pulling the human
|
||||
/// reason out of AWS's error envelope. Bedrock uses several shapes for the
|
||||
/// same field — top-level `message` (SigV4 path) and `Message` (API-key
|
||||
/// path), occasionally nested `error.message` — and tags the type in
|
||||
/// `__type`. The generic codec parser only reads `error.message`, so without
|
||||
/// this these surface as "Unknown error".
|
||||
pub(super) fn bedrock_error(
|
||||
status: u16,
|
||||
body: &str,
|
||||
provider: &str,
|
||||
retry_after: Option<f64>,
|
||||
) -> Error {
|
||||
let raw: Option<Value> = serde_json::from_str(body).ok();
|
||||
let message = raw
|
||||
.as_ref()
|
||||
.and_then(extract_error_message)
|
||||
.unwrap_or_else(|| {
|
||||
if body.trim().is_empty() {
|
||||
"Unknown error".to_string()
|
||||
} else {
|
||||
body.to_string()
|
||||
}
|
||||
});
|
||||
// `__type` is often an ARN-ish `prefix#ThrottlingException`; keep the tail.
|
||||
let code = raw
|
||||
.as_ref()
|
||||
.and_then(|v| {
|
||||
v.get("__type")
|
||||
.or_else(|| v.get("code"))
|
||||
.and_then(Value::as_str)
|
||||
})
|
||||
.map(|t| t.rsplit('#').next().unwrap_or(t).to_string());
|
||||
error_from_status_code(
|
||||
status,
|
||||
message,
|
||||
provider.to_string(),
|
||||
code,
|
||||
raw,
|
||||
retry_after,
|
||||
)
|
||||
}
|
||||
|
||||
fn extract_error_message(v: &Value) -> Option<String> {
|
||||
v.get("message")
|
||||
.and_then(Value::as_str)
|
||||
.or_else(|| v.get("Message").and_then(Value::as_str))
|
||||
.or_else(|| {
|
||||
v.get("error")
|
||||
.and_then(|e| e.get("message"))
|
||||
.and_then(Value::as_str)
|
||||
})
|
||||
.map(String::from)
|
||||
}
|
||||
|
||||
pub(super) fn decode_response(
|
||||
body: &str,
|
||||
ctx: &CodecCtx<'_>,
|
||||
rate_limit: Option<RateLimitInfo>,
|
||||
) -> Result<Response, Error> {
|
||||
let raw: Value = serde_json::from_str(body)
|
||||
.map_err(|e| Error::network(format!("failed to parse converse response: {e}"), e))?;
|
||||
|
||||
let content_parts = raw
|
||||
.pointer("/output/message/content")
|
||||
.and_then(Value::as_array)
|
||||
.map(|blocks| blocks.iter().filter_map(decode_content_block).collect())
|
||||
.unwrap_or_default();
|
||||
|
||||
let finish_reason = map_stop_reason(raw.get("stopReason").and_then(Value::as_str));
|
||||
let usage = token_counts_from_usage(raw.get("usage"));
|
||||
|
||||
Ok(Response {
|
||||
// Converse responses carry no id; synthesize one like the gemini
|
||||
// codec does so downstream consumers always see a non-empty id.
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
model: ctx.request.model.clone(),
|
||||
provider: ctx.provider_name.to_string(),
|
||||
message: Message {
|
||||
role: Role::Assistant,
|
||||
content: content_parts,
|
||||
name: None,
|
||||
tool_call_id: None,
|
||||
},
|
||||
finish_reason,
|
||||
usage,
|
||||
raw: Some(raw),
|
||||
warnings: vec![],
|
||||
rate_limit,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Decode one Converse content block into a canonical part. Unknown block
|
||||
/// kinds are skipped (the union grows: `citationsContent`, `searchResult`,
|
||||
/// `video`, ...).
|
||||
pub(super) fn decode_content_block(block: &Value) -> Option<ContentPart> {
|
||||
if let Some(text) = block.get("text").and_then(Value::as_str) {
|
||||
if text.is_empty() {
|
||||
return None;
|
||||
}
|
||||
return Some(ContentPart::text(text));
|
||||
}
|
||||
if let Some(tool_use) = block.get("toolUse") {
|
||||
let id = tool_use.get("toolUseId").and_then(Value::as_str)?;
|
||||
let name = tool_use.get("name").and_then(Value::as_str)?;
|
||||
// A no-argument tool call is canonically `{}`, not null (so it
|
||||
// re-encodes to a valid Converse `toolUse.input` object).
|
||||
let input = match tool_use.get("input") {
|
||||
Some(Value::Null) | None => Value::Object(serde_json::Map::new()),
|
||||
Some(value) => value.clone(),
|
||||
};
|
||||
return Some(ContentPart::ToolCall(ToolCall::new(id, name, input)));
|
||||
}
|
||||
if let Some(reasoning) = block.get("reasoningContent") {
|
||||
if let Some(text_block) = reasoning.get("reasoningText") {
|
||||
return Some(ContentPart::Thinking(ThinkingData {
|
||||
text: text_block
|
||||
.get("text")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.to_string(),
|
||||
signature: text_block
|
||||
.get("signature")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string),
|
||||
redacted: false,
|
||||
}));
|
||||
}
|
||||
if let Some(redacted) = reasoning.get("redactedContent").and_then(Value::as_str) {
|
||||
return Some(ContentPart::Thinking(ThinkingData {
|
||||
text: redacted.to_string(),
|
||||
signature: None,
|
||||
redacted: true,
|
||||
}));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Map a Converse `stopReason` onto the canonical finish vocabulary.
|
||||
pub(super) fn map_stop_reason(reason: Option<&str>) -> FinishReason {
|
||||
match reason {
|
||||
None | Some("end_turn" | "stop_sequence") => FinishReason::Stop,
|
||||
Some("max_tokens" | "model_context_window_exceeded") => FinishReason::Length,
|
||||
Some("tool_use") => FinishReason::ToolCalls,
|
||||
// `refusal` is the Claude 5 blocking-classifier stop, passed through
|
||||
// by Bedrock for Fable-class models.
|
||||
Some("guardrail_intervened" | "content_filtered" | "refusal") => {
|
||||
FinishReason::ContentFilter
|
||||
}
|
||||
Some(other) => FinishReason::Other(other.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Converse usage maps directly onto the disjoint buckets: `inputTokens`
|
||||
/// already excludes cached tokens (documented), so no subtraction applies.
|
||||
pub(super) fn token_counts_from_usage(usage: Option<&Value>) -> TokenCounts {
|
||||
let Some(usage) = usage else {
|
||||
return TokenCounts::default();
|
||||
};
|
||||
let count = |key: &str| usage.get(key).and_then(Value::as_i64).unwrap_or(0);
|
||||
TokenCounts {
|
||||
input_tokens: count("inputTokens"),
|
||||
output_tokens: count("outputTokens"),
|
||||
reasoning_tokens: 0,
|
||||
cache_read_tokens: count("cacheReadInputTokens"),
|
||||
cache_write_tokens: count("cacheWriteInputTokens"),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn stop_reasons_map_to_canonical_vocabulary() {
|
||||
assert_eq!(map_stop_reason(Some("end_turn")), FinishReason::Stop);
|
||||
assert_eq!(map_stop_reason(Some("stop_sequence")), FinishReason::Stop);
|
||||
assert_eq!(map_stop_reason(Some("max_tokens")), FinishReason::Length);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("model_context_window_exceeded")),
|
||||
FinishReason::Length
|
||||
);
|
||||
assert_eq!(map_stop_reason(Some("tool_use")), FinishReason::ToolCalls);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("guardrail_intervened")),
|
||||
FinishReason::ContentFilter
|
||||
);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("content_filtered")),
|
||||
FinishReason::ContentFilter
|
||||
);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("refusal")),
|
||||
FinishReason::ContentFilter
|
||||
);
|
||||
assert_eq!(
|
||||
map_stop_reason(Some("malformed_tool_use")),
|
||||
FinishReason::Other("malformed_tool_use".to_string())
|
||||
);
|
||||
assert_eq!(map_stop_reason(None), FinishReason::Stop);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_maps_without_subtraction() {
|
||||
let usage = serde_json::json!({
|
||||
"inputTokens": 30,
|
||||
"outputTokens": 628,
|
||||
"totalTokens": 658,
|
||||
"cacheReadInputTokens": 1024,
|
||||
"cacheWriteInputTokens": 512,
|
||||
});
|
||||
let counts = token_counts_from_usage(Some(&usage));
|
||||
assert_eq!(counts.input_tokens, 30);
|
||||
assert_eq!(counts.output_tokens, 628);
|
||||
assert_eq!(counts.cache_read_tokens, 1024);
|
||||
assert_eq!(counts.cache_write_tokens, 512);
|
||||
assert_eq!(counts.reasoning_tokens, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bedrock_error_extracts_aws_message_shapes() {
|
||||
// SigV4 path: top-level lowercase `message`.
|
||||
let sigv4 = bedrock_error(
|
||||
403,
|
||||
r#"{"message":"Model access is denied due to IAM ..."}"#,
|
||||
"bedrock",
|
||||
None,
|
||||
);
|
||||
assert!(
|
||||
sigv4.to_string().contains("Model access is denied"),
|
||||
"{sigv4}"
|
||||
);
|
||||
|
||||
// API-key path: top-level capitalized `Message`.
|
||||
let api_key = bedrock_error(
|
||||
403,
|
||||
r#"{"Message":"Authentication failed: Please make sure your API Key is valid."}"#,
|
||||
"bedrock",
|
||||
None,
|
||||
);
|
||||
assert!(
|
||||
api_key.to_string().contains("Authentication failed"),
|
||||
"{api_key}"
|
||||
);
|
||||
|
||||
// `__type` becomes the error code (tail after `#`).
|
||||
let typed = bedrock_error(
|
||||
429,
|
||||
r#"{"__type":"com.amazon.coral.service#ThrottlingException","message":"slow down"}"#,
|
||||
"bedrock",
|
||||
None,
|
||||
);
|
||||
let Error::Provider { detail, .. } = &typed else {
|
||||
panic!("expected provider error: {typed}");
|
||||
};
|
||||
assert_eq!(detail.error_code.as_deref(), Some("ThrottlingException"));
|
||||
|
||||
// Garbage body falls back rather than panicking.
|
||||
let opaque = bedrock_error(500, "not json", "bedrock", None);
|
||||
assert!(opaque.to_string().contains("not json"), "{opaque}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_content_blocks_are_skipped() {
|
||||
assert!(decode_content_block(&serde_json::json!({"citationsContent": {}})).is_none());
|
||||
assert!(decode_content_block(&serde_json::json!({"text": ""})).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reasoning_text_block_round_trips_signature() {
|
||||
let block = serde_json::json!({
|
||||
"reasoningContent": {
|
||||
"reasoningText": { "text": "thinking...", "signature": "sig-1" }
|
||||
}
|
||||
});
|
||||
let Some(ContentPart::Thinking(thinking)) = decode_content_block(&block) else {
|
||||
panic!("expected thinking part");
|
||||
};
|
||||
assert_eq!(thinking.text, "thinking...");
|
||||
assert_eq!(thinking.signature.as_deref(), Some("sig-1"));
|
||||
assert!(!thinking.redacted);
|
||||
}
|
||||
}
|
||||
643
lib/crates/fabro-llm/src/codec/bedrock_converse/encode.rs
Normal file
643
lib/crates/fabro-llm/src/codec/bedrock_converse/encode.rs
Normal file
|
|
@ -0,0 +1,643 @@
|
|||
//! Request encoding: canonical `Request` → Converse envelope.
|
||||
|
||||
use base64::Engine;
|
||||
use base64::engine::general_purpose::STANDARD as BASE64;
|
||||
use serde_json::{Map, Value, json};
|
||||
|
||||
use crate::codec::{CodecCtx, EncodedRequest, extract_system_prompt, merge_named_provider_options};
|
||||
use crate::error::Error;
|
||||
use crate::types::{ContentPart, Message, Request, Role, ToolChoice};
|
||||
|
||||
pub(super) fn encode(ctx: &CodecCtx<'_>, stream: bool) -> Result<EncodedRequest, Error> {
|
||||
let request = ctx.request;
|
||||
if request.response_format.is_some() {
|
||||
return Err(Error::Configuration {
|
||||
message: format!(
|
||||
"provider '{}' does not support response_format yet (Bedrock Converse \
|
||||
structured output is a named follow-up)",
|
||||
ctx.provider_name
|
||||
),
|
||||
source: None,
|
||||
});
|
||||
}
|
||||
|
||||
let caching = supports_prompt_cache(ctx);
|
||||
let (system, conversation) = extract_system_prompt(&request.messages);
|
||||
|
||||
let mut body = Map::new();
|
||||
|
||||
if let Some(system) = system {
|
||||
let mut blocks = vec![json!({ "text": system })];
|
||||
if caching {
|
||||
blocks.push(cache_point());
|
||||
}
|
||||
body.insert("system".to_string(), Value::Array(blocks));
|
||||
}
|
||||
|
||||
let mut messages = Vec::new();
|
||||
for message in conversation {
|
||||
if let Some(value) = encode_message(message) {
|
||||
messages.push(value);
|
||||
}
|
||||
}
|
||||
if caching {
|
||||
apply_cache_point_to_conversation_prefix(&mut messages);
|
||||
}
|
||||
body.insert("messages".to_string(), Value::Array(messages));
|
||||
|
||||
// Models with `sampling_params = false` reject classic sampling knobs
|
||||
// (Claude Fable 5 pins temperature on Bedrock too).
|
||||
let (temperature, top_p) = if ctx
|
||||
.model
|
||||
.is_none_or(fabro_model::Model::supports_sampling_params)
|
||||
{
|
||||
(request.temperature, request.top_p)
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
|
||||
let mut inference = Map::new();
|
||||
if let Some(max_tokens) = request.max_tokens {
|
||||
inference.insert("maxTokens".to_string(), json!(max_tokens));
|
||||
}
|
||||
if let Some(temperature) = temperature {
|
||||
inference.insert("temperature".to_string(), json!(temperature));
|
||||
}
|
||||
if let Some(top_p) = top_p {
|
||||
inference.insert("topP".to_string(), json!(top_p));
|
||||
}
|
||||
if let Some(stop) = &request.stop_sequences {
|
||||
if !stop.is_empty() {
|
||||
inference.insert("stopSequences".to_string(), json!(stop));
|
||||
}
|
||||
}
|
||||
if !inference.is_empty() {
|
||||
body.insert("inferenceConfig".to_string(), Value::Object(inference));
|
||||
}
|
||||
|
||||
if let Some(tool_config) = encode_tool_config(request, caching) {
|
||||
body.insert("toolConfig".to_string(), tool_config);
|
||||
}
|
||||
|
||||
let mut body = Value::Object(body);
|
||||
merge_provider_options(
|
||||
&mut body,
|
||||
request.provider_options.as_ref(),
|
||||
ctx.provider_name,
|
||||
);
|
||||
|
||||
let action = if stream {
|
||||
"converse-stream"
|
||||
} else {
|
||||
"converse"
|
||||
};
|
||||
Ok(EncodedRequest {
|
||||
body,
|
||||
endpoint: format!("/model/{}/{action}", ctx.deployment_id),
|
||||
headers: Vec::new(),
|
||||
})
|
||||
}
|
||||
|
||||
fn supports_prompt_cache(ctx: &CodecCtx<'_>) -> bool {
|
||||
ctx.model.is_some_and(|m| m.features.prompt_cache)
|
||||
}
|
||||
|
||||
fn cache_point() -> Value {
|
||||
json!({ "cachePoint": { "type": "default" } })
|
||||
}
|
||||
|
||||
/// Encode one conversation message. Tool-role messages carry their results in
|
||||
/// user-role messages (Converse has no tool role). Returns `None` when no
|
||||
/// block survives translation.
|
||||
fn encode_message(message: &Message) -> Option<Value> {
|
||||
let role = match message.role {
|
||||
Role::Assistant => "assistant",
|
||||
// Tool results ride in user messages on the Converse wire.
|
||||
_ => "user",
|
||||
};
|
||||
|
||||
let mut blocks: Vec<Value> = message
|
||||
.content
|
||||
.iter()
|
||||
.filter_map(encode_content_part)
|
||||
.collect();
|
||||
|
||||
// Tool-role messages whose result lives on the message rather than in a
|
||||
// ToolResult part.
|
||||
if blocks.is_empty() && message.role == Role::Tool {
|
||||
if let Some(tool_call_id) = &message.tool_call_id {
|
||||
let text = message.text();
|
||||
blocks.push(json!({
|
||||
"toolResult": {
|
||||
"toolUseId": tool_call_id,
|
||||
"content": [{ "text": text }],
|
||||
}
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
if blocks.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(json!({ "role": role, "content": blocks }))
|
||||
}
|
||||
|
||||
fn encode_content_part(part: &ContentPart) -> Option<Value> {
|
||||
match part {
|
||||
ContentPart::Text(text) => {
|
||||
if text.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(json!({ "text": text }))
|
||||
}
|
||||
}
|
||||
// Converse has no URL sources; the adapter's attachment resolution
|
||||
// inlines file-backed parts ahead of encoding, and URL-only parts are
|
||||
// dropped (the established drop-don't-fail attachment contract).
|
||||
ContentPart::Image(image) => {
|
||||
let bytes = image.data.as_ref()?;
|
||||
Some(json!({
|
||||
"image": {
|
||||
"format": media_format(image.media_type.as_deref(), "png"),
|
||||
"source": { "bytes": BASE64.encode(bytes) },
|
||||
}
|
||||
}))
|
||||
}
|
||||
ContentPart::Document(document) => {
|
||||
let bytes = document.data.as_ref()?;
|
||||
Some(json!({
|
||||
"document": {
|
||||
"format": media_format(document.media_type.as_deref(), "pdf"),
|
||||
"name": document.file_name.as_deref().unwrap_or("document"),
|
||||
"source": { "bytes": BASE64.encode(bytes) },
|
||||
}
|
||||
}))
|
||||
}
|
||||
ContentPart::ToolCall(tool_call) => {
|
||||
// Converse requires `toolUse.input` to be a JSON object document.
|
||||
// A no-argument tool call carries `Null` (the stream decoder gets
|
||||
// no input fragments to parse), which Bedrock rejects as
|
||||
// "toolUse.input is empty". Coerce any non-object to `{}` so the
|
||||
// wire is always valid, regardless of where the call originated.
|
||||
let input = match &tool_call.arguments {
|
||||
Value::Object(_) => tool_call.arguments.clone(),
|
||||
_ => json!({}),
|
||||
};
|
||||
Some(json!({
|
||||
"toolUse": {
|
||||
"toolUseId": tool_call.id,
|
||||
"name": tool_call.name,
|
||||
"input": input,
|
||||
}
|
||||
}))
|
||||
}
|
||||
ContentPart::ToolResult(result) => {
|
||||
let content = match &result.content {
|
||||
Value::String(text) => json!([{ "text": text }]),
|
||||
other => json!([{ "json": other }]),
|
||||
};
|
||||
let mut block = Map::new();
|
||||
block.insert("toolUseId".to_string(), json!(result.tool_call_id));
|
||||
block.insert("content".to_string(), content);
|
||||
if result.is_error {
|
||||
block.insert("status".to_string(), json!("error"));
|
||||
}
|
||||
Some(json!({ "toolResult": Value::Object(block) }))
|
||||
}
|
||||
ContentPart::Thinking(thinking) => {
|
||||
if thinking.redacted {
|
||||
Some(json!({
|
||||
"reasoningContent": { "redactedContent": thinking.text }
|
||||
}))
|
||||
} else {
|
||||
let mut text_block = Map::new();
|
||||
text_block.insert("text".to_string(), json!(thinking.text));
|
||||
if let Some(signature) = &thinking.signature {
|
||||
// Echoed back unmodified — Bedrock validates it.
|
||||
text_block.insert("signature".to_string(), json!(signature));
|
||||
}
|
||||
Some(json!({
|
||||
"reasoningContent": { "reasoningText": Value::Object(text_block) }
|
||||
}))
|
||||
}
|
||||
}
|
||||
// Audio input and opaque foreign parts have no Converse encoding.
|
||||
ContentPart::Audio(_) | ContentPart::Other { .. } => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert common MIME types into Bedrock's media `format` enum values.
|
||||
fn media_format<'a>(media_type: Option<&str>, default: &'a str) -> &'a str {
|
||||
match media_type {
|
||||
Some("image/png") => "png",
|
||||
Some("image/jpeg" | "image/jpg") => "jpeg",
|
||||
Some("image/gif") => "gif",
|
||||
Some("image/webp") => "webp",
|
||||
Some("application/pdf") => "pdf",
|
||||
Some("text/plain") => "txt",
|
||||
Some("text/markdown") => "md",
|
||||
Some("text/html") => "html",
|
||||
Some("text/csv") => "csv",
|
||||
Some(
|
||||
"application/msword"
|
||||
| "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
|
||||
) => "docx",
|
||||
Some(
|
||||
"application/vnd.ms-excel"
|
||||
| "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
) => "xlsx",
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn encode_tool_config(request: &Request, caching: bool) -> Option<Value> {
|
||||
let tools = request.tools.as_ref()?;
|
||||
if tools.is_empty() {
|
||||
return None;
|
||||
}
|
||||
// `tool_choice: none` is rejected at the adapter's validate_request;
|
||||
// defensively drop the toolConfig if it slips through.
|
||||
if request.tool_choice == Some(ToolChoice::None) {
|
||||
return None;
|
||||
}
|
||||
|
||||
let mut entries: Vec<Value> = tools
|
||||
.iter()
|
||||
.map(|tool| {
|
||||
json!({
|
||||
"toolSpec": {
|
||||
"name": tool.name,
|
||||
"description": tool.description,
|
||||
"inputSchema": { "json": tool_input_schema(&tool.parameters) },
|
||||
}
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
if caching {
|
||||
entries.push(cache_point());
|
||||
}
|
||||
|
||||
let mut config = Map::new();
|
||||
config.insert("tools".to_string(), Value::Array(entries));
|
||||
match &request.tool_choice {
|
||||
Some(ToolChoice::Required) => {
|
||||
config.insert("toolChoice".to_string(), json!({ "any": {} }));
|
||||
}
|
||||
Some(ToolChoice::Named { tool_name }) => {
|
||||
config.insert(
|
||||
"toolChoice".to_string(),
|
||||
json!({ "tool": { "name": tool_name } }),
|
||||
);
|
||||
}
|
||||
// Auto is the wire default; ToolChoice::None dropped the config above.
|
||||
Some(ToolChoice::Auto | ToolChoice::None) | None => {}
|
||||
}
|
||||
Some(Value::Object(config))
|
||||
}
|
||||
|
||||
/// Normalize a tool's JSON-Schema for Bedrock's `toolSpec.inputSchema.json`.
|
||||
/// Converse strictly validates the schema and requires a top-level `type`;
|
||||
/// some model families (e.g. DeepSeek) reject a typeless schema that Claude
|
||||
/// tolerates. Tools may arrive with a loose schema (no top-level `type`, or a
|
||||
/// bare `{}` for a no-argument tool), so default the type to `object`.
|
||||
fn tool_input_schema(parameters: &Value) -> Value {
|
||||
match parameters {
|
||||
Value::Object(map) => {
|
||||
let mut map = map.clone();
|
||||
map.entry("type").or_insert_with(|| json!("object"));
|
||||
Value::Object(map)
|
||||
}
|
||||
// A non-object schema is not a valid tool input schema; substitute the
|
||||
// empty-object schema Bedrock accepts.
|
||||
_ => json!({ "type": "object", "properties": {} }),
|
||||
}
|
||||
}
|
||||
|
||||
/// Mirror the anthropic codec's conversation-prefix cache placement: a
|
||||
/// `cachePoint` at the end of the second-to-last user message, so the prior
|
||||
/// turns stay cached while the newest turn streams.
|
||||
fn apply_cache_point_to_conversation_prefix(messages: &mut [Value]) {
|
||||
let mut previous_user = None;
|
||||
let mut last_user = None;
|
||||
for (index, message) in messages.iter().enumerate() {
|
||||
if message.get("role").and_then(Value::as_str) == Some("user") {
|
||||
previous_user = last_user;
|
||||
last_user = Some(index);
|
||||
}
|
||||
}
|
||||
|
||||
let Some(target) = previous_user else {
|
||||
return;
|
||||
};
|
||||
if let Some(content) = messages[target]
|
||||
.get_mut("content")
|
||||
.and_then(Value::as_array_mut)
|
||||
{
|
||||
content.push(cache_point());
|
||||
}
|
||||
}
|
||||
|
||||
/// Merge `provider_options.<provider_name>` keys into the top level of the
|
||||
/// body (the same adapter-name-keyed contract as the openai_compatible
|
||||
/// codec). This is the passthrough for `additionalModelRequestFields`,
|
||||
/// `guardrailConfig`, `serviceTier`, and other Converse extensions.
|
||||
fn merge_provider_options(body: &mut Value, provider_options: Option<&Value>, provider_name: &str) {
|
||||
merge_named_provider_options(body, provider_options, provider_name);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use fabro_model::Catalog;
|
||||
use fabro_model::catalog::LlmCatalogSettings;
|
||||
use serde_json::json;
|
||||
|
||||
use super::*;
|
||||
use crate::codec::CodecParams;
|
||||
use crate::types::{
|
||||
ResponseFormat, ResponseFormatType, ThinkingData, ToolCall, ToolDefinition, ToolResult,
|
||||
};
|
||||
|
||||
fn base_request(model: &str) -> Request {
|
||||
Request {
|
||||
model: model.to_string(),
|
||||
messages: vec![Message::user("Hello")],
|
||||
provider: Some("bedrock".to_string()),
|
||||
tools: None,
|
||||
tool_choice: None,
|
||||
response_format: None,
|
||||
temperature: Some(0.5),
|
||||
top_p: None,
|
||||
max_tokens: Some(256),
|
||||
stop_sequences: None,
|
||||
reasoning_effort: None,
|
||||
speed: None,
|
||||
metadata: None,
|
||||
provider_options: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn encode_with(request: &Request) -> EncodedRequest {
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "us.anthropic.claude-sonnet-4-6",
|
||||
model: None,
|
||||
params: ¶ms,
|
||||
};
|
||||
encode(&ctx, false).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn endpoint_carries_model_and_action() {
|
||||
let request = base_request("claude");
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request: &request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "us.anthropic.claude-sonnet-4-6",
|
||||
model: None,
|
||||
params: ¶ms,
|
||||
};
|
||||
assert_eq!(
|
||||
encode(&ctx, false).unwrap().endpoint,
|
||||
"/model/us.anthropic.claude-sonnet-4-6/converse"
|
||||
);
|
||||
assert_eq!(
|
||||
encode(&ctx, true).unwrap().endpoint,
|
||||
"/model/us.anthropic.claude-sonnet-4-6/converse-stream"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn system_messages_become_top_level_system_blocks() {
|
||||
let mut request = base_request("claude");
|
||||
request.messages = vec![Message::system("Be brief"), Message::user("Hi")];
|
||||
let encoded = encode_with(&request);
|
||||
assert_eq!(encoded.body["system"][0]["text"], "Be brief");
|
||||
assert_eq!(encoded.body["messages"][0]["role"], "user");
|
||||
assert_eq!(encoded.body["messages"][0]["content"][0]["text"], "Hi");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inference_config_uses_camel_case() {
|
||||
let encoded = encode_with(&base_request("claude"));
|
||||
assert_eq!(encoded.body["inferenceConfig"]["maxTokens"], 256);
|
||||
assert_eq!(encoded.body["inferenceConfig"]["temperature"], 0.5);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tools_encode_as_tool_specs_with_choice() {
|
||||
let mut request = base_request("claude");
|
||||
request.tools = Some(vec![ToolDefinition::function(
|
||||
"search",
|
||||
"Search things",
|
||||
json!({"type": "object"}),
|
||||
)]);
|
||||
request.tool_choice = Some(ToolChoice::named("search"));
|
||||
let encoded = encode_with(&request);
|
||||
let spec = &encoded.body["toolConfig"]["tools"][0]["toolSpec"];
|
||||
assert_eq!(spec["name"], "search");
|
||||
assert_eq!(spec["inputSchema"]["json"]["type"], "object");
|
||||
assert_eq!(
|
||||
encoded.body["toolConfig"]["toolChoice"]["tool"]["name"],
|
||||
"search"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn typeless_tool_schema_gains_object_type() {
|
||||
// Bedrock rejects a tool inputSchema without a top-level `type` (some
|
||||
// model families validate strictly); the encoder must default it.
|
||||
let mut request = base_request("claude");
|
||||
request.tools = Some(vec![
|
||||
ToolDefinition::function("no_type", "schema without a type", json!({})),
|
||||
ToolDefinition::function(
|
||||
"props_only",
|
||||
"properties but no top-level type",
|
||||
json!({"properties": {"q": {"type": "string"}}}),
|
||||
),
|
||||
]);
|
||||
let encoded = encode_with(&request);
|
||||
let tools = &encoded.body["toolConfig"]["tools"];
|
||||
assert_eq!(
|
||||
tools[0]["toolSpec"]["inputSchema"]["json"]["type"],
|
||||
"object"
|
||||
);
|
||||
assert_eq!(
|
||||
tools[1]["toolSpec"]["inputSchema"]["json"]["type"],
|
||||
"object"
|
||||
);
|
||||
// An existing nested schema is preserved, not clobbered.
|
||||
assert_eq!(
|
||||
tools[1]["toolSpec"]["inputSchema"]["json"]["properties"]["q"]["type"],
|
||||
"string"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_results_ride_in_user_messages() {
|
||||
let mut request = base_request("claude");
|
||||
request.messages = vec![Message {
|
||||
role: Role::Tool,
|
||||
content: vec![ContentPart::ToolResult(ToolResult {
|
||||
tool_call_id: "tool-1".to_string(),
|
||||
content: json!("42"),
|
||||
is_error: false,
|
||||
image_data: None,
|
||||
image_media_type: None,
|
||||
})],
|
||||
name: None,
|
||||
tool_call_id: Some("tool-1".to_string()),
|
||||
}];
|
||||
let encoded = encode_with(&request);
|
||||
let message = &encoded.body["messages"][0];
|
||||
assert_eq!(message["role"], "user");
|
||||
assert_eq!(message["content"][0]["toolResult"]["toolUseId"], "tool-1");
|
||||
assert_eq!(
|
||||
message["content"][0]["toolResult"]["content"][0]["text"],
|
||||
"42"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_argument_tool_call_encodes_empty_object_input() {
|
||||
// A no-arg tool call decodes to `Null` arguments; Bedrock rejects a
|
||||
// null/empty `toolUse.input`, so the encoder must emit `{}`.
|
||||
let mut request = base_request("claude");
|
||||
request.messages = vec![Message {
|
||||
role: Role::Assistant,
|
||||
content: vec![ContentPart::ToolCall(ToolCall::new(
|
||||
"tool-1",
|
||||
"TaskList",
|
||||
Value::Null,
|
||||
))],
|
||||
name: None,
|
||||
tool_call_id: None,
|
||||
}];
|
||||
let encoded = encode_with(&request);
|
||||
let tool_use = &encoded.body["messages"][0]["content"][0]["toolUse"];
|
||||
assert_eq!(tool_use["toolUseId"], "tool-1");
|
||||
assert_eq!(tool_use["name"], "TaskList");
|
||||
assert_eq!(tool_use["input"], json!({}));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn thinking_parts_restructure_into_reasoning_text_blocks() {
|
||||
let mut request = base_request("claude");
|
||||
request.messages = vec![Message {
|
||||
role: Role::Assistant,
|
||||
content: vec![ContentPart::Thinking(ThinkingData {
|
||||
text: "prior thoughts".to_string(),
|
||||
signature: Some("sig-1".to_string()),
|
||||
redacted: false,
|
||||
})],
|
||||
name: None,
|
||||
tool_call_id: None,
|
||||
}];
|
||||
let encoded = encode_with(&request);
|
||||
let block = &encoded.body["messages"][0]["content"][0]["reasoningContent"]["reasoningText"];
|
||||
assert_eq!(block["text"], "prior thoughts");
|
||||
assert_eq!(block["signature"], "sig-1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn media_format_maps_common_mime_types_to_bedrock_formats() {
|
||||
assert_eq!(media_format(Some("image/jpeg"), "png"), "jpeg");
|
||||
assert_eq!(media_format(Some("text/plain"), "pdf"), "txt");
|
||||
assert_eq!(media_format(Some("text/markdown"), "pdf"), "md");
|
||||
assert_eq!(media_format(Some("application/octet-stream"), "pdf"), "pdf");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_options_merge_top_level() {
|
||||
let mut request = base_request("claude");
|
||||
request.provider_options = Some(json!({
|
||||
"bedrock": {
|
||||
"additionalModelRequestFields": {"top_k": 200},
|
||||
"serviceTier": {"type": "flex"}
|
||||
}
|
||||
}));
|
||||
let encoded = encode_with(&request);
|
||||
assert_eq!(encoded.body["additionalModelRequestFields"]["top_k"], 200);
|
||||
assert_eq!(encoded.body["serviceTier"]["type"], "flex");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn response_format_is_rejected() {
|
||||
let mut request = base_request("claude");
|
||||
request.response_format = Some(ResponseFormat {
|
||||
kind: ResponseFormatType::JsonSchema,
|
||||
json_schema: Some(json!({"type": "object"})),
|
||||
strict: false,
|
||||
});
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request: &request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "m",
|
||||
model: None,
|
||||
params: ¶ms,
|
||||
};
|
||||
assert!(encode(&ctx, false).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sampling_params_false_drops_temperature_and_top_p() {
|
||||
let settings: LlmCatalogSettings = toml::from_str(
|
||||
r#"
|
||||
[providers.bedrock]
|
||||
adapter = "bedrock"
|
||||
enabled = true
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
|
||||
[models."pinned-model"]
|
||||
provider = "bedrock"
|
||||
display_name = "Pinned"
|
||||
family = "claude-5"
|
||||
default = true
|
||||
|
||||
[models."pinned-model".limits]
|
||||
context_window = 100000
|
||||
|
||||
[models."pinned-model".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
sampling_params = false
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let catalog = Catalog::from_settings(&settings).unwrap();
|
||||
|
||||
let mut request = base_request("pinned-model");
|
||||
request.top_p = Some(0.9);
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request: &request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "pinned-model",
|
||||
model: catalog.get("pinned-model"),
|
||||
params: ¶ms,
|
||||
};
|
||||
let encoded = encode(&ctx, false).unwrap();
|
||||
|
||||
let inference = &encoded.body["inferenceConfig"];
|
||||
assert!(inference.get("temperature").is_none());
|
||||
assert!(inference.get("topP").is_none());
|
||||
assert_eq!(inference["maxTokens"], 256);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_points_follow_the_anthropic_placement() {
|
||||
let mut messages = vec![
|
||||
json!({"role": "user", "content": [{"text": "turn 1"}]}),
|
||||
json!({"role": "assistant", "content": [{"text": "reply 1"}]}),
|
||||
json!({"role": "user", "content": [{"text": "turn 2"}]}),
|
||||
];
|
||||
apply_cache_point_to_conversation_prefix(&mut messages);
|
||||
// Second-to-last user message gains the cachePoint.
|
||||
assert!(messages[0]["content"][1].get("cachePoint").is_some());
|
||||
assert_eq!(messages[2]["content"].as_array().unwrap().len(), 1);
|
||||
}
|
||||
}
|
||||
58
lib/crates/fabro-llm/src/codec/bedrock_converse/mod.rs
Normal file
58
lib/crates/fabro-llm/src/codec/bedrock_converse/mod.rs
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
//! The Amazon Bedrock Converse codec.
|
||||
//!
|
||||
//! Pure translation: no HTTP, auth, signing, or event-stream framing — the
|
||||
//! Bedrock adapter shell owns those. Converse is Bedrock's model-agnostic
|
||||
//! envelope (AWS translates it to each hosted family's native dialect
|
||||
//! server-side), which is what makes this one codec serve Claude, Nova,
|
||||
//! Llama, Mistral, DeepSeek, Qwen, Kimi, GLM, MiniMax, Nemotron, and
|
||||
//! gpt-oss alike. The codec fully forms its endpoints (model-in-path,
|
||||
//! `/converse` vs `/converse-stream`), mirrors the anthropic codec's prompt
|
||||
//! cache placement with `cachePoint` blocks, and round-trips
|
||||
//! `reasoningContent` thinking signatures unmodified.
|
||||
|
||||
mod decode;
|
||||
mod encode;
|
||||
mod stream;
|
||||
|
||||
use crate::codec::{Codec, CodecCtx, EncodedRequest, StreamDecoder};
|
||||
use crate::error::Error;
|
||||
use crate::types::{RateLimitInfo, Response};
|
||||
|
||||
/// Codec for the Bedrock Converse wire dialect.
|
||||
pub(crate) struct BedrockConverse;
|
||||
|
||||
impl Codec for BedrockConverse {
|
||||
fn encode(&self, ctx: &CodecCtx<'_>, stream: bool) -> Result<EncodedRequest, Error> {
|
||||
encode::encode(ctx, stream)
|
||||
}
|
||||
|
||||
fn decode_response(
|
||||
&self,
|
||||
body: &str,
|
||||
ctx: &CodecCtx<'_>,
|
||||
rate_limit: Option<RateLimitInfo>,
|
||||
) -> Result<Response, Error> {
|
||||
decode::decode_response(body, ctx, rate_limit)
|
||||
}
|
||||
|
||||
fn stream_decoder(
|
||||
&self,
|
||||
ctx: &CodecCtx<'_>,
|
||||
rate_limit: Option<RateLimitInfo>,
|
||||
) -> Box<dyn StreamDecoder> {
|
||||
Box::new(stream::ConverseStreamDecoder::new(ctx, rate_limit))
|
||||
}
|
||||
|
||||
/// Bedrock error bodies are AWS-shaped (top-level `message`/`Message`,
|
||||
/// `__type`), which the default parser misses — extract them so failures
|
||||
/// surface the real reason instead of "Unknown error".
|
||||
fn decode_error(
|
||||
&self,
|
||||
status: u16,
|
||||
body: &str,
|
||||
ctx: &CodecCtx<'_>,
|
||||
retry_after: Option<f64>,
|
||||
) -> Error {
|
||||
decode::bedrock_error(status, body, ctx.provider_name, retry_after)
|
||||
}
|
||||
}
|
||||
512
lib/crates/fabro-llm/src/codec/bedrock_converse/stream.rs
Normal file
512
lib/crates/fabro-llm/src/codec/bedrock_converse/stream.rs
Normal file
|
|
@ -0,0 +1,512 @@
|
|||
//! Streaming decoder: ConverseStream events → canonical `StreamEvent`s.
|
||||
//!
|
||||
//! Event names arrive in the transport's `RawEvent::event` (the frame's
|
||||
//! `:event-type` header); payloads are the event JSON. The documented
|
||||
//! sequence is `messageStart` → per content block (`contentBlockStart`
|
||||
//! [tool use only] → `contentBlockDelta`* → `contentBlockStop`) →
|
||||
//! `messageStop{stopReason}` → `metadata{usage}`. Usage arrives ONLY in the
|
||||
//! terminal `metadata` event, which is also where the final `Finish` is
|
||||
//! synthesized.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use super::decode::{map_stop_reason, token_counts_from_usage};
|
||||
use crate::codec::{CodecCtx, RawEvent, StreamDecoder, parse_tool_arguments_or_empty};
|
||||
use crate::error::Error;
|
||||
use crate::types::{
|
||||
ContentPart, FinishReason, Message, RateLimitInfo, Response, Role, StreamEvent, ThinkingData,
|
||||
TokenCounts, ToolCall,
|
||||
};
|
||||
|
||||
/// Per-content-block accumulation state, keyed by `contentBlockIndex`.
|
||||
enum BlockState {
|
||||
Text(String),
|
||||
Reasoning {
|
||||
text: String,
|
||||
signature: Option<String>,
|
||||
redacted: Option<String>,
|
||||
},
|
||||
ToolUse {
|
||||
id: String,
|
||||
name: String,
|
||||
input: String,
|
||||
},
|
||||
}
|
||||
|
||||
/// Accumulated state while decoding one ConverseStream response.
|
||||
pub(super) struct ConverseStreamDecoder {
|
||||
provider_name: String,
|
||||
model: String,
|
||||
blocks: BTreeMap<u64, BlockState>,
|
||||
/// Completed blocks in arrival order, for the final response message.
|
||||
parts: Vec<ContentPart>,
|
||||
finish_reason: FinishReason,
|
||||
usage: TokenCounts,
|
||||
text_started: bool,
|
||||
finished: bool,
|
||||
rate_limit: Option<RateLimitInfo>,
|
||||
}
|
||||
|
||||
impl ConverseStreamDecoder {
|
||||
pub(super) fn new(ctx: &CodecCtx<'_>, rate_limit: Option<RateLimitInfo>) -> Self {
|
||||
Self {
|
||||
provider_name: ctx.provider_name.to_string(),
|
||||
model: ctx.request.model.clone(),
|
||||
blocks: BTreeMap::new(),
|
||||
parts: Vec::new(),
|
||||
finish_reason: FinishReason::Stop,
|
||||
usage: TokenCounts::default(),
|
||||
text_started: false,
|
||||
finished: false,
|
||||
rate_limit,
|
||||
}
|
||||
}
|
||||
|
||||
fn block_index(payload: &Value) -> u64 {
|
||||
payload
|
||||
.get("contentBlockIndex")
|
||||
.and_then(Value::as_u64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
fn on_block_start(&mut self, payload: &Value) -> Vec<StreamEvent> {
|
||||
let index = Self::block_index(payload);
|
||||
if let Some(tool_use) = payload.pointer("/start/toolUse") {
|
||||
let id = tool_use
|
||||
.get("toolUseId")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.to_string();
|
||||
let name = tool_use
|
||||
.get("name")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.to_string();
|
||||
let started = ToolCall::new(&id, &name, Value::Null);
|
||||
self.blocks.insert(index, BlockState::ToolUse {
|
||||
id,
|
||||
name,
|
||||
input: String::new(),
|
||||
});
|
||||
return vec![StreamEvent::ToolCallStart { tool_call: started }];
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
|
||||
fn on_block_delta(&mut self, payload: &Value) -> Vec<StreamEvent> {
|
||||
let index = Self::block_index(payload);
|
||||
let Some(delta) = payload.get("delta") else {
|
||||
return Vec::new();
|
||||
};
|
||||
|
||||
if let Some(text) = delta.get("text").and_then(Value::as_str) {
|
||||
if text.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
let mut events = Vec::new();
|
||||
if !self.text_started {
|
||||
self.text_started = true;
|
||||
events.push(StreamEvent::TextStart { text_id: None });
|
||||
}
|
||||
match self
|
||||
.blocks
|
||||
.entry(index)
|
||||
.or_insert_with(|| BlockState::Text(String::new()))
|
||||
{
|
||||
BlockState::Text(buffer) => buffer.push_str(text),
|
||||
// A text delta against a non-text block: tolerate by ignoring
|
||||
// the mismatch rather than corrupting tool/reasoning state.
|
||||
_ => return events,
|
||||
}
|
||||
events.push(StreamEvent::text_delta(text, None));
|
||||
return events;
|
||||
}
|
||||
|
||||
if let Some(input) = delta.pointer("/toolUse/input").and_then(Value::as_str) {
|
||||
if let Some(BlockState::ToolUse {
|
||||
id,
|
||||
name,
|
||||
input: buffer,
|
||||
}) = self.blocks.get_mut(&index)
|
||||
{
|
||||
buffer.push_str(input);
|
||||
let partial = ToolCall::new(id.as_str(), name.as_str(), Value::Null);
|
||||
return vec![StreamEvent::ToolCallDelta { tool_call: partial }];
|
||||
}
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
if let Some(reasoning) = delta.get("reasoningContent") {
|
||||
let entry = self
|
||||
.blocks
|
||||
.entry(index)
|
||||
.or_insert_with(|| BlockState::Reasoning {
|
||||
text: String::new(),
|
||||
signature: None,
|
||||
redacted: None,
|
||||
});
|
||||
let BlockState::Reasoning {
|
||||
text,
|
||||
signature,
|
||||
redacted,
|
||||
} = entry
|
||||
else {
|
||||
return Vec::new();
|
||||
};
|
||||
let mut events = Vec::new();
|
||||
if text.is_empty() && signature.is_none() && redacted.is_none() {
|
||||
events.push(StreamEvent::ReasoningStart);
|
||||
}
|
||||
// Streaming reasoning deltas carry text/signature as FLAT union
|
||||
// members (unlike the nested request-side reasoningText block).
|
||||
if let Some(fragment) = reasoning.get("text").and_then(Value::as_str) {
|
||||
text.push_str(fragment);
|
||||
events.push(StreamEvent::ReasoningDelta {
|
||||
delta: fragment.to_string(),
|
||||
});
|
||||
}
|
||||
if let Some(sig) = reasoning.get("signature").and_then(Value::as_str) {
|
||||
*signature = Some(sig.to_string());
|
||||
}
|
||||
if let Some(blob) = reasoning.get("redactedContent").and_then(Value::as_str) {
|
||||
*redacted = Some(blob.to_string());
|
||||
}
|
||||
return events;
|
||||
}
|
||||
|
||||
Vec::new()
|
||||
}
|
||||
|
||||
fn on_block_stop(&mut self, payload: &Value) -> Vec<StreamEvent> {
|
||||
let index = Self::block_index(payload);
|
||||
let Some(block) = self.blocks.remove(&index) else {
|
||||
return Vec::new();
|
||||
};
|
||||
match block {
|
||||
BlockState::Text(text) => {
|
||||
let mut events = Vec::new();
|
||||
if self.text_started {
|
||||
self.text_started = false;
|
||||
events.push(StreamEvent::TextEnd { text_id: None });
|
||||
}
|
||||
if !text.is_empty() {
|
||||
self.parts.push(ContentPart::text(&text));
|
||||
}
|
||||
events
|
||||
}
|
||||
BlockState::Reasoning {
|
||||
text,
|
||||
signature,
|
||||
redacted,
|
||||
} => {
|
||||
let part = if let Some(blob) = redacted {
|
||||
ThinkingData {
|
||||
text: blob,
|
||||
signature: None,
|
||||
redacted: true,
|
||||
}
|
||||
} else {
|
||||
ThinkingData {
|
||||
text,
|
||||
signature,
|
||||
redacted: false,
|
||||
}
|
||||
};
|
||||
self.parts.push(ContentPart::Thinking(part));
|
||||
vec![StreamEvent::ReasoningEnd]
|
||||
}
|
||||
BlockState::ToolUse { id, name, input } => {
|
||||
// A no-argument tool call streams no input fragments, leaving
|
||||
// the buffer empty; canonically that is an empty object, not
|
||||
// null (matching the anthropic/openai codecs, and what Bedrock
|
||||
// wants back on re-encode).
|
||||
let arguments = parse_tool_arguments_or_empty(&input);
|
||||
let mut tool_call = ToolCall::new(&id, &name, arguments);
|
||||
tool_call.raw_arguments = Some(input);
|
||||
self.parts.push(ContentPart::ToolCall(tool_call.clone()));
|
||||
vec![StreamEvent::ToolCallEnd { tool_call }]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Build the final `Finish` from accumulated state.
|
||||
fn finish_event(&mut self) -> StreamEvent {
|
||||
self.finished = true;
|
||||
// Flush any blocks that never saw a contentBlockStop.
|
||||
let dangling: Vec<u64> = self.blocks.keys().copied().collect();
|
||||
for index in dangling {
|
||||
let _ = self.on_block_stop(&serde_json::json!({ "contentBlockIndex": index }));
|
||||
}
|
||||
|
||||
let response = Response {
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
model: self.model.clone(),
|
||||
provider: self.provider_name.clone(),
|
||||
message: Message {
|
||||
role: Role::Assistant,
|
||||
content: std::mem::take(&mut self.parts),
|
||||
name: None,
|
||||
tool_call_id: None,
|
||||
},
|
||||
finish_reason: self.finish_reason.clone(),
|
||||
usage: self.usage.clone(),
|
||||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: self.rate_limit.clone(),
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
StreamEvent::finish(self.finish_reason.clone(), self.usage.clone(), response)
|
||||
}
|
||||
}
|
||||
|
||||
impl StreamDecoder for ConverseStreamDecoder {
|
||||
fn on_event(&mut self, ev: RawEvent<'_>) -> Result<Vec<StreamEvent>, Error> {
|
||||
let Some(event_type) = ev.event else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let payload: Value = serde_json::from_str(ev.data)
|
||||
.map_err(|e| Error::stream_error(format!("converse stream event json: {e}"), e))?;
|
||||
|
||||
Ok(match event_type {
|
||||
"messageStart" => vec![StreamEvent::StreamStart],
|
||||
"contentBlockStart" => self.on_block_start(&payload),
|
||||
"contentBlockDelta" => self.on_block_delta(&payload),
|
||||
"contentBlockStop" => self.on_block_stop(&payload),
|
||||
"messageStop" => {
|
||||
self.finish_reason =
|
||||
map_stop_reason(payload.get("stopReason").and_then(Value::as_str));
|
||||
Vec::new()
|
||||
}
|
||||
"metadata" => {
|
||||
self.usage = token_counts_from_usage(payload.get("usage"));
|
||||
vec![self.finish_event()]
|
||||
}
|
||||
// Tolerate unknown event types — the union grows.
|
||||
_ => Vec::new(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Byte-stream end: `metadata` is the documented terminus, but if the
|
||||
/// stream ends without one, synthesize the `Finish` from accumulated
|
||||
/// state so callers still receive a response (mirrors the gemini
|
||||
/// decoder's unconditional synthesis).
|
||||
fn finish(&mut self) -> Vec<StreamEvent> {
|
||||
if self.finished {
|
||||
return Vec::new();
|
||||
}
|
||||
vec![self.finish_event()]
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::codec::CodecParams;
|
||||
use crate::types::{Message as RequestMessage, Request};
|
||||
|
||||
fn decoder() -> ConverseStreamDecoder {
|
||||
let request = Request {
|
||||
model: "us.anthropic.claude-sonnet-4-6".to_string(),
|
||||
messages: vec![RequestMessage::user("hi")],
|
||||
provider: Some("bedrock".to_string()),
|
||||
tools: None,
|
||||
tool_choice: None,
|
||||
response_format: None,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: None,
|
||||
stop_sequences: None,
|
||||
reasoning_effort: None,
|
||||
speed: None,
|
||||
metadata: None,
|
||||
provider_options: None,
|
||||
};
|
||||
let params = CodecParams::default();
|
||||
let ctx = CodecCtx {
|
||||
request: &request,
|
||||
provider_name: "bedrock",
|
||||
deployment_id: "us.anthropic.claude-sonnet-4-6",
|
||||
model: None,
|
||||
params: ¶ms,
|
||||
};
|
||||
ConverseStreamDecoder::new(&ctx, None)
|
||||
}
|
||||
|
||||
fn feed(decoder: &mut ConverseStreamDecoder, event: &str, data: &str) -> Vec<StreamEvent> {
|
||||
decoder
|
||||
.on_event(RawEvent {
|
||||
event: Some(event),
|
||||
data,
|
||||
})
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_happy_path_finishes_on_metadata() {
|
||||
let mut d = decoder();
|
||||
assert!(matches!(
|
||||
feed(&mut d, "messageStart", r#"{"role":"assistant"}"#)[0],
|
||||
StreamEvent::StreamStart
|
||||
));
|
||||
let events = feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"Hel"},"contentBlockIndex":0}"#,
|
||||
);
|
||||
assert!(matches!(events[0], StreamEvent::TextStart { .. }));
|
||||
assert!(matches!(events[1], StreamEvent::TextDelta { .. }));
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"lo"},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let stop = feed(&mut d, "contentBlockStop", r#"{"contentBlockIndex":0}"#);
|
||||
assert!(matches!(stop[0], StreamEvent::TextEnd { .. }));
|
||||
assert!(feed(&mut d, "messageStop", r#"{"stopReason":"end_turn"}"#).is_empty());
|
||||
|
||||
let finish = feed(
|
||||
&mut d,
|
||||
"metadata",
|
||||
r#"{"usage":{"inputTokens":12,"outputTokens":5,"totalTokens":17}}"#,
|
||||
);
|
||||
let StreamEvent::Finish {
|
||||
finish_reason,
|
||||
usage,
|
||||
response,
|
||||
} = &finish[0]
|
||||
else {
|
||||
panic!("expected Finish");
|
||||
};
|
||||
assert_eq!(*finish_reason, FinishReason::Stop);
|
||||
assert_eq!(usage.input_tokens, 12);
|
||||
assert_eq!(response.text(), "Hello");
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
// Byte-stream end after metadata adds nothing.
|
||||
assert!(d.finish().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_argument_tool_call_decodes_empty_object_not_null() {
|
||||
// A no-arg tool call (e.g. TaskList) streams no input fragments; the
|
||||
// arguments must be `{}` so it re-encodes to a valid Converse input.
|
||||
let mut d = decoder();
|
||||
feed(&mut d, "messageStart", r#"{"role":"assistant"}"#);
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockStart",
|
||||
r#"{"start":{"toolUse":{"toolUseId":"tool-1","name":"TaskList"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let stop = feed(&mut d, "contentBlockStop", r#"{"contentBlockIndex":0}"#);
|
||||
let StreamEvent::ToolCallEnd { tool_call } = &stop[0] else {
|
||||
panic!("expected ToolCallEnd");
|
||||
};
|
||||
assert_eq!(tool_call.arguments, serde_json::json!({}));
|
||||
assert!(!tool_call.arguments.is_null());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_use_accumulates_string_input_fragments() {
|
||||
let mut d = decoder();
|
||||
feed(&mut d, "messageStart", r#"{"role":"assistant"}"#);
|
||||
let start = feed(
|
||||
&mut d,
|
||||
"contentBlockStart",
|
||||
r#"{"start":{"toolUse":{"toolUseId":"tool-1","name":"search"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
assert!(matches!(start[0], StreamEvent::ToolCallStart { .. }));
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"toolUse":{"input":"{\"que"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"toolUse":{"input":"ry\":\"foo\"}"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let stop = feed(&mut d, "contentBlockStop", r#"{"contentBlockIndex":0}"#);
|
||||
let StreamEvent::ToolCallEnd { tool_call } = &stop[0] else {
|
||||
panic!("expected ToolCallEnd");
|
||||
};
|
||||
assert_eq!(tool_call.id, "tool-1");
|
||||
assert_eq!(tool_call.arguments["query"], "foo");
|
||||
|
||||
feed(&mut d, "messageStop", r#"{"stopReason":"tool_use"}"#);
|
||||
let finish = feed(
|
||||
&mut d,
|
||||
"metadata",
|
||||
r#"{"usage":{"inputTokens":1,"outputTokens":1}}"#,
|
||||
);
|
||||
let StreamEvent::Finish { finish_reason, .. } = &finish[0] else {
|
||||
panic!("expected Finish");
|
||||
};
|
||||
assert_eq!(*finish_reason, FinishReason::ToolCalls);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reasoning_deltas_round_trip_signature() {
|
||||
let mut d = decoder();
|
||||
let events = feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"reasoningContent":{"text":"thinking"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
assert!(matches!(events[0], StreamEvent::ReasoningStart));
|
||||
assert!(matches!(events[1], StreamEvent::ReasoningDelta { .. }));
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"reasoningContent":{"signature":"sig-9"}},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let stop = feed(&mut d, "contentBlockStop", r#"{"contentBlockIndex":0}"#);
|
||||
assert!(matches!(stop[0], StreamEvent::ReasoningEnd));
|
||||
|
||||
let finish = feed(
|
||||
&mut d,
|
||||
"metadata",
|
||||
r#"{"usage":{"inputTokens":1,"outputTokens":1}}"#,
|
||||
);
|
||||
let StreamEvent::Finish { response, .. } = &finish[0] else {
|
||||
panic!("expected Finish");
|
||||
};
|
||||
let ContentPart::Thinking(thinking) = &response.message.content[0] else {
|
||||
panic!("expected thinking part");
|
||||
};
|
||||
assert_eq!(thinking.text, "thinking");
|
||||
assert_eq!(thinking.signature.as_deref(), Some("sig-9"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stream_end_without_metadata_synthesizes_finish() {
|
||||
let mut d = decoder();
|
||||
feed(
|
||||
&mut d,
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"partial"},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let events = d.finish();
|
||||
let StreamEvent::Finish { response, .. } = &events[0] else {
|
||||
panic!("expected synthesized Finish");
|
||||
};
|
||||
assert_eq!(response.text(), "partial");
|
||||
// Synthesis happens once.
|
||||
assert!(d.finish().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_events_are_tolerated() {
|
||||
let mut d = decoder();
|
||||
assert!(feed(&mut d, "futureEventKind", r#"{"anything":1}"#).is_empty());
|
||||
assert!(
|
||||
d.on_event(RawEvent {
|
||||
event: None,
|
||||
data: "{}",
|
||||
})
|
||||
.unwrap()
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -11,6 +11,7 @@
|
|||
//! methods, never extend the contract.
|
||||
|
||||
pub(crate) mod anthropic_messages;
|
||||
pub(crate) mod bedrock_converse;
|
||||
pub(crate) mod gemini_generate;
|
||||
pub(crate) mod openai_compatible;
|
||||
pub(crate) mod openai_responses;
|
||||
|
|
@ -20,6 +21,35 @@ use fabro_model::Model;
|
|||
use crate::error::{Error, error_from_status_code};
|
||||
use crate::types::{Message, RateLimitInfo, Request, Response, Role, StreamEvent};
|
||||
|
||||
/// Parse a streamed/generated tool-argument JSON string, defaulting malformed
|
||||
/// or absent arguments to the canonical no-argument object.
|
||||
pub(crate) fn parse_tool_arguments_or_empty(raw_arguments: &str) -> serde_json::Value {
|
||||
serde_json::from_str(raw_arguments).unwrap_or_else(|_| serde_json::json!({}))
|
||||
}
|
||||
|
||||
/// Merge `provider_options.<provider_name>` fields into an encoded request
|
||||
/// body. Used by codecs whose provider-options namespace is adapter-name keyed
|
||||
/// rather than a single fixed provider.
|
||||
pub(crate) fn merge_named_provider_options(
|
||||
body: &mut serde_json::Value,
|
||||
provider_options: Option<&serde_json::Value>,
|
||||
provider_name: &str,
|
||||
) {
|
||||
let Some(opts) = provider_options.and_then(|opts| opts.get(provider_name)) else {
|
||||
return;
|
||||
};
|
||||
let Some(body_map) = body.as_object_mut() else {
|
||||
return;
|
||||
};
|
||||
let Some(opts_map) = opts.as_object() else {
|
||||
return;
|
||||
};
|
||||
|
||||
for (key, value) in opts_map {
|
||||
body_map.insert(key.clone(), value.clone());
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-request context. Borrowed — the codec reads what it needs and returns.
|
||||
pub(crate) struct CodecCtx<'a> {
|
||||
/// The canonical request being translated. Decoders read it too
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
use super::translate;
|
||||
use super::wire::ApiRequest;
|
||||
use crate::codec::{CodecCtx, EncodedRequest};
|
||||
use crate::codec::{CodecCtx, EncodedRequest, merge_named_provider_options};
|
||||
|
||||
/// Build the Chat Completions request for `ctx.request`. `stream` toggles the
|
||||
/// `stream` body field. The body is assembled as a `serde_json::Value` so
|
||||
|
|
@ -63,19 +63,7 @@ pub(super) fn merge_provider_options(
|
|||
provider_options: Option<&serde_json::Value>,
|
||||
provider_name: &str,
|
||||
) {
|
||||
let Some(opts) = provider_options.and_then(|opts| opts.get(provider_name)) else {
|
||||
return;
|
||||
};
|
||||
let Some(body_map) = body.as_object_mut() else {
|
||||
return;
|
||||
};
|
||||
let Some(opts_map) = opts.as_object() else {
|
||||
return;
|
||||
};
|
||||
|
||||
for (key, value) in opts_map {
|
||||
body_map.insert(key.clone(), value.clone());
|
||||
}
|
||||
merge_named_provider_options(body, provider_options, provider_name);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
use serde::Deserialize;
|
||||
|
||||
use super::wire::{ApiResponse, ApiUsage, InputTokensResponse};
|
||||
use crate::codec::CodecCtx;
|
||||
use crate::codec::{CodecCtx, parse_tool_arguments_or_empty};
|
||||
use crate::error::Error;
|
||||
use crate::types::{
|
||||
ContentPart, FinishReason, Message, RateLimitInfo, Response, Role, TokenCounts, ToolCall,
|
||||
|
|
@ -75,7 +75,7 @@ pub(super) fn tool_call_from_item(item: &serde_json::Value, custom: bool) -> Too
|
|||
.get("arguments")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.unwrap_or("{}");
|
||||
let arguments = serde_json::from_str(args_str).unwrap_or_else(|_| serde_json::json!({}));
|
||||
let arguments = parse_tool_arguments_or_empty(args_str);
|
||||
let mut tc = ToolCall::new(call_id, name, arguments);
|
||||
tc.raw_arguments = Some(args_str.to_string());
|
||||
tc
|
||||
|
|
|
|||
219
lib/crates/fabro-llm/src/providers/bedrock/eventstream.rs
Normal file
219
lib/crates/fabro-llm/src/providers/bedrock/eventstream.rs
Normal file
|
|
@ -0,0 +1,219 @@
|
|||
//! Decoder for Bedrock's `application/vnd.amazon.eventstream` streaming
|
||||
//! responses.
|
||||
//!
|
||||
//! ConverseStream wraps each event in a binary event-stream frame: the event
|
||||
//! name (`messageStart`, `contentBlockDelta`, `metadata`, ...) travels in the
|
||||
//! frame's `:event-type` header and the payload is that event's JSON
|
||||
//! directly. (The base64 `{"bytes": ...}` wrapping belongs to
|
||||
//! `InvokeModelWithResponseStream`'s `PayloadPart` and does not apply here.)
|
||||
//! Exception and error frames are surfaced as stream errors.
|
||||
|
||||
use aws_smithy_eventstream::frame::{DecodedFrame, MessageFrameDecoder};
|
||||
use aws_smithy_types::event_stream::Message;
|
||||
use aws_smithy_types::str_bytes::StrBytes;
|
||||
use bytes::BytesMut;
|
||||
|
||||
use crate::error::Error;
|
||||
|
||||
/// One decoded ConverseStream event: the `:event-type` header value plus the
|
||||
/// frame's JSON payload, ready to feed a stream decoder.
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct DecodedEvent {
|
||||
pub event_type: String,
|
||||
pub payload: String,
|
||||
}
|
||||
|
||||
/// Incremental decoder over event-stream bytes.
|
||||
pub(crate) struct FrameDecoder {
|
||||
inner: MessageFrameDecoder,
|
||||
buffer: BytesMut,
|
||||
}
|
||||
|
||||
impl FrameDecoder {
|
||||
pub(crate) fn new() -> Self {
|
||||
Self {
|
||||
inner: MessageFrameDecoder::new(),
|
||||
buffer: BytesMut::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Feed newly received bytes and return any complete events decoded from
|
||||
/// them. Bedrock exception and error frames are surfaced as errors.
|
||||
pub(crate) fn push(&mut self, bytes: &[u8]) -> Result<Vec<DecodedEvent>, Error> {
|
||||
self.buffer.extend_from_slice(bytes);
|
||||
let mut events = Vec::new();
|
||||
loop {
|
||||
// `decode_frame` advances `self.buffer` and retains partial-frame
|
||||
// state internally, so repeated calls over a growing buffer work.
|
||||
let frame = self.inner.decode_frame(&mut self.buffer).map_err(|e| {
|
||||
Error::stream_error(
|
||||
format!("bedrock event-stream decode: {e}"),
|
||||
std::io::Error::other(e.to_string()),
|
||||
)
|
||||
})?;
|
||||
match frame {
|
||||
DecodedFrame::Complete(message) => {
|
||||
if let Some(event) = Self::message_to_event(&message)? {
|
||||
events.push(event);
|
||||
}
|
||||
}
|
||||
DecodedFrame::Incomplete => break,
|
||||
}
|
||||
}
|
||||
Ok(events)
|
||||
}
|
||||
|
||||
/// Classify one event-stream message.
|
||||
///
|
||||
/// `event` frames yield their `:event-type` name and JSON payload;
|
||||
/// `exception` frames (modeled AWS errors such as `throttlingException`,
|
||||
/// arriving in-band after HTTP 200) and `error` frames (unmodeled) are
|
||||
/// turned into errors. Frames without an event type are skipped.
|
||||
fn message_to_event(message: &Message) -> Result<Option<DecodedEvent>, Error> {
|
||||
match header_str(message, ":message-type") {
|
||||
Some("exception") => {
|
||||
let kind = header_str(message, ":exception-type").unwrap_or("unknown");
|
||||
let body = String::from_utf8_lossy(message.payload());
|
||||
Err(Error::stream_error(
|
||||
format!("bedrock stream exception ({kind}): {body}"),
|
||||
std::io::Error::other("bedrock event-stream exception frame"),
|
||||
))
|
||||
}
|
||||
Some("error") => {
|
||||
let code = header_str(message, ":error-code").unwrap_or("unknown");
|
||||
let detail = header_str(message, ":error-message").unwrap_or("");
|
||||
Err(Error::stream_error(
|
||||
format!("bedrock stream error ({code}): {detail}"),
|
||||
std::io::Error::other("bedrock event-stream error frame"),
|
||||
))
|
||||
}
|
||||
_ => {
|
||||
let Some(event_type) = header_str(message, ":event-type") else {
|
||||
return Ok(None);
|
||||
};
|
||||
Ok(Some(DecodedEvent {
|
||||
event_type: event_type.to_string(),
|
||||
payload: String::from_utf8_lossy(message.payload()).into_owned(),
|
||||
}))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Read a string-valued event-stream header by name.
|
||||
fn header_str<'a>(message: &'a Message, name: &str) -> Option<&'a str> {
|
||||
message
|
||||
.headers()
|
||||
.iter()
|
||||
.find(|header| header.name().as_str() == name)
|
||||
.and_then(|header| header.value().as_string().ok())
|
||||
.map(StrBytes::as_str)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) mod tests {
|
||||
use aws_smithy_eventstream::frame::write_message_to;
|
||||
use aws_smithy_types::event_stream::{Header, HeaderValue, Message};
|
||||
|
||||
use super::*;
|
||||
|
||||
/// Build one ConverseStream event frame: event name in `:event-type`,
|
||||
/// payload = the event JSON directly.
|
||||
fn encode_event_frame(event_type: &str, payload_json: &str) -> Vec<u8> {
|
||||
let message = Message::new(payload_json.as_bytes().to_vec())
|
||||
.add_header(Header::new(
|
||||
":message-type",
|
||||
HeaderValue::String("event".into()),
|
||||
))
|
||||
.add_header(Header::new(
|
||||
":event-type",
|
||||
HeaderValue::String(event_type.to_string().into()),
|
||||
))
|
||||
.add_header(Header::new(
|
||||
":content-type",
|
||||
HeaderValue::String("application/json".into()),
|
||||
));
|
||||
let mut buf = Vec::new();
|
||||
write_message_to(&message, &mut buf).unwrap();
|
||||
buf
|
||||
}
|
||||
|
||||
/// Build a full streaming body from `(event_type, payload_json)` pairs.
|
||||
pub(crate) fn build_stream_body(events: &[(&str, &str)]) -> Vec<u8> {
|
||||
let mut body = Vec::new();
|
||||
for (event_type, payload) in events {
|
||||
body.extend_from_slice(&encode_event_frame(event_type, payload));
|
||||
}
|
||||
body
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decodes_event_frame_to_typed_payload() {
|
||||
let frame = encode_event_frame(
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"hi"},"contentBlockIndex":0}"#,
|
||||
);
|
||||
let mut decoder = FrameDecoder::new();
|
||||
let events = decoder.push(&frame).unwrap();
|
||||
assert_eq!(events.len(), 1);
|
||||
assert_eq!(events[0].event_type, "contentBlockDelta");
|
||||
let payload: serde_json::Value = serde_json::from_str(&events[0].payload).unwrap();
|
||||
assert_eq!(payload["delta"]["text"], "hi");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reassembles_frame_split_across_pushes() {
|
||||
let frame = encode_event_frame("messageStop", r#"{"stopReason":"end_turn"}"#);
|
||||
let split = frame.len() / 2;
|
||||
let mut decoder = FrameDecoder::new();
|
||||
assert!(decoder.push(&frame[..split]).unwrap().is_empty());
|
||||
let events = decoder.push(&frame[split..]).unwrap();
|
||||
assert_eq!(events.len(), 1);
|
||||
assert_eq!(events[0].event_type, "messageStop");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exception_frame_surfaces_as_error() {
|
||||
let message = Message::new(br#"{"message":"Too many requests"}"#.to_vec())
|
||||
.add_header(Header::new(
|
||||
":message-type",
|
||||
HeaderValue::String("exception".into()),
|
||||
))
|
||||
.add_header(Header::new(
|
||||
":exception-type",
|
||||
HeaderValue::String("throttlingException".into()),
|
||||
));
|
||||
let mut buf = Vec::new();
|
||||
write_message_to(&message, &mut buf).unwrap();
|
||||
|
||||
let mut decoder = FrameDecoder::new();
|
||||
let err = decoder.push(&buf).unwrap_err();
|
||||
let rendered = err.to_string();
|
||||
assert!(rendered.contains("throttlingException"), "{rendered}");
|
||||
assert!(rendered.contains("Too many requests"), "{rendered}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unmodeled_error_frame_surfaces_as_error() {
|
||||
let message = Message::new(Vec::new())
|
||||
.add_header(Header::new(
|
||||
":message-type",
|
||||
HeaderValue::String("error".into()),
|
||||
))
|
||||
.add_header(Header::new(
|
||||
":error-code",
|
||||
HeaderValue::String("InternalError".into()),
|
||||
))
|
||||
.add_header(Header::new(
|
||||
":error-message",
|
||||
HeaderValue::String("stream broke".into()),
|
||||
));
|
||||
let mut buf = Vec::new();
|
||||
write_message_to(&message, &mut buf).unwrap();
|
||||
|
||||
let mut decoder = FrameDecoder::new();
|
||||
let err = decoder.push(&buf).unwrap_err();
|
||||
let rendered = err.to_string();
|
||||
assert!(rendered.contains("InternalError"), "{rendered}");
|
||||
}
|
||||
}
|
||||
691
lib/crates/fabro-llm/src/providers/bedrock/mod.rs
Normal file
691
lib/crates/fabro-llm/src/providers/bedrock/mod.rs
Normal file
|
|
@ -0,0 +1,691 @@
|
|||
//! Provider adapter for Amazon Bedrock (Converse/ConverseStream).
|
||||
//!
|
||||
//! A thin transport shell over the `bedrock_converse` codec: it owns auth
|
||||
//! (SigV4 signing or a bearer Bedrock API key), the region derivation, and
|
||||
//! the AWS event-stream byte loop. All wire translation lives in the codec;
|
||||
//! one codec serves every Converse-capable family because AWS translates the
|
||||
//! envelope server-side.
|
||||
|
||||
pub(crate) mod eventstream;
|
||||
pub(crate) mod sigv4;
|
||||
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use eventstream::FrameDecoder;
|
||||
use fabro_auth::ApiKeyHeader;
|
||||
use fabro_model::Catalog;
|
||||
use futures::stream;
|
||||
use sigv4::Sigv4Signer;
|
||||
use tokio::sync::OnceCell;
|
||||
use tokio::time;
|
||||
|
||||
use crate::adapter_registry::AdapterConfig;
|
||||
#[cfg(test)]
|
||||
use crate::adapter_registry::AdapterKindOptions;
|
||||
use crate::attachments::{self, AttachmentPolicy};
|
||||
use crate::codec::bedrock_converse::BedrockConverse;
|
||||
use crate::codec::{Codec, CodecCtx, CodecParams, EncodedRequest, RawEvent, StreamDecoder};
|
||||
use crate::error::Error;
|
||||
use crate::provider::{self, ProviderAdapter, StreamEventStream};
|
||||
use crate::providers::common::{self as common};
|
||||
use crate::transport::{self, HttpTransport};
|
||||
use crate::types::{AdapterTimeout, Request, Response, StreamEvent};
|
||||
|
||||
/// How the adapter authenticates to Bedrock.
|
||||
pub(crate) enum BedrockAuth {
|
||||
/// Bedrock API key, sent as an `Authorization: Bearer` token.
|
||||
ApiKey(String),
|
||||
/// SigV4 signing. The signer (holding the AWS default credential chain)
|
||||
/// is resolved on first use and cached; the chain itself re-resolves
|
||||
/// expiring credentials per request. Tests pre-seed the cell with a
|
||||
/// static signer.
|
||||
Sigv4(OnceCell<Sigv4Signer>),
|
||||
}
|
||||
|
||||
/// Build a boxed Bedrock adapter from a resolved [`AdapterConfig`].
|
||||
///
|
||||
/// Kept in this module (rather than the generic adapter registry) so that
|
||||
/// Bedrock-specific construction stays encapsulated here. The auth mode is
|
||||
/// implied by the resolved credential: an `aws_sigv4` credential signs with
|
||||
/// the AWS chain; a static token is sent as a bearer API key.
|
||||
pub(crate) fn build(config: AdapterConfig) -> Result<Arc<dyn ProviderAdapter>, Error> {
|
||||
let base_url = config
|
||||
.base_url
|
||||
.clone()
|
||||
.ok_or_else(|| Error::Configuration {
|
||||
message: format!(
|
||||
"bedrock provider '{}' requires a base_url (the Bedrock runtime endpoint)",
|
||||
config.provider_id
|
||||
),
|
||||
source: None,
|
||||
})?;
|
||||
let adapter = match config.auth_header {
|
||||
Some(ApiKeyHeader::AwsSigv4) => Adapter::new_sigv4(base_url)?,
|
||||
Some(ApiKeyHeader::Bearer(token)) => Adapter::new_api_key(token, base_url)?,
|
||||
Some(ApiKeyHeader::Custom { name, .. }) => {
|
||||
return Err(Error::Configuration {
|
||||
message: format!(
|
||||
"bedrock provider '{}' does not support custom auth header '{}' (use bearer \
|
||||
credentials or aws_sigv4)",
|
||||
config.provider_id, name
|
||||
),
|
||||
source: None,
|
||||
});
|
||||
}
|
||||
None => {
|
||||
return Err(Error::Configuration {
|
||||
message: format!(
|
||||
"bedrock provider '{}' has no resolved credential (configure `aws_sigv4` or \
|
||||
an API key)",
|
||||
config.provider_id
|
||||
),
|
||||
source: None,
|
||||
});
|
||||
}
|
||||
};
|
||||
let mut adapter = adapter.with_name(config.provider_id);
|
||||
if !config.extra_headers.is_empty() {
|
||||
adapter = adapter.with_default_headers(config.extra_headers);
|
||||
}
|
||||
if let Some(catalog) = config.catalog {
|
||||
adapter = adapter.with_catalog(catalog);
|
||||
}
|
||||
Ok(Arc::new(adapter))
|
||||
}
|
||||
|
||||
/// Provider adapter for Amazon Bedrock.
|
||||
pub struct Adapter {
|
||||
pub(crate) http: HttpTransport,
|
||||
provider_name: String,
|
||||
region: String,
|
||||
auth: BedrockAuth,
|
||||
catalog: Option<Arc<Catalog>>,
|
||||
}
|
||||
|
||||
impl Adapter {
|
||||
/// Construct an adapter that authenticates with a Bedrock API key.
|
||||
/// `base_url` is the Bedrock runtime endpoint; the signing region is
|
||||
/// parsed from it.
|
||||
pub fn new_api_key(
|
||||
token: impl Into<String>,
|
||||
base_url: impl Into<String>,
|
||||
) -> Result<Self, Error> {
|
||||
Self::with_auth(base_url, BedrockAuth::ApiKey(token.into()))
|
||||
}
|
||||
|
||||
/// Construct a SigV4 adapter. Credentials resolve lazily from the AWS
|
||||
/// default chain on the first request, so construction stays synchronous.
|
||||
pub fn new_sigv4(base_url: impl Into<String>) -> Result<Self, Error> {
|
||||
Self::with_auth(base_url, BedrockAuth::Sigv4(OnceCell::new()))
|
||||
}
|
||||
|
||||
fn with_auth(base_url: impl Into<String>, auth: BedrockAuth) -> Result<Self, Error> {
|
||||
let base_url = base_url.into();
|
||||
let region = region_from_base_url(&base_url)?;
|
||||
Ok(Self {
|
||||
http: HttpTransport::new_optional(None, base_url),
|
||||
provider_name: "bedrock".to_string(),
|
||||
region,
|
||||
auth,
|
||||
catalog: None,
|
||||
})
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_name(mut self, name: impl Into<String>) -> Self {
|
||||
self.provider_name = name.into();
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_catalog(mut self, catalog: Arc<Catalog>) -> Self {
|
||||
self.catalog = Some(catalog);
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_default_headers(mut self, headers: HashMap<String, String>) -> Self {
|
||||
self.http = self.http.with_default_headers(headers);
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_timeout(self, timeout: AdapterTimeout) -> Self {
|
||||
Self {
|
||||
http: self.http.with_timeout(timeout),
|
||||
..self
|
||||
}
|
||||
}
|
||||
|
||||
fn codec_ctx<'a>(
|
||||
&'a self,
|
||||
request: &'a Request,
|
||||
deployment_id: &'a str,
|
||||
params: &'a CodecParams,
|
||||
) -> CodecCtx<'a> {
|
||||
CodecCtx {
|
||||
request,
|
||||
provider_name: &self.provider_name,
|
||||
deployment_id,
|
||||
model: common::catalog_model(self.catalog.as_deref(), &request.model),
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve file-backed attachments to inline data first: Converse takes
|
||||
/// inline image and document bytes (no URL sources).
|
||||
async fn resolve_request<'a>(&self, request: &'a Request) -> std::borrow::Cow<'a, Request> {
|
||||
let policy = AttachmentPolicy {
|
||||
images: true,
|
||||
documents: true,
|
||||
audio: false,
|
||||
};
|
||||
attachments::resolve(request, policy).await
|
||||
}
|
||||
|
||||
/// Build the signed/bearer HTTP request for an encoded Converse call.
|
||||
async fn build_http_request(
|
||||
&self,
|
||||
encoded: &EncodedRequest,
|
||||
stream: bool,
|
||||
) -> Result<fabro_http::RequestBuilder, Error> {
|
||||
let url = format!("{}{}", self.http.base_url, encoded.endpoint);
|
||||
let body = serde_json::to_vec(&encoded.body).map_err(|e| Error::Configuration {
|
||||
message: format!("failed to serialize converse request: {e}"),
|
||||
source: None,
|
||||
})?;
|
||||
|
||||
let mut req = self.http.client.post(&url);
|
||||
for (key, value) in &self.http.default_headers {
|
||||
req = req.header(key, value);
|
||||
}
|
||||
for (key, value) in &encoded.headers {
|
||||
req = req.header(key, value);
|
||||
}
|
||||
|
||||
req = match &self.auth {
|
||||
BedrockAuth::ApiKey(token) => req.bearer_auth(token).body(body),
|
||||
BedrockAuth::Sigv4(cell) => {
|
||||
let signer = cell
|
||||
.get_or_try_init(Sigv4Signer::from_default_chain)
|
||||
.await?;
|
||||
signer.sign_post(req, &self.region, &url, body).await?
|
||||
}
|
||||
};
|
||||
|
||||
req = req.header("content-type", "application/json");
|
||||
if stream {
|
||||
req = req.header("accept", "application/vnd.amazon.eventstream");
|
||||
}
|
||||
if let Some(t) = self.http.request_timeout {
|
||||
if !stream {
|
||||
req = req.timeout(t);
|
||||
}
|
||||
}
|
||||
Ok(req)
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl ProviderAdapter for Adapter {
|
||||
fn name(&self) -> &str {
|
||||
&self.provider_name
|
||||
}
|
||||
|
||||
async fn complete(&self, request: &Request) -> Result<Response, Error> {
|
||||
self.validate_request(request)?;
|
||||
|
||||
let resolved = self.resolve_request(request).await;
|
||||
let codec = BedrockConverse;
|
||||
let deployment_id = common::api_model_id(self.catalog.as_deref(), &resolved.model);
|
||||
let params = CodecParams::default();
|
||||
let ctx = self.codec_ctx(&resolved, &deployment_id, ¶ms);
|
||||
|
||||
let encoded = codec.encode(&ctx, false)?;
|
||||
let req = self.build_http_request(&encoded, false).await?;
|
||||
transport::complete_via_http(req, &codec, &ctx).await
|
||||
}
|
||||
|
||||
async fn stream(&self, request: &Request) -> Result<StreamEventStream, Error> {
|
||||
self.validate_request(request)?;
|
||||
|
||||
let resolved = self.resolve_request(request).await;
|
||||
let codec = BedrockConverse;
|
||||
let deployment_id = common::api_model_id(self.catalog.as_deref(), &resolved.model);
|
||||
let params = CodecParams::default();
|
||||
let ctx = self.codec_ctx(&resolved, &deployment_id, ¶ms);
|
||||
|
||||
let encoded = codec.encode(&ctx, true)?;
|
||||
let req = self.build_http_request(&encoded, true).await?;
|
||||
|
||||
let http_resp = req
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| Error::network(e.to_string(), e))?;
|
||||
let status = http_resp.status();
|
||||
if !status.is_success() {
|
||||
let retry_after = transport::parse_retry_after(http_resp.headers());
|
||||
let body = http_resp
|
||||
.text()
|
||||
.await
|
||||
.map_err(|e| Error::network(e.to_string(), e))?;
|
||||
return Err(codec.decode_error(status.as_u16(), &body, &ctx, retry_after));
|
||||
}
|
||||
|
||||
let rate_limit = transport::parse_rate_limit_headers(http_resp.headers());
|
||||
let decoder = codec.stream_decoder(&ctx, rate_limit);
|
||||
Ok(decode_eventstream(
|
||||
http_resp,
|
||||
decoder,
|
||||
self.http.stream_read_timeout,
|
||||
))
|
||||
}
|
||||
|
||||
fn supports_tool_choice(&self, mode: &str) -> bool {
|
||||
// Converse has no `none` tool choice on the wire.
|
||||
matches!(mode, "auto" | "required" | "named")
|
||||
}
|
||||
|
||||
fn validate_request(&self, request: &Request) -> Result<(), Error> {
|
||||
if let Some(tool_choice) = &request.tool_choice {
|
||||
provider::validate_tool_choice(self, tool_choice)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// State driving the event-stream byte loop: the codec's decoder plus the
|
||||
/// frame decoder, with a buffer that flattens batched events.
|
||||
struct EventStreamLoop {
|
||||
response: fabro_http::Response,
|
||||
frames: FrameDecoder,
|
||||
decoder: Box<dyn StreamDecoder>,
|
||||
pending: VecDeque<StreamEvent>,
|
||||
done: bool,
|
||||
/// `finish()` already drained.
|
||||
finished: bool,
|
||||
timeout: Option<Duration>,
|
||||
}
|
||||
|
||||
/// Drive `decoder` over the AWS event-stream byte stream of `response`: the
|
||||
/// event-stream sibling of the transport's shared SSE loop, anticipated by
|
||||
/// the transport consolidation notes.
|
||||
fn decode_eventstream(
|
||||
response: fabro_http::Response,
|
||||
decoder: Box<dyn StreamDecoder>,
|
||||
timeout: Option<Duration>,
|
||||
) -> StreamEventStream {
|
||||
let out = stream::unfold(
|
||||
EventStreamLoop {
|
||||
response,
|
||||
frames: FrameDecoder::new(),
|
||||
decoder,
|
||||
pending: VecDeque::new(),
|
||||
done: false,
|
||||
finished: false,
|
||||
timeout,
|
||||
},
|
||||
move |mut state| async move {
|
||||
loop {
|
||||
if let Some(event) = state.pending.pop_front() {
|
||||
return Some((Ok(event), state));
|
||||
}
|
||||
|
||||
if state.done {
|
||||
if state.finished {
|
||||
return None;
|
||||
}
|
||||
state.finished = true;
|
||||
state.pending.extend(state.decoder.finish());
|
||||
if state.pending.is_empty() {
|
||||
return None;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
let chunk_result = match state.timeout {
|
||||
Some(timeout) => time::timeout(timeout, state.response.chunk()).await,
|
||||
None => Ok(state.response.chunk().await),
|
||||
};
|
||||
match chunk_result {
|
||||
Ok(Ok(Some(bytes))) => {
|
||||
let frames = match state.frames.push(&bytes) {
|
||||
Ok(frames) => frames,
|
||||
Err(e) => return Some((Err(e), state)),
|
||||
};
|
||||
for frame in frames {
|
||||
let raw = RawEvent {
|
||||
event: Some(frame.event_type.as_str()),
|
||||
data: frame.payload.as_str(),
|
||||
};
|
||||
match state.decoder.on_event(raw) {
|
||||
Ok(events) => state.pending.extend(events),
|
||||
Err(e) => return Some((Err(e), state)),
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(Ok(None)) => state.done = true,
|
||||
Ok(Err(e)) => {
|
||||
return Some((Err(Error::stream_error(e.to_string(), e)), state));
|
||||
}
|
||||
Err(_) => {
|
||||
return Some((
|
||||
Err(Error::Stream {
|
||||
message: "stream read timed out waiting for next event".to_string(),
|
||||
source: None,
|
||||
}),
|
||||
state,
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
);
|
||||
Box::pin(out)
|
||||
}
|
||||
|
||||
/// Derive the AWS region from a Bedrock runtime endpoint URL.
|
||||
///
|
||||
/// The region is a SigV4 signing parameter, so it is parsed from the
|
||||
/// configured base URL rather than carried as a separate AWS-specific config
|
||||
/// field. It is validated as `[a-z0-9-]` since it ultimately appears in a
|
||||
/// signed request.
|
||||
fn region_from_base_url(base_url: &str) -> Result<String, Error> {
|
||||
let invalid = || Error::Configuration {
|
||||
message: format!(
|
||||
"bedrock base_url '{base_url}' is not a recognized Bedrock runtime endpoint \
|
||||
(expected https://bedrock-runtime[-fips].<region>.amazonaws.com[.cn])"
|
||||
),
|
||||
source: None,
|
||||
};
|
||||
#[expect(
|
||||
clippy::disallowed_types,
|
||||
reason = "Bedrock region derivation needs URL host parsing; the raw URL is not logged or rendered."
|
||||
)]
|
||||
let parsed = fabro_http::Url::parse(base_url).map_err(|_| invalid())?;
|
||||
let host = parsed.host_str().ok_or_else(invalid)?;
|
||||
let rest = host
|
||||
.strip_prefix("bedrock-runtime-fips.")
|
||||
.or_else(|| host.strip_prefix("bedrock-runtime."))
|
||||
.ok_or_else(invalid)?;
|
||||
let region = rest
|
||||
.strip_suffix(".amazonaws.com.cn")
|
||||
.or_else(|| rest.strip_suffix(".amazonaws.com"))
|
||||
.ok_or_else(invalid)?;
|
||||
let valid = !region.is_empty()
|
||||
&& region
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-');
|
||||
if valid {
|
||||
Ok(region.to_string())
|
||||
} else {
|
||||
Err(invalid())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use futures::StreamExt;
|
||||
use httpmock::prelude::*;
|
||||
|
||||
use super::*;
|
||||
use crate::types::{FinishReason, Message};
|
||||
|
||||
fn make_request(model: &str) -> Request {
|
||||
Request {
|
||||
model: model.to_string(),
|
||||
messages: vec![Message::user("Hello")],
|
||||
provider: Some("bedrock".to_string()),
|
||||
tools: None,
|
||||
tool_choice: None,
|
||||
response_format: None,
|
||||
temperature: None,
|
||||
top_p: None,
|
||||
max_tokens: Some(64),
|
||||
stop_sequences: None,
|
||||
reasoning_effort: None,
|
||||
speed: None,
|
||||
metadata: None,
|
||||
provider_options: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Adapter pointed at httpmock: region parsing only applies to real
|
||||
/// bedrock-runtime URLs, so the test constructor sets the region field
|
||||
/// directly.
|
||||
fn test_adapter(server: &MockServer) -> Adapter {
|
||||
Adapter {
|
||||
http: HttpTransport::new_optional(None, server.base_url()),
|
||||
provider_name: "bedrock".to_string(),
|
||||
region: "us-east-1".to_string(),
|
||||
auth: BedrockAuth::ApiKey("test-bedrock-key".to_string()),
|
||||
catalog: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn region_parses_from_standard_endpoint() {
|
||||
assert_eq!(
|
||||
region_from_base_url("https://bedrock-runtime.eu-west-1.amazonaws.com").unwrap(),
|
||||
"eu-west-1"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn region_parses_from_fips_endpoint() {
|
||||
assert_eq!(
|
||||
region_from_base_url("https://bedrock-runtime-fips.us-gov-west-1.amazonaws.com")
|
||||
.unwrap(),
|
||||
"us-gov-west-1"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn region_parses_from_china_endpoint() {
|
||||
assert_eq!(
|
||||
region_from_base_url("https://bedrock-runtime.cn-north-1.amazonaws.com.cn").unwrap(),
|
||||
"cn-north-1"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn region_rejects_non_bedrock_hosts() {
|
||||
for url in [
|
||||
"https://example.com",
|
||||
"https://bedrock.us-east-1.amazonaws.com",
|
||||
"https://bedrock-runtime.amazonaws.com",
|
||||
] {
|
||||
assert!(region_from_base_url(url).is_err(), "{url}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn region_normalizes_hostname_case() {
|
||||
assert_eq!(
|
||||
region_from_base_url("https://bedrock-runtime.US-EAST-1.amazonaws.com").unwrap(),
|
||||
"us-east-1"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_posts_converse_body_with_bearer_auth() {
|
||||
let server = MockServer::start();
|
||||
let mock = server.mock(|when, then| {
|
||||
when.method(POST)
|
||||
.path("/model/us.anthropic.claude-sonnet-4-6/converse")
|
||||
.header("authorization", "Bearer test-bedrock-key")
|
||||
.json_body_includes(
|
||||
r#"{"messages":[{"role":"user","content":[{"text":"Hello"}]}],"inferenceConfig":{"maxTokens":64}}"#,
|
||||
);
|
||||
then.status(200)
|
||||
.header("content-type", "application/json")
|
||||
.json_body(serde_json::json!({
|
||||
"output": {"message": {"role": "assistant", "content": [{"text": "Hi!"}]}},
|
||||
"stopReason": "end_turn",
|
||||
"usage": {"inputTokens": 8, "outputTokens": 2, "totalTokens": 10}
|
||||
}));
|
||||
});
|
||||
|
||||
let adapter = test_adapter(&server);
|
||||
let response = adapter
|
||||
.complete(&make_request("us.anthropic.claude-sonnet-4-6"))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
mock.assert();
|
||||
assert_eq!(response.text(), "Hi!");
|
||||
assert_eq!(response.finish_reason, FinishReason::Stop);
|
||||
assert_eq!(response.usage.input_tokens, 8);
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_applies_default_headers() {
|
||||
let server = MockServer::start();
|
||||
let mock = server.mock(|when, then| {
|
||||
when.method(POST)
|
||||
.path("/model/m/converse")
|
||||
.header("x-fabro-test", "present");
|
||||
then.status(200)
|
||||
.header("content-type", "application/json")
|
||||
.json_body(serde_json::json!({
|
||||
"output": {"message": {"role": "assistant", "content": [{"text": "ok"}]}},
|
||||
"stopReason": "end_turn",
|
||||
"usage": {"inputTokens": 1, "outputTokens": 1, "totalTokens": 2}
|
||||
}));
|
||||
});
|
||||
|
||||
let adapter = test_adapter(&server).with_default_headers(HashMap::from([(
|
||||
"x-fabro-test".to_string(),
|
||||
"present".to_string(),
|
||||
)]));
|
||||
let response = adapter.complete(&make_request("m")).await.unwrap();
|
||||
|
||||
mock.assert();
|
||||
assert_eq!(response.text(), "ok");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn factory_rejects_custom_auth_header() {
|
||||
let result = build(AdapterConfig {
|
||||
provider_id: "bedrock".to_string(),
|
||||
auth_header: Some(ApiKeyHeader::Custom {
|
||||
name: "x-api-key".to_string(),
|
||||
value: "secret".to_string(),
|
||||
}),
|
||||
base_url: Some("https://bedrock-runtime.us-east-1.amazonaws.com".to_string()),
|
||||
extra_headers: HashMap::new(),
|
||||
kind_options: AdapterKindOptions::None,
|
||||
catalog: None,
|
||||
});
|
||||
|
||||
let Err(err) = result else {
|
||||
panic!("expected custom auth header to be rejected");
|
||||
};
|
||||
assert!(
|
||||
err.to_string()
|
||||
.contains("does not support custom auth header")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_signs_with_sigv4_when_configured() {
|
||||
let server = MockServer::start();
|
||||
let mock = server.mock(|when, then| {
|
||||
when.method(POST)
|
||||
.path("/model/m/converse")
|
||||
.header_exists("authorization")
|
||||
.header_exists("x-amz-date");
|
||||
then.status(200)
|
||||
.header("content-type", "application/json")
|
||||
.json_body(serde_json::json!({
|
||||
"output": {"message": {"role": "assistant", "content": [{"text": "ok"}]}},
|
||||
"stopReason": "end_turn",
|
||||
"usage": {"inputTokens": 1, "outputTokens": 1, "totalTokens": 2}
|
||||
}));
|
||||
});
|
||||
|
||||
let mut adapter = test_adapter(&server);
|
||||
let cell = OnceCell::new();
|
||||
cell.set(Sigv4Signer::from_static("AKIDEXAMPLE", "secret", None))
|
||||
.ok();
|
||||
adapter.auth = BedrockAuth::Sigv4(cell);
|
||||
|
||||
let response = adapter.complete(&make_request("m")).await.unwrap();
|
||||
mock.assert();
|
||||
assert_eq!(response.text(), "ok");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_decodes_eventstream_frames() {
|
||||
let server = MockServer::start();
|
||||
let body = eventstream::tests::build_stream_body(&[
|
||||
("messageStart", r#"{"role":"assistant"}"#),
|
||||
(
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"Hel"},"contentBlockIndex":0}"#,
|
||||
),
|
||||
(
|
||||
"contentBlockDelta",
|
||||
r#"{"delta":{"text":"lo"},"contentBlockIndex":0}"#,
|
||||
),
|
||||
("contentBlockStop", r#"{"contentBlockIndex":0}"#),
|
||||
("messageStop", r#"{"stopReason":"end_turn"}"#),
|
||||
(
|
||||
"metadata",
|
||||
r#"{"usage":{"inputTokens":9,"outputTokens":3,"totalTokens":12}}"#,
|
||||
),
|
||||
]);
|
||||
server.mock(|when, then| {
|
||||
when.method(POST)
|
||||
.path("/model/m/converse-stream")
|
||||
.header("accept", "application/vnd.amazon.eventstream");
|
||||
then.status(200)
|
||||
.header("content-type", "application/vnd.amazon.eventstream")
|
||||
.body(body);
|
||||
});
|
||||
|
||||
let adapter = test_adapter(&server);
|
||||
let mut stream = adapter.stream(&make_request("m")).await.unwrap();
|
||||
|
||||
let mut text = String::new();
|
||||
let mut finish: Option<Response> = None;
|
||||
while let Some(event) = stream.next().await {
|
||||
match event.unwrap() {
|
||||
StreamEvent::TextDelta { delta, .. } => text.push_str(&delta),
|
||||
StreamEvent::Finish { response, .. } => finish = Some(*response),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
assert_eq!(text, "Hello");
|
||||
let response = finish.expect("stream should finish");
|
||||
assert_eq!(response.text(), "Hello");
|
||||
assert_eq!(response.usage.input_tokens, 9);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_surfaces_http_error_before_bytes() {
|
||||
let server = MockServer::start();
|
||||
server.mock(|when, then| {
|
||||
when.method(POST).path("/model/m/converse-stream");
|
||||
then.status(429)
|
||||
.json_body(serde_json::json!({"message": "Too many requests"}));
|
||||
});
|
||||
|
||||
let adapter = test_adapter(&server);
|
||||
let Err(err) = adapter.stream(&make_request("m")).await else {
|
||||
panic!("expected an HTTP error before any stream bytes");
|
||||
};
|
||||
assert_eq!(err.status_code(), Some(429));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_choice_none_is_rejected() {
|
||||
let server = MockServer::start();
|
||||
let adapter = test_adapter(&server);
|
||||
assert!(!adapter.supports_tool_choice("none"));
|
||||
assert!(adapter.supports_tool_choice("auto"));
|
||||
}
|
||||
}
|
||||
249
lib/crates/fabro-llm/src/providers/bedrock/sigv4.rs
Normal file
249
lib/crates/fabro-llm/src/providers/bedrock/sigv4.rs
Normal file
|
|
@ -0,0 +1,249 @@
|
|||
//! AWS Signature Version 4 signing for Bedrock requests.
|
||||
//!
|
||||
//! Wraps the `aws-sigv4` crate to compute the `Authorization`, `x-amz-date`,
|
||||
//! and (for temporary credentials) `x-amz-security-token` headers for a fully
|
||||
//! built request. The headers are then attached to the shared `fabro-http`
|
||||
//! request builder, so signed Bedrock requests still flow through the same
|
||||
//! retry/redaction/transport layers as every other adapter.
|
||||
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
|
||||
use aws_credential_types::Credentials;
|
||||
use aws_credential_types::provider::SharedCredentialsProvider;
|
||||
use aws_sigv4::http_request::{SignableBody, SignableRequest, SigningSettings, sign};
|
||||
use aws_sigv4::sign::v4;
|
||||
use aws_smithy_runtime_api::client::identity::Identity;
|
||||
|
||||
use crate::error::Error;
|
||||
|
||||
/// Service name used in the SigV4 credential scope for Bedrock runtime calls.
|
||||
pub(crate) const SERVICE: &str = "bedrock";
|
||||
|
||||
/// Where the signer's credentials come from.
|
||||
enum CredentialSource {
|
||||
/// Fixed credentials (tests / explicitly supplied keys).
|
||||
#[cfg(test)]
|
||||
Static(Credentials),
|
||||
/// The AWS default provider chain. Credentials are resolved per request
|
||||
/// so expiring session credentials (STS, IRSA, instance roles) refresh
|
||||
/// through the chain's identity cache instead of being snapshotted once
|
||||
/// at startup.
|
||||
Chain(SharedCredentialsProvider),
|
||||
}
|
||||
|
||||
/// Signs HTTP requests for AWS services with SigV4.
|
||||
pub(crate) struct Sigv4Signer {
|
||||
credentials: CredentialSource,
|
||||
}
|
||||
|
||||
impl Sigv4Signer {
|
||||
/// Build a signer from static keys. Test-only: production paths resolve
|
||||
/// credentials through the AWS chain.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn from_static(
|
||||
access_key_id: &str,
|
||||
secret_access_key: &str,
|
||||
session_token: Option<String>,
|
||||
) -> Self {
|
||||
Self {
|
||||
credentials: CredentialSource::Static(Credentials::from_keys(
|
||||
access_key_id,
|
||||
secret_access_key,
|
||||
session_token,
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a signer over the standard AWS provider chain (environment,
|
||||
/// IRSA/web identity, EC2/ECS instance profile, SSO, assume-role). The
|
||||
/// chain is resolved once; the credentials it yields are fetched per
|
||||
/// signing call so they stay fresh over long-lived adapters.
|
||||
pub(crate) async fn from_default_chain() -> Result<Self, Error> {
|
||||
let config = aws_config::defaults(aws_config::BehaviorVersion::latest())
|
||||
.load()
|
||||
.await;
|
||||
let provider = config
|
||||
.credentials_provider()
|
||||
.ok_or_else(|| Error::Configuration {
|
||||
message: "no AWS credentials provider found in the default chain".to_string(),
|
||||
source: None,
|
||||
})?;
|
||||
Ok(Self {
|
||||
credentials: CredentialSource::Chain(provider),
|
||||
})
|
||||
}
|
||||
|
||||
/// The credentials to sign the next request with.
|
||||
async fn current_credentials(&self) -> Result<Credentials, Error> {
|
||||
use aws_credential_types::provider::ProvideCredentials;
|
||||
|
||||
match &self.credentials {
|
||||
#[cfg(test)]
|
||||
CredentialSource::Static(credentials) => Ok(credentials.clone()),
|
||||
CredentialSource::Chain(provider) => {
|
||||
provider
|
||||
.provide_credentials()
|
||||
.await
|
||||
.map_err(|e| Error::Configuration {
|
||||
message: format!("failed to resolve AWS credentials: {e}"),
|
||||
source: None,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute the SigV4 headers for a request: `Authorization`, `x-amz-date`,
|
||||
/// and `x-amz-security-token` when the credentials carry a session token.
|
||||
fn signed_headers(
|
||||
credentials: &Credentials,
|
||||
region: &str,
|
||||
service: &str,
|
||||
method: &str,
|
||||
url: &str,
|
||||
body: &[u8],
|
||||
epoch_secs: u64,
|
||||
) -> Result<Vec<(String, String)>, Error> {
|
||||
let identity: Identity = credentials.clone().into();
|
||||
let signing_params = v4::SigningParams::builder()
|
||||
.identity(&identity)
|
||||
.region(region)
|
||||
.name(service)
|
||||
.time(UNIX_EPOCH + Duration::from_secs(epoch_secs))
|
||||
.settings(SigningSettings::default())
|
||||
.build()
|
||||
.map_err(|e| Error::Configuration {
|
||||
message: format!("sigv4 params: {e}"),
|
||||
source: None,
|
||||
})?
|
||||
.into();
|
||||
|
||||
let signable =
|
||||
SignableRequest::new(method, url, std::iter::empty(), SignableBody::Bytes(body))
|
||||
.map_err(|e| Error::Configuration {
|
||||
message: format!("sigv4 signable request: {e}"),
|
||||
source: None,
|
||||
})?;
|
||||
|
||||
let (instructions, _signature) = sign(signable, &signing_params)
|
||||
.map_err(|e| Error::Configuration {
|
||||
message: format!("sigv4 signing failed: {e}"),
|
||||
source: None,
|
||||
})?
|
||||
.into_parts();
|
||||
|
||||
Ok(instructions
|
||||
.headers()
|
||||
.map(|(name, value)| (name.to_string(), value.to_string()))
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Apply SigV4 signed headers to a `fabro-http` request builder for a
|
||||
/// `POST` to `url` carrying `body`.
|
||||
pub(crate) async fn sign_post(
|
||||
&self,
|
||||
mut req: fabro_http::RequestBuilder,
|
||||
region: &str,
|
||||
url: &str,
|
||||
body: Vec<u8>,
|
||||
) -> Result<fabro_http::RequestBuilder, Error> {
|
||||
let credentials = self.current_credentials().await?;
|
||||
let now = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map_err(|e| Error::Configuration {
|
||||
message: format!("system clock before epoch: {e}"),
|
||||
source: None,
|
||||
})?
|
||||
.as_secs();
|
||||
for (name, value) in
|
||||
Self::signed_headers(&credentials, region, SERVICE, "POST", url, &body, now)?
|
||||
{
|
||||
req = req.header(name, value);
|
||||
}
|
||||
Ok(req.body(body))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
// Fixed credentials + time produce a deterministic Authorization header.
|
||||
// The expected value is locked below after the first green run so the test
|
||||
// guards against accidental changes to the signing logic.
|
||||
const ACCESS_KEY: &str = "AKIDEXAMPLE";
|
||||
const SECRET_KEY: &str = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";
|
||||
const FIXED_EPOCH: u64 = 1_716_960_000; // 2024-05-29T04:00:00Z
|
||||
const URL: &str = "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-sonnet-4-6/converse";
|
||||
|
||||
fn static_credentials(signer: &Sigv4Signer) -> Credentials {
|
||||
match &signer.credentials {
|
||||
CredentialSource::Static(credentials) => credentials.clone(),
|
||||
CredentialSource::Chain(_) => panic!("test signer should hold static credentials"),
|
||||
}
|
||||
}
|
||||
|
||||
fn auth_header(headers: &[(String, String)]) -> &str {
|
||||
headers
|
||||
.iter()
|
||||
.find(|(name, _)| name.eq_ignore_ascii_case("authorization"))
|
||||
.map(|(_, value)| value.as_str())
|
||||
.expect("authorization header must be present")
|
||||
}
|
||||
|
||||
fn sign_fixed(signer: &Sigv4Signer, body: &[u8]) -> Vec<(String, String)> {
|
||||
Sigv4Signer::signed_headers(
|
||||
&static_credentials(signer),
|
||||
"us-east-1",
|
||||
SERVICE,
|
||||
"POST",
|
||||
URL,
|
||||
body,
|
||||
FIXED_EPOCH,
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn produces_authorization_and_date_headers() {
|
||||
let signer = Sigv4Signer::from_static(ACCESS_KEY, SECRET_KEY, None);
|
||||
let headers = sign_fixed(&signer, br#"{"messages":[]}"#);
|
||||
|
||||
assert!(
|
||||
headers
|
||||
.iter()
|
||||
.any(|(n, _)| n.eq_ignore_ascii_case("authorization"))
|
||||
);
|
||||
assert!(
|
||||
headers
|
||||
.iter()
|
||||
.any(|(n, _)| n.eq_ignore_ascii_case("x-amz-date"))
|
||||
);
|
||||
let auth = auth_header(&headers);
|
||||
assert!(auth.starts_with("AWS4-HMAC-SHA256 "));
|
||||
assert!(auth.contains("Credential=AKIDEXAMPLE/20240529/us-east-1/bedrock/aws4_request"));
|
||||
assert!(auth.contains("SignedHeaders="));
|
||||
assert!(auth.contains("Signature="));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deterministic_signature_is_stable() {
|
||||
let signer = Sigv4Signer::from_static(ACCESS_KEY, SECRET_KEY, None);
|
||||
// Same inputs must yield an identical signature (regression lock).
|
||||
assert_eq!(
|
||||
auth_header(&sign_fixed(&signer, br#"{"messages":[]}"#)),
|
||||
auth_header(&sign_fixed(&signer, br#"{"messages":[]}"#)),
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn session_token_adds_security_token_header() {
|
||||
let signer =
|
||||
Sigv4Signer::from_static(ACCESS_KEY, SECRET_KEY, Some("session-tok".to_string()));
|
||||
let headers = sign_fixed(&signer, b"{}");
|
||||
assert!(
|
||||
headers
|
||||
.iter()
|
||||
.any(|(n, v)| n.eq_ignore_ascii_case("x-amz-security-token") && v == "session-tok")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
pub mod anthropic;
|
||||
pub(crate) mod bedrock;
|
||||
pub mod common;
|
||||
pub mod fabro_server;
|
||||
pub mod gemini;
|
||||
|
|
@ -6,6 +7,7 @@ pub mod openai;
|
|||
pub mod openai_compatible;
|
||||
|
||||
pub use anthropic::Adapter as AnthropicAdapter;
|
||||
pub use bedrock::Adapter as BedrockAdapter;
|
||||
pub use fabro_server::Adapter as FabroServerAdapter;
|
||||
pub use gemini::Adapter as GeminiAdapter;
|
||||
pub use openai::Adapter as OpenAiAdapter;
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ use std::sync::Arc;
|
|||
use fabro_llm::error::ProviderErrorKind;
|
||||
use fabro_llm::provider::ProviderAdapter;
|
||||
use fabro_llm::providers::{
|
||||
AnthropicAdapter, GeminiAdapter, OpenAiAdapter, OpenAiCompatibleAdapter,
|
||||
AnthropicAdapter, BedrockAdapter, GeminiAdapter, OpenAiAdapter, OpenAiCompatibleAdapter,
|
||||
};
|
||||
use fabro_llm::types::{CostSource, FinishReason, Message, Request};
|
||||
use fabro_model::Catalog;
|
||||
|
|
@ -176,6 +176,69 @@ async fn gemini_complete() {
|
|||
assert_eq!(response.provider, "gemini");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_BEARER_TOKEN_BEDROCK"))]
|
||||
async fn bedrock_complete_with_api_key() {
|
||||
let token = std::env::var(EnvVars::AWS_BEARER_TOKEN_BEDROCK)
|
||||
.expect("AWS_BEARER_TOKEN_BEDROCK must be set");
|
||||
let adapter =
|
||||
BedrockAdapter::new_api_key(token, "https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
.unwrap()
|
||||
.with_name("bedrock");
|
||||
// Amazon Nova: first-party, no Anthropic-approval gate and no third-party
|
||||
// marketplace subscription, so this runs on any Bedrock-enabled account.
|
||||
let request = make_request("us.amazon.nova-2-lite-v1:0");
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert!(response.usage.output_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_ACCESS_KEY_ID"))]
|
||||
async fn bedrock_complete_with_sigv4() {
|
||||
let adapter = BedrockAdapter::new_sigv4("https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
.unwrap()
|
||||
.with_name("bedrock");
|
||||
// First-party Nova — see bedrock_complete_with_api_key for why.
|
||||
let request = make_request("us.amazon.nova-2-lite-v1:0");
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("AWS_BEARER_TOKEN_BEDROCK"))]
|
||||
async fn bedrock_openai_frontier_complete() {
|
||||
let token = std::env::var(EnvVars::AWS_BEARER_TOKEN_BEDROCK)
|
||||
.expect("AWS_BEARER_TOKEN_BEDROCK must be set");
|
||||
// GPT-5.x on Bedrock is the bedrock-mantle Responses surface: the plain
|
||||
// openai adapter pointed at the mantle endpoint with the Bedrock key as
|
||||
// the bearer token.
|
||||
let adapter = OpenAiAdapter::new(token)
|
||||
.with_base_url("https://bedrock-mantle.us-east-1.api.aws/openai/v1")
|
||||
.with_name("bedrock-openai");
|
||||
let request = Request {
|
||||
temperature: None,
|
||||
..make_request("openai.gpt-5.5")
|
||||
};
|
||||
let response = adapter.complete(&request).await.unwrap();
|
||||
|
||||
assert!(
|
||||
!response.text().is_empty(),
|
||||
"response text should not be empty"
|
||||
);
|
||||
assert!(response.usage.input_tokens > 0);
|
||||
assert_eq!(response.provider, "bedrock-openai");
|
||||
}
|
||||
|
||||
#[fabro_macros::e2e_test(live("OPENROUTER_API_KEY"))]
|
||||
async fn openrouter_complete() {
|
||||
let api_key =
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ pub enum AdapterKind {
|
|||
#[serde(rename = "openai_compatible")]
|
||||
#[strum(to_string = "openai_compatible")]
|
||||
OpenAiCompatible,
|
||||
Bedrock,
|
||||
}
|
||||
|
||||
impl AdapterKind {
|
||||
|
|
@ -87,6 +88,16 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bedrock_adapter_kind_roundtrips() {
|
||||
assert_eq!(AdapterKind::Bedrock.as_str(), "bedrock");
|
||||
assert_eq!(
|
||||
"bedrock".parse::<AdapterKind>().unwrap(),
|
||||
AdapterKind::Bedrock
|
||||
);
|
||||
assert!(AdapterKind::VARIANTS.contains(&AdapterKind::Bedrock));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn agent_profile_kind_round_trips_as_settings_strings() {
|
||||
for (kind, expected) in [
|
||||
|
|
|
|||
|
|
@ -171,11 +171,19 @@ pub struct CostRates {
|
|||
pub cache_input_cost_per_mtok: Option<f64>,
|
||||
}
|
||||
|
||||
/// Where a provider's credential comes from.
|
||||
///
|
||||
/// `Vault`/`Env` reference a stored secret resolved to an auth header.
|
||||
/// `AwsSigv4` is an opaque source: the credential comes from the AWS default
|
||||
/// credential chain and the request is SigV4-signed rather than carrying a
|
||||
/// static secret. It is only valid on Bedrock providers, which catalog
|
||||
/// validation enforces before adapter construction.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(into = "String", try_from = "String")]
|
||||
pub enum CredentialRef {
|
||||
Vault(String),
|
||||
Env(String),
|
||||
AwsSigv4,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for CredentialRef {
|
||||
|
|
@ -183,6 +191,7 @@ impl std::fmt::Display for CredentialRef {
|
|||
match self {
|
||||
Self::Vault(name) => write!(f, "vault:{name}"),
|
||||
Self::Env(name) => write!(f, "env:{name}"),
|
||||
Self::AwsSigv4 => write!(f, "aws_sigv4"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -209,6 +218,9 @@ impl FromStr for CredentialRef {
|
|||
}
|
||||
return Ok(Self::Env(name.to_string()));
|
||||
}
|
||||
if value == "aws_sigv4" {
|
||||
return Ok(Self::AwsSigv4);
|
||||
}
|
||||
Err(CredentialRefParseError::Invalid)
|
||||
}
|
||||
}
|
||||
|
|
@ -223,7 +235,7 @@ impl TryFrom<String> for CredentialRef {
|
|||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum CredentialRefParseError {
|
||||
#[error("credential reference must be `vault:<name>` or `env:<NAME>`")]
|
||||
#[error("credential reference must be `vault:<name>`, `env:<NAME>`, or `aws_sigv4`")]
|
||||
Invalid,
|
||||
#[error("credential reference is missing a name after `vault:`")]
|
||||
EmptyVault,
|
||||
|
|
@ -234,6 +246,9 @@ pub enum CredentialRefParseError {
|
|||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct ProviderAuthConfig {
|
||||
/// Ordered credential sources; the first that resolves wins. Static secrets
|
||||
/// use `env:<NAME>` / `vault:<NAME>`; AWS SigV4 (Bedrock) uses `aws_sigv4`,
|
||||
/// which resolves opaquely from the AWS credential chain.
|
||||
pub credentials: Vec<CredentialRef>,
|
||||
#[serde(default)]
|
||||
pub header: ApiKeyHeaderPolicy,
|
||||
|
|
@ -511,7 +526,7 @@ impl CatalogProvider {
|
|||
.iter()
|
||||
.find_map(|credential_ref| match credential_ref {
|
||||
CredentialRef::Vault(name) => Some(name.as_str()),
|
||||
CredentialRef::Env(_) => None,
|
||||
CredentialRef::Env(_) | CredentialRef::AwsSigv4 => None,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -600,6 +615,13 @@ pub enum CatalogBuildError {
|
|||
},
|
||||
#[error("provider '{provider}' API-key auth must declare at least one credential")]
|
||||
EmptyApiKeyCredentials { provider: ProviderId },
|
||||
#[error(
|
||||
"provider '{provider}' uses aws_sigv4 credentials, but adapter '{adapter}' does not support SigV4"
|
||||
)]
|
||||
UnsupportedAwsSigv4Credential {
|
||||
provider: ProviderId,
|
||||
adapter: AdapterKind,
|
||||
},
|
||||
#[error("provider identifier '{identifier}' is declared by both '{first}' and '{second}'")]
|
||||
DuplicateProviderIdentifier {
|
||||
identifier: String,
|
||||
|
|
@ -1339,7 +1361,7 @@ fn build_providers(
|
|||
let codec = resolve_provider_codec(&provider_id, adapter, settings.codec)?;
|
||||
let agent_profile = settings.agent_profile.unwrap_or(defaults.agent_profile);
|
||||
let auth = settings.auth.clone();
|
||||
validate_provider_auth(&provider_id, auth.as_ref())?;
|
||||
validate_provider_auth(&provider_id, adapter, auth.as_ref())?;
|
||||
|
||||
providers.push(CatalogProvider {
|
||||
id: provider_id,
|
||||
|
|
@ -1367,7 +1389,9 @@ struct AdapterDefaults {
|
|||
|
||||
fn adapter_defaults(adapter: AdapterKind) -> AdapterDefaults {
|
||||
match adapter {
|
||||
AdapterKind::Anthropic => AdapterDefaults {
|
||||
// Bedrock hosts Anthropic-family models, so it shares the Anthropic
|
||||
// agent profile and billing policy by default.
|
||||
AdapterKind::Anthropic | AdapterKind::Bedrock => AdapterDefaults {
|
||||
agent_profile: AgentProfileKind::Anthropic,
|
||||
billing_policy: BillingPolicy::Anthropic,
|
||||
},
|
||||
|
|
@ -1423,6 +1447,7 @@ fn resolve_model_codec(
|
|||
|
||||
fn validate_provider_auth(
|
||||
provider: &ProviderId,
|
||||
adapter: AdapterKind,
|
||||
auth: Option<&ProviderAuthConfig>,
|
||||
) -> Result<(), CatalogBuildError> {
|
||||
match auth {
|
||||
|
|
@ -1431,6 +1456,18 @@ fn validate_provider_auth(
|
|||
provider: provider.clone(),
|
||||
})
|
||||
}
|
||||
Some(auth)
|
||||
if adapter != AdapterKind::Bedrock
|
||||
&& auth
|
||||
.credentials
|
||||
.iter()
|
||||
.any(|credential| matches!(credential, CredentialRef::AwsSigv4)) =>
|
||||
{
|
||||
Err(CatalogBuildError::UnsupportedAwsSigv4Credential {
|
||||
provider: provider.clone(),
|
||||
adapter,
|
||||
})
|
||||
}
|
||||
_ => Ok(()),
|
||||
}
|
||||
}
|
||||
|
|
@ -1832,6 +1869,56 @@ mod tests {
|
|||
toml::from_str(source).expect("fixture should parse as an LLM settings layer")
|
||||
}
|
||||
|
||||
const BEDROCK_SIGV4_LAYER: &str = r#"
|
||||
[providers.bedrock]
|
||||
adapter = "bedrock"
|
||||
base_url = "https://bedrock-runtime.eu-west-1.amazonaws.com"
|
||||
|
||||
[providers.bedrock.auth]
|
||||
credentials = ["aws_sigv4"]
|
||||
|
||||
[models."bedrock-sonnet"]
|
||||
provider = "bedrock"
|
||||
api_id = "anthropic.claude-sonnet-4-6"
|
||||
display_name = "Bedrock Sonnet"
|
||||
family = "claude-4"
|
||||
default = true
|
||||
|
||||
[models."bedrock-sonnet".limits]
|
||||
context_window = 200000
|
||||
max_output = 64000
|
||||
|
||||
[models."bedrock-sonnet".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn provider_parses_bedrock_base_url_and_sigv4_credential() {
|
||||
let catalog = Catalog::from_settings(&minimal_settings(BEDROCK_SIGV4_LAYER)).unwrap();
|
||||
let provider = catalog.provider(&ProviderId::from("bedrock")).unwrap();
|
||||
assert_eq!(
|
||||
provider.base_url.as_deref(),
|
||||
Some("https://bedrock-runtime.eu-west-1.amazonaws.com")
|
||||
);
|
||||
assert_eq!(provider.auth.as_ref().unwrap().credentials, vec![
|
||||
CredentialRef::AwsSigv4
|
||||
]);
|
||||
// Bedrock inherits the Anthropic agent profile and billing by default.
|
||||
assert_eq!(provider.agent_profile, AgentProfileKind::Anthropic);
|
||||
assert_eq!(provider.billing_policy, BillingPolicy::Anthropic);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn aws_sigv4_credential_round_trips() {
|
||||
assert_eq!(
|
||||
"aws_sigv4".parse::<CredentialRef>().unwrap(),
|
||||
CredentialRef::AwsSigv4
|
||||
);
|
||||
assert_eq!(CredentialRef::AwsSigv4.to_string(), "aws_sigv4");
|
||||
}
|
||||
|
||||
// ---- Catalog struct tests ----
|
||||
|
||||
#[test]
|
||||
|
|
@ -1914,6 +2001,117 @@ reasoning = false
|
|||
assert_eq!(model.provider, ProviderId::new("acme"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_bedrock_provider_is_opt_in() {
|
||||
let bedrock = ProviderId::new("bedrock");
|
||||
let builtin = Catalog::builtin();
|
||||
|
||||
assert!(builtin.provider(&bedrock).is_none());
|
||||
assert!(builtin.list(Some(&bedrock)).is_empty());
|
||||
|
||||
let catalog = Catalog::from_builtin_with_overrides(&minimal_settings(
|
||||
r"
|
||||
[providers.bedrock]
|
||||
enabled = true
|
||||
",
|
||||
))
|
||||
.expect("enabled Bedrock override should build from the built-in provider settings");
|
||||
|
||||
let provider = catalog
|
||||
.provider(&bedrock)
|
||||
.expect("enabled Bedrock provider should be present");
|
||||
assert_eq!(provider.adapter, AdapterKind::Bedrock);
|
||||
assert_eq!(provider.codec, CodecKind::BedrockConverse);
|
||||
assert_eq!(
|
||||
provider.base_url.as_deref(),
|
||||
Some("https://bedrock-runtime.us-east-1.amazonaws.com")
|
||||
);
|
||||
// Bearer key first (env then vault, like every other provider), under
|
||||
// either the AWS-canonical name or Fabro's `<PROVIDER>_API_KEY`
|
||||
// convention; SigV4 chain as the fallback.
|
||||
assert_eq!(provider.auth.as_ref().unwrap().credentials, vec![
|
||||
CredentialRef::Env("AWS_BEARER_TOKEN_BEDROCK".to_string()),
|
||||
CredentialRef::Env("BEDROCK_API_KEY".to_string()),
|
||||
CredentialRef::Vault("AWS_BEARER_TOKEN_BEDROCK".to_string()),
|
||||
CredentialRef::Vault("BEDROCK_API_KEY".to_string()),
|
||||
CredentialRef::AwsSigv4,
|
||||
]);
|
||||
|
||||
// Claude rows bill Anthropic-style; open-weights rows override the
|
||||
// provider's Anthropic defaults the other way.
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-sonnet-4-6")
|
||||
.unwrap()
|
||||
.billing_policy,
|
||||
BillingPolicy::Anthropic
|
||||
);
|
||||
assert_eq!(
|
||||
catalog.model_settings("zai.glm-5").unwrap().billing_policy,
|
||||
BillingPolicy::OpenAi
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-haiku-4-5")
|
||||
.unwrap()
|
||||
.api_id,
|
||||
"us.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.default_for_provider(&bedrock)
|
||||
.map(|model| model.id.as_str()),
|
||||
Some("us.anthropic.claude-sonnet-4-6")
|
||||
);
|
||||
// Fable 5 ships with sampling params pinned off (the Converse
|
||||
// encoder drops temperature/top_p for it).
|
||||
let fable = catalog
|
||||
.get("us.anthropic.claude-fable-5")
|
||||
.expect("fable row should be present");
|
||||
assert!(!fable.features.sampling_params);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.model_settings("us.anthropic.claude-fable-5")
|
||||
.unwrap()
|
||||
.billing_policy,
|
||||
BillingPolicy::Anthropic
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_bedrock_openai_provider_is_opt_in() {
|
||||
let provider_id = ProviderId::new("bedrock-openai");
|
||||
let builtin = Catalog::builtin();
|
||||
|
||||
assert!(builtin.provider(&provider_id).is_none());
|
||||
|
||||
let catalog = Catalog::from_builtin_with_overrides(&minimal_settings(
|
||||
r"
|
||||
[providers.bedrock-openai]
|
||||
enabled = true
|
||||
",
|
||||
))
|
||||
.expect("enabled bedrock-openai override should build");
|
||||
|
||||
let provider = catalog
|
||||
.provider(&provider_id)
|
||||
.expect("enabled bedrock-openai provider should be present");
|
||||
// OpenAI frontier on Bedrock rides the existing openai_responses
|
||||
// dialect against the bedrock-mantle endpoint — pure configuration.
|
||||
assert_eq!(provider.adapter, AdapterKind::OpenAi);
|
||||
assert_eq!(provider.codec, CodecKind::OpenAiResponses);
|
||||
assert_eq!(
|
||||
provider.base_url.as_deref(),
|
||||
Some("https://bedrock-mantle.us-east-1.api.aws/openai/v1")
|
||||
);
|
||||
assert_eq!(
|
||||
catalog
|
||||
.default_for_provider(&provider_id)
|
||||
.map(|model| model.id.as_str()),
|
||||
Some("openai.gpt-5.5")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_openrouter_provider_is_opt_in() {
|
||||
let openrouter = ProviderId::new("openrouter");
|
||||
|
|
@ -3627,6 +3825,23 @@ credentials = []
|
|||
CatalogBuildError::EmptyApiKeyCredentials { provider }
|
||||
if provider == ProviderId::new("test")
|
||||
));
|
||||
|
||||
let sigv4_on_openai = minimal_settings(
|
||||
r#"
|
||||
[providers.test]
|
||||
display_name = "Test"
|
||||
adapter = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[providers.test.auth]
|
||||
credentials = ["aws_sigv4"]
|
||||
"#,
|
||||
);
|
||||
assert!(matches!(
|
||||
Catalog::from_settings(&sigv4_on_openai).unwrap_err(),
|
||||
CatalogBuildError::UnsupportedAwsSigv4Credential { provider, adapter }
|
||||
if provider == ProviderId::new("test") && adapter == AdapterKind::OpenAi
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,75 @@
|
|||
[providers.bedrock-openai]
|
||||
display_name = "Amazon Bedrock (OpenAI frontier)"
|
||||
adapter = "openai"
|
||||
api_key_url = "https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html"
|
||||
base_url = "https://bedrock-mantle.us-east-1.api.aws/openai/v1"
|
||||
priority = 19
|
||||
enabled = false
|
||||
|
||||
[providers.bedrock-openai.auth]
|
||||
credentials = [
|
||||
"env:AWS_BEARER_TOKEN_BEDROCK",
|
||||
"env:BEDROCK_API_KEY",
|
||||
"vault:AWS_BEARER_TOKEN_BEDROCK",
|
||||
"vault:BEDROCK_API_KEY",
|
||||
]
|
||||
|
||||
# OpenAI's frontier models on Bedrock (GPT-5.5/5.4) are served ONLY by the
|
||||
# bedrock-mantle endpoint's OpenAI Responses API — they are not reachable
|
||||
# through Converse or InvokeModel on bedrock-runtime. That surface speaks
|
||||
# the openai_responses dialect with a Bedrock API key as the bearer token,
|
||||
# so this companion provider row is pure configuration over the existing
|
||||
# openai adapter: same AWS account and key as the `bedrock` provider, a
|
||||
# different endpoint and wire dialect.
|
||||
#
|
||||
# Notes:
|
||||
# - Auth is Bedrock-API-key only on this row (SigV4 on mantle uses the
|
||||
# `bedrock-mantle` signing name, which the openai adapter does not do).
|
||||
# - bedrock-mantle is regional (13 regions); change base_url to
|
||||
# `https://bedrock-mantle.<region>.api.aws/openai/v1` as needed.
|
||||
# - Responses state: Fabro always sends `store: false`, so nothing is
|
||||
# retained under mantle's default 30-day Project retention.
|
||||
#
|
||||
# To enable, add to ~/.fabro/settings.toml:
|
||||
#
|
||||
# [llm.providers.bedrock-openai]
|
||||
# enabled = true
|
||||
|
||||
[models."openai.gpt-5.5"]
|
||||
provider = "bedrock-openai"
|
||||
display_name = "GPT-5.5 (Bedrock)"
|
||||
family = "gpt-5"
|
||||
default = true
|
||||
|
||||
[models."openai.gpt-5.5".limits]
|
||||
context_window = 272000
|
||||
max_output = 128000
|
||||
|
||||
[models."openai.gpt-5.5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
reasoning_effort = "levels"
|
||||
|
||||
[models."openai.gpt-5.5".costs]
|
||||
input_cost_per_mtok = 5.5
|
||||
output_cost_per_mtok = 33.0
|
||||
|
||||
[models."openai.gpt-5.4"]
|
||||
provider = "bedrock-openai"
|
||||
display_name = "GPT-5.4 (Bedrock)"
|
||||
family = "gpt-5"
|
||||
|
||||
[models."openai.gpt-5.4".limits]
|
||||
context_window = 272000
|
||||
max_output = 128000
|
||||
|
||||
[models."openai.gpt-5.4".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
reasoning_effort = "levels"
|
||||
|
||||
[models."openai.gpt-5.4".costs]
|
||||
input_cost_per_mtok = 2.75
|
||||
output_cost_per_mtok = 16.5
|
||||
388
lib/crates/fabro-model/src/catalog/providers/bedrock.toml
Normal file
388
lib/crates/fabro-model/src/catalog/providers/bedrock.toml
Normal file
|
|
@ -0,0 +1,388 @@
|
|||
[providers.bedrock]
|
||||
display_name = "Amazon Bedrock"
|
||||
adapter = "bedrock"
|
||||
api_key_url = "https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html"
|
||||
base_url = "https://bedrock-runtime.us-east-1.amazonaws.com"
|
||||
priority = 20
|
||||
enabled = false
|
||||
|
||||
[providers.bedrock.auth]
|
||||
# An explicit Bedrock API key wins (from the process env, or the server
|
||||
# vault via `fabro secret set <name>`, matching every other provider's
|
||||
# env-then-vault order); SigV4 (the AWS default credential chain, resolved
|
||||
# at request time) is the fallback. `aws_sigv4` always resolves, which is
|
||||
# why this provider ships disabled: enabling it is the operator's statement
|
||||
# that AWS credentials are expected to work. The key is read from either
|
||||
# `AWS_BEARER_TOKEN_BEDROCK` (the AWS-canonical name, also honored by the
|
||||
# AWS SDKs/CLI) or `BEDROCK_API_KEY` (Fabro's `<PROVIDER>_API_KEY`
|
||||
# convention); the env names are checked before the vault.
|
||||
credentials = [
|
||||
"env:AWS_BEARER_TOKEN_BEDROCK",
|
||||
"env:BEDROCK_API_KEY",
|
||||
"vault:AWS_BEARER_TOKEN_BEDROCK",
|
||||
"vault:BEDROCK_API_KEY",
|
||||
"aws_sigv4",
|
||||
]
|
||||
|
||||
# To enable Bedrock, add the following to ~/.fabro/settings.toml:
|
||||
#
|
||||
# [llm.providers.bedrock]
|
||||
# enabled = true
|
||||
# base_url = "https://bedrock-runtime.<your-region>.amazonaws.com"
|
||||
#
|
||||
# The signing region is derived from the base_url. Authenticate with
|
||||
# either a Bedrock API key (AWS_BEARER_TOKEN_BEDROCK or BEDROCK_API_KEY) or
|
||||
# any AWS default credential chain source (env keys, profile, IMDS, IRSA, SSO).
|
||||
#
|
||||
# Model ids use cross-region inference profiles (`us.` / `global.`
|
||||
# prefixes) where on-demand access requires them. Pricing rows are
|
||||
# best-effort estimates from June 2026 list prices.
|
||||
|
||||
# ---------- Anthropic Claude ----------
|
||||
#
|
||||
# Claude bills Anthropic-style cache reads/writes, so these rows override
|
||||
# the provider's billing default. Claude Fable 5 appears at the end of this
|
||||
# file because its Bedrock deployment pins sampling parameters and requires an
|
||||
# extra data-sharing opt-in.
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Sonnet 4.6 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
default = true
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".limits]
|
||||
context_window = 1000000
|
||||
max_output = 64000
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-sonnet-4-6".costs]
|
||||
input_cost_per_mtok = 3.0
|
||||
output_cost_per_mtok = 15.0
|
||||
cache_input_cost_per_mtok = 0.3
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Opus 4.8 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".limits]
|
||||
context_window = 1000000
|
||||
max_output = 128000
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-opus-4-8".costs]
|
||||
input_cost_per_mtok = 5.0
|
||||
output_cost_per_mtok = 25.0
|
||||
cache_input_cost_per_mtok = 0.5
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5"]
|
||||
provider = "bedrock"
|
||||
api_id = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
display_name = "Claude Haiku 4.5 (Bedrock)"
|
||||
family = "claude-4"
|
||||
billing_policy = "anthropic"
|
||||
small_default = true
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".limits]
|
||||
context_window = 200000
|
||||
max_output = 64000
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
prompt_cache = true
|
||||
|
||||
[models."us.anthropic.claude-haiku-4-5".costs]
|
||||
input_cost_per_mtok = 1.0
|
||||
output_cost_per_mtok = 5.0
|
||||
cache_input_cost_per_mtok = 0.1
|
||||
|
||||
# ---------- OpenAI open-weights ----------
|
||||
#
|
||||
# GPT-5.5/5.4 are NOT here: on Bedrock they are Responses-API-only on the
|
||||
# bedrock-mantle endpoint (no Converse), a named follow-up route.
|
||||
|
||||
[models."openai.gpt-oss-120b"]
|
||||
provider = "bedrock"
|
||||
api_id = "openai.gpt-oss-120b-1:0"
|
||||
display_name = "GPT-OSS 120B (Bedrock)"
|
||||
family = "gpt-oss"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."openai.gpt-oss-120b".limits]
|
||||
context_window = 128000
|
||||
max_output = 16384
|
||||
|
||||
[models."openai.gpt-oss-120b".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."openai.gpt-oss-120b".costs]
|
||||
input_cost_per_mtok = 0.15
|
||||
output_cost_per_mtok = 0.60
|
||||
|
||||
[models."openai.gpt-oss-20b"]
|
||||
provider = "bedrock"
|
||||
api_id = "openai.gpt-oss-20b-1:0"
|
||||
display_name = "GPT-OSS 20B (Bedrock)"
|
||||
family = "gpt-oss"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."openai.gpt-oss-20b".limits]
|
||||
context_window = 128000
|
||||
max_output = 16384
|
||||
|
||||
[models."openai.gpt-oss-20b".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."openai.gpt-oss-20b".costs]
|
||||
input_cost_per_mtok = 0.07
|
||||
output_cost_per_mtok = 0.30
|
||||
|
||||
# ---------- Amazon Nova ----------
|
||||
|
||||
[models."amazon.nova-2-lite"]
|
||||
provider = "bedrock"
|
||||
api_id = "global.amazon.nova-2-lite-v1:0"
|
||||
display_name = "Nova 2 Lite (Bedrock)"
|
||||
family = "nova-2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."amazon.nova-2-lite".limits]
|
||||
context_window = 1000000
|
||||
# Bedrock caps Nova output at 65535 (2^16 - 1); 65536 trips
|
||||
# "maximum tokens exceeds the model limit of 65535" since the prompt handler
|
||||
# defaults max_tokens to max_output.
|
||||
max_output = 65535
|
||||
|
||||
[models."amazon.nova-2-lite".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."amazon.nova-2-lite".costs]
|
||||
input_cost_per_mtok = 0.30
|
||||
output_cost_per_mtok = 2.50
|
||||
|
||||
# ---------- Open-weights ----------
|
||||
|
||||
[models."meta.llama4-maverick"]
|
||||
provider = "bedrock"
|
||||
api_id = "us.meta.llama4-maverick-17b-instruct-v1:0"
|
||||
display_name = "Llama 4 Maverick (Bedrock)"
|
||||
family = "llama-4"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."meta.llama4-maverick".limits]
|
||||
context_window = 1000000
|
||||
max_output = 8192
|
||||
|
||||
[models."meta.llama4-maverick".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."mistral.mistral-large-3"]
|
||||
provider = "bedrock"
|
||||
api_id = "mistral.mistral-large-3-675b-instruct"
|
||||
display_name = "Mistral Large 3 (Bedrock)"
|
||||
family = "mistral-large"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."mistral.mistral-large-3".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."mistral.mistral-large-3".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."mistral.mistral-large-3".costs]
|
||||
input_cost_per_mtok = 0.50
|
||||
output_cost_per_mtok = 1.50
|
||||
|
||||
[models."mistral.devstral-2"]
|
||||
provider = "bedrock"
|
||||
api_id = "mistral.devstral-2-123b"
|
||||
display_name = "Devstral 2 (Bedrock)"
|
||||
family = "devstral"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."mistral.devstral-2".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."mistral.devstral-2".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."deepseek.v3-2"]
|
||||
provider = "bedrock"
|
||||
api_id = "deepseek.v3.2"
|
||||
display_name = "DeepSeek V3.2 (Bedrock)"
|
||||
family = "deepseek-v3"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."deepseek.v3-2".limits]
|
||||
context_window = 164000
|
||||
max_output = 8192
|
||||
|
||||
[models."deepseek.v3-2".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = true
|
||||
|
||||
[models."deepseek.v3-2".costs]
|
||||
input_cost_per_mtok = 0.62
|
||||
output_cost_per_mtok = 1.85
|
||||
|
||||
# Qwen3 Coder Next: omitted pending a verified Bedrock model/inference-profile
|
||||
# id. The fabro id is not itself a valid Bedrock identifier (Converse returns
|
||||
# "The provided model identifier is invalid"), so this row needs an explicit
|
||||
# `api_id` confirmed against `aws bedrock list-inference-profiles` before it
|
||||
# ships. Re-add with:
|
||||
# [models."qwen.qwen3-coder-next"]
|
||||
# provider = "bedrock"
|
||||
# api_id = "<verified bedrock id>"
|
||||
# display_name = "Qwen3 Coder Next (Bedrock)"
|
||||
# family = "qwen3"
|
||||
# billing_policy = "openai"
|
||||
# agent_profile = "openai"
|
||||
# [models."qwen.qwen3-coder-next".limits]
|
||||
# context_window = 256000
|
||||
# max_output = 16384
|
||||
# [models."qwen.qwen3-coder-next".features]
|
||||
# tools = true
|
||||
|
||||
[models."moonshotai.kimi-k2.5"]
|
||||
provider = "bedrock"
|
||||
display_name = "Kimi K2.5 (Bedrock)"
|
||||
family = "kimi-k2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."moonshotai.kimi-k2.5".limits]
|
||||
context_window = 262144
|
||||
max_output = 16384
|
||||
|
||||
[models."moonshotai.kimi-k2.5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = false
|
||||
|
||||
[models."moonshotai.kimi-k2.5".costs]
|
||||
input_cost_per_mtok = 0.60
|
||||
output_cost_per_mtok = 3.00
|
||||
|
||||
[models."zai.glm-5"]
|
||||
provider = "bedrock"
|
||||
display_name = "GLM 5 (Bedrock)"
|
||||
family = "glm"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."zai.glm-5".limits]
|
||||
context_window = 200000
|
||||
max_output = 128000
|
||||
|
||||
[models."zai.glm-5".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."zai.glm-5".costs]
|
||||
input_cost_per_mtok = 1.00
|
||||
output_cost_per_mtok = 3.20
|
||||
|
||||
[models."minimax.minimax-m2.5"]
|
||||
provider = "bedrock"
|
||||
display_name = "MiniMax M2.5 (Bedrock)"
|
||||
family = "minimax-m2"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."minimax.minimax-m2.5".limits]
|
||||
context_window = 196000
|
||||
max_output = 8192
|
||||
|
||||
[models."minimax.minimax-m2.5".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."minimax.minimax-m2.5".costs]
|
||||
input_cost_per_mtok = 0.30
|
||||
output_cost_per_mtok = 1.20
|
||||
|
||||
[models."nvidia.nemotron-3-super"]
|
||||
provider = "bedrock"
|
||||
api_id = "nvidia.nemotron-super-3-120b"
|
||||
display_name = "Nemotron 3 Super (Bedrock)"
|
||||
family = "nemotron-3"
|
||||
billing_policy = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."nvidia.nemotron-3-super".limits]
|
||||
context_window = 256000
|
||||
max_output = 32768
|
||||
|
||||
[models."nvidia.nemotron-3-super".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
# Claude Fable 5: adaptive thinking is always on server-side; the row pins
|
||||
# sampling_params = false so the Converse encoder drops temperature/top_p
|
||||
# (Bedrock rejects them for this model). Requires the account-level
|
||||
# provider_data_share opt-in in the Bedrock console. Effort-level mapping
|
||||
# through additionalModelRequestFields is a named follow-up, so
|
||||
# reasoning_effort stays undeclared here (requests carrying one are
|
||||
# rejected up front rather than silently dropped).
|
||||
|
||||
[models."us.anthropic.claude-fable-5"]
|
||||
provider = "bedrock"
|
||||
display_name = "Claude Fable 5 (Bedrock)"
|
||||
family = "claude-5"
|
||||
billing_policy = "anthropic"
|
||||
|
||||
[models."us.anthropic.claude-fable-5".limits]
|
||||
context_window = 1000000
|
||||
max_output = 128000
|
||||
|
||||
[models."us.anthropic.claude-fable-5".features]
|
||||
tools = true
|
||||
vision = true
|
||||
reasoning = true
|
||||
prompt_cache = true
|
||||
sampling_params = false
|
||||
|
||||
[models."us.anthropic.claude-fable-5".costs]
|
||||
input_cost_per_mtok = 10.0
|
||||
output_cost_per_mtok = 50.0
|
||||
cache_input_cost_per_mtok = 1.0
|
||||
|
|
@ -40,6 +40,10 @@ pub enum CodecKind {
|
|||
#[strum(to_string = "openai_compatible")]
|
||||
OpenAiCompatible,
|
||||
GeminiGenerate,
|
||||
/// Amazon Bedrock's unified Converse/ConverseStream dialect: one
|
||||
/// model-agnostic envelope AWS translates to each hosted family's
|
||||
/// native format server-side.
|
||||
BedrockConverse,
|
||||
}
|
||||
|
||||
impl CodecKind {
|
||||
|
|
@ -53,6 +57,7 @@ impl CodecKind {
|
|||
AdapterKind::OpenAi => Self::OpenAiResponses,
|
||||
AdapterKind::Gemini => Self::GeminiGenerate,
|
||||
AdapterKind::OpenAiCompatible => Self::OpenAiCompatible,
|
||||
AdapterKind::Bedrock => Self::BedrockConverse,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -90,6 +95,7 @@ mod tests {
|
|||
(CodecKind::OpenAiResponses, "openai_responses"),
|
||||
(CodecKind::OpenAiCompatible, "openai_compatible"),
|
||||
(CodecKind::GeminiGenerate, "gemini_generate"),
|
||||
(CodecKind::BedrockConverse, "bedrock_converse"),
|
||||
] {
|
||||
assert_eq!(kind.as_str(), expected);
|
||||
assert_eq!(kind.to_string(), expected);
|
||||
|
|
@ -103,6 +109,7 @@ mod tests {
|
|||
(AdapterKind::OpenAi, CodecKind::OpenAiResponses),
|
||||
(AdapterKind::Gemini, CodecKind::GeminiGenerate),
|
||||
(AdapterKind::OpenAiCompatible, CodecKind::OpenAiCompatible),
|
||||
(AdapterKind::Bedrock, CodecKind::BedrockConverse),
|
||||
] {
|
||||
assert_eq!(CodecKind::default_for(adapter), expected);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2546,6 +2546,20 @@ regex = '''\b(sk-[a-zA-Z0-9]{20}T3BlbkFJ[a-zA-Z0-9]{20})(?:['|\"|\n|\r|\s|\x60|;
|
|||
entropy = 3
|
||||
keywords = ["t3blbkfj"]
|
||||
|
||||
[[rules]]
|
||||
id = "aws-bedrock-long-term-api-key"
|
||||
description = "Found an AWS Bedrock long-term API key, posing a risk of unauthorized model invocation and billing."
|
||||
regex = '''\b(ABSK[A-Za-z0-9+/]{109,269}={0,2})(?:[\x60'\"\s;]|\\[nr]|$)'''
|
||||
entropy = 3
|
||||
keywords = ["absk"]
|
||||
|
||||
[[rules]]
|
||||
id = "aws-bedrock-short-term-api-key"
|
||||
description = "Found an AWS Bedrock short-term API key, posing a risk of unauthorized model invocation."
|
||||
regex = '''\b(bedrock-api-key-YmVkcm9jay5hbWF6b25hd3MuY29t[A-Za-z0-9+/=]{16,})(?:[\x60'\"\s;]|\\[nr]|$)'''
|
||||
entropy = 3
|
||||
keywords = ["bedrock-api-key-"]
|
||||
|
||||
[[rules]]
|
||||
id = "openrouter-api-key"
|
||||
description = "Found an OpenRouter API Key, posing a risk of unauthorized access to LLM provider routing and billing."
|
||||
|
|
|
|||
|
|
@ -17,6 +17,32 @@ const WORKER_ENV_ALLOWLIST: &[&str] = &[
|
|||
EnvVars::NO_COLOR,
|
||||
EnvVars::CLICOLOR,
|
||||
EnvVars::CLICOLOR_FORCE,
|
||||
// AWS credential-chain inputs for the Bedrock provider. Other providers'
|
||||
// secrets reach the worker through the server vault (read via FABRO_HOME),
|
||||
// but Bedrock SigV4 has no stored secret — it re-resolves from the ambient
|
||||
// AWS chain on every request so STS/SSO/IRSA sessions can refresh, which
|
||||
// means the chain's *inputs* must survive `env_clear()` in the worker, not
|
||||
// a snapshot taken at launch. We pass the identity surface only (static
|
||||
// keys, session token, profile/region selectors, and the web-identity/ECS
|
||||
// role vars); HOME already carries the shared
|
||||
// `~/.aws` config + SSO cache. Endpoint/metadata overrides
|
||||
// (AWS_ENDPOINT_*, AWS_METADATA_ENDPOINT, AWS_IMDSV1_FALLBACK) are
|
||||
// deliberately excluded — they belong to the server's S3 path, not to the
|
||||
// worker's outbound model calls. Bedrock bearer API keys are optional LLM
|
||||
// provider secrets, so server workers read them through the vault rather
|
||||
// than inheriting process env.
|
||||
EnvVars::AWS_ACCESS_KEY_ID,
|
||||
EnvVars::AWS_SECRET_ACCESS_KEY,
|
||||
EnvVars::AWS_SESSION_TOKEN,
|
||||
EnvVars::AWS_PROFILE,
|
||||
EnvVars::AWS_REGION,
|
||||
EnvVars::AWS_DEFAULT_REGION,
|
||||
EnvVars::AWS_ROLE_ARN,
|
||||
EnvVars::AWS_ROLE_SESSION_NAME,
|
||||
EnvVars::AWS_WEB_IDENTITY_TOKEN_FILE,
|
||||
EnvVars::AWS_CONTAINER_CREDENTIALS_RELATIVE_URI,
|
||||
EnvVars::AWS_CONTAINER_CREDENTIALS_FULL_URI,
|
||||
EnvVars::AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE,
|
||||
];
|
||||
|
||||
const RENDER_GRAPH_ENV_ALLOWLIST: &[&str] = &[EnvVars::PATH, EnvVars::HOME, EnvVars::TMPDIR];
|
||||
|
|
@ -91,6 +117,12 @@ mod tests {
|
|||
("NO_COLOR".to_string(), "1".to_string()),
|
||||
("CLICOLOR".to_string(), "0".to_string()),
|
||||
("CLICOLOR_FORCE".to_string(), "1".to_string()),
|
||||
("AWS_ACCESS_KEY_ID".to_string(), "AKIAEXAMPLE".to_string()),
|
||||
("AWS_SECRET_ACCESS_KEY".to_string(), "secret".to_string()),
|
||||
("AWS_SESSION_TOKEN".to_string(), "session".to_string()),
|
||||
("AWS_BEARER_TOKEN_BEDROCK".to_string(), "bearer".to_string()),
|
||||
("BEDROCK_API_KEY".to_string(), "alias-bearer".to_string()),
|
||||
("AWS_REGION".to_string(), "us-east-2".to_string()),
|
||||
("SESSION_SECRET".to_string(), "leak".to_string()),
|
||||
("FABRO_JWT_PRIVATE_KEY".to_string(), "leak".to_string()),
|
||||
("FABRO_JWT_PUBLIC_KEY".to_string(), "leak".to_string()),
|
||||
|
|
@ -122,6 +154,27 @@ mod tests {
|
|||
assert_eq!(actual.get("NO_COLOR").map(String::as_str), Some("1"));
|
||||
assert_eq!(actual.get("CLICOLOR").map(String::as_str), Some("0"));
|
||||
assert_eq!(actual.get("CLICOLOR_FORCE").map(String::as_str), Some("1"));
|
||||
// Bedrock SigV4 chain inputs cross into the worker so it can re-resolve
|
||||
// credentials per request; a generic secret with no allowlist entry
|
||||
// still does not.
|
||||
assert_eq!(
|
||||
actual.get("AWS_ACCESS_KEY_ID").map(String::as_str),
|
||||
Some("AKIAEXAMPLE")
|
||||
);
|
||||
assert_eq!(
|
||||
actual.get("AWS_SECRET_ACCESS_KEY").map(String::as_str),
|
||||
Some("secret")
|
||||
);
|
||||
assert_eq!(
|
||||
actual.get("AWS_SESSION_TOKEN").map(String::as_str),
|
||||
Some("session")
|
||||
);
|
||||
assert_eq!(
|
||||
actual.get("AWS_REGION").map(String::as_str),
|
||||
Some("us-east-2")
|
||||
);
|
||||
assert!(!actual.contains_key("AWS_BEARER_TOKEN_BEDROCK"));
|
||||
assert!(!actual.contains_key("BEDROCK_API_KEY"));
|
||||
assert!(!actual.contains_key("FABRO_LOG_DESTINATION"));
|
||||
assert_eq!(
|
||||
actual.get("FABRO_DEV_TOKEN").map(String::as_str),
|
||||
|
|
|
|||
|
|
@ -40,7 +40,9 @@ impl EnvVars {
|
|||
|
||||
// LLM providers and tool integrations
|
||||
pub const ANTHROPIC_API_KEY: &'static str = "ANTHROPIC_API_KEY";
|
||||
pub const AWS_BEARER_TOKEN_BEDROCK: &'static str = "AWS_BEARER_TOKEN_BEDROCK";
|
||||
pub const ANTHROPIC_BASE_URL: &'static str = "ANTHROPIC_BASE_URL";
|
||||
pub const BEDROCK_API_KEY: &'static str = "BEDROCK_API_KEY";
|
||||
pub const BRAVE_SEARCH_API_KEY: &'static str = "BRAVE_SEARCH_API_KEY";
|
||||
pub const CHATGPT_ACCOUNT_ID: &'static str = "CHATGPT_ACCOUNT_ID";
|
||||
pub const GEMINI_API_KEY: &'static str = "GEMINI_API_KEY";
|
||||
|
|
@ -81,11 +83,14 @@ impl EnvVars {
|
|||
"AWS_CONTAINER_CREDENTIALS_FULL_URI";
|
||||
pub const AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: &'static str =
|
||||
"AWS_CONTAINER_CREDENTIALS_RELATIVE_URI";
|
||||
pub const AWS_DEFAULT_REGION: &'static str = "AWS_DEFAULT_REGION";
|
||||
pub const AWS_ENDPOINT: &'static str = "AWS_ENDPOINT";
|
||||
pub const AWS_ENDPOINT_URL_S3: &'static str = "AWS_ENDPOINT_URL_S3";
|
||||
pub const AWS_ENDPOINT_URL_STS: &'static str = "AWS_ENDPOINT_URL_STS";
|
||||
pub const AWS_IMDSV1_FALLBACK: &'static str = "AWS_IMDSV1_FALLBACK";
|
||||
pub const AWS_METADATA_ENDPOINT: &'static str = "AWS_METADATA_ENDPOINT";
|
||||
pub const AWS_PROFILE: &'static str = "AWS_PROFILE";
|
||||
pub const AWS_REGION: &'static str = "AWS_REGION";
|
||||
pub const AWS_ROLE_ARN: &'static str = "AWS_ROLE_ARN";
|
||||
pub const AWS_ROLE_SESSION_NAME: &'static str = "AWS_ROLE_SESSION_NAME";
|
||||
pub const AWS_SECRET_ACCESS_KEY: &'static str = "AWS_SECRET_ACCESS_KEY";
|
||||
|
|
@ -179,6 +184,8 @@ mod tests {
|
|||
EnvVars::FABRO_WORKER_TOKEN,
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::ANTHROPIC_BASE_URL,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::BEDROCK_API_KEY,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::CHATGPT_ACCOUNT_ID,
|
||||
EnvVars::GEMINI_API_KEY,
|
||||
|
|
@ -212,11 +219,14 @@ mod tests {
|
|||
EnvVars::AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE,
|
||||
EnvVars::AWS_CONTAINER_CREDENTIALS_FULL_URI,
|
||||
EnvVars::AWS_CONTAINER_CREDENTIALS_RELATIVE_URI,
|
||||
EnvVars::AWS_DEFAULT_REGION,
|
||||
EnvVars::AWS_ENDPOINT,
|
||||
EnvVars::AWS_ENDPOINT_URL_S3,
|
||||
EnvVars::AWS_ENDPOINT_URL_STS,
|
||||
EnvVars::AWS_IMDSV1_FALLBACK,
|
||||
EnvVars::AWS_METADATA_ENDPOINT,
|
||||
EnvVars::AWS_PROFILE,
|
||||
EnvVars::AWS_REGION,
|
||||
EnvVars::AWS_ROLE_ARN,
|
||||
EnvVars::AWS_ROLE_SESSION_NAME,
|
||||
EnvVars::AWS_SECRET_ACCESS_KEY,
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ const BOOTSTRAP_SECRETS: &[&str] = &[
|
|||
|
||||
const OPTIONAL_VAULT_SECRETS: &[&str] = &[
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::BEDROCK_API_KEY,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::FABRO_SLACK_APP_TOKEN,
|
||||
EnvVars::FABRO_SLACK_BOT_TOKEN,
|
||||
|
|
@ -86,6 +88,8 @@ mod tests {
|
|||
EnvVars::DAYTONA_API_KEY,
|
||||
EnvVars::BRAVE_SEARCH_API_KEY,
|
||||
EnvVars::ANTHROPIC_API_KEY,
|
||||
EnvVars::AWS_BEARER_TOKEN_BEDROCK,
|
||||
EnvVars::BEDROCK_API_KEY,
|
||||
EnvVars::GEMINI_API_KEY,
|
||||
EnvVars::INCEPTION_API_KEY,
|
||||
EnvVars::KIMI_API_KEY,
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue