mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
d5b2220ed3
Adds **Amazon Bedrock** as an opt-in built-in provider, over Bedrock's unified **Converse / ConverseStream** API. One codec serves every Converse-capable family — Claude, Amazon Nova, Meta Llama, Mistral, DeepSeek, Moonshot Kimi, Z.AI GLM, MiniMax, NVIDIA Nemotron, and OpenAI gpt-oss — because AWS translates the envelope to each model's native dialect server-side. Auth is either **AWS SigV4** (the default credential chain — env / profile / IMDS / IRSA / SSO, resolved per request so sessions refresh) or a **Bedrock API key** (`AWS_BEARER_TOKEN_BEDROCK`, bearer). Disabled by default (the Ollama / OpenRouter opt-in pattern). This is the redo of #459's original Claude-only `InvokeModel` adapter, rebuilt on the gateway-refactor seams (#481–#497). @depopry's SigV4 signer, AWS event-stream frame decoder, `BedrockAuth`, the `aws_sigv4` credential grammar, `AdapterKind::Bedrock`, region-from-base_url, and the lean-deps decision are preserved and authored by him on the first two commits; the per-family `BedrockCodec` trait he wrote turned out to be the crate-wide `Codec` seam in miniature, so the refactor promoted exactly that shape. The original Claude-only description is preserved in a comment below. ## What's here - **`AdapterKind::Bedrock` × `CodecKind::BedrockConverse`** on the route, plus the `aws_sigv4` credential source (no static secret — the adapter signs at request time; `fabro-auth` stays AWS-free). *(@depopry)* - **SigV4 signer + AWS event-stream `FrameDecoder`** on the lean AWS stack (no `aws-sdk-bedrockruntime`; transport stays on `fabro-http`). Re-targeted at Converse's direct-JSON stream frames; the signer resolves credentials per request. *(@depopry)* - **`bedrock_converse` codec** — Converse envelope (`system[]`, typed content blocks, `inferenceConfig`, `toolConfig`), prompt caching via `cachePoint`, thinking-signature round-trip through `reasoningContent`, usage mapped onto the disjoint `TokenCounts` buckets, `provider_options.bedrock` passthrough. Plus the adapter shell and an event-stream byte loop beside the transport's shared SSE loop. - **Catalog**: `bedrock.toml` (Claude incl. Fable 5, Nova 2, Llama 4, Mistral, DeepSeek, Kimi, GLM, MiniMax, Nemotron, gpt-oss — cross-region inference-profile ids, per-model `billing_policy` so Claude bills Anthropic-style) and a companion **`bedrock-openai`** provider for GPT-5.5/5.4 over the `bedrock-mantle` Responses endpoint (pure config over the existing `openai_responses` codec, zero new code). - Secrets registry (`AWS_BEARER_TOKEN_BEDROCK`), gitleaks rules for both Bedrock key formats, the `docs/integrations/bedrock` guide, and live e2e tests. ## Live verification (confirmed end-to-end against a real AWS account) Verified on a real Bedrock account (us-east-2, SigV4 + bearer): - **SigV4 + Converse** — multiple families (Claude, Nova, DeepSeek, …) via the full settings → catalog → route → adapter → codec path. - **ConverseStream** — streaming deltas through the workflow engine. - **Multi-turn tool use** — agent loop with tool calls round-tripping (no-arg tools included). - **Multi-model routing** — Claude + DeepSeek pinned in one run through the single Converse codec. - **mantle Responses** — `openai.gpt-5.5` answered via the `bedrock-openai` provider (bearer auth). The exercise caught and fixed several issues that unit tests (static creds, mocked transports) could not — see the follow-up commits below. ## Follow-up fixes from live testing (commits on top of the foundation) 1. **Worker AWS env** — the workflow worker scrubs its env to an allowlist, so SigV4 (which re-resolves from the ambient chain per request) couldn't work through `fabro run`. The AWS credential-chain inputs now cross into the worker. 2. **Vault bearer key** — Bedrock was the only key-based provider missing a `vault:` credential ref, so `fabro secret set AWS_BEARER_TOKEN_BEDROCK` silently didn't feed it. Now resolves env → vault → SigV4. 3. **Converse tool-encoding hardening** — a no-arg tool call's `toolUse.input` is now a `{}` object (Bedrock rejects null), and every tool `inputSchema` gets a top-level `type: "object"` (strict families like DeepSeek reject a typeless schema Claude tolerates). 4. **Nova output cap** — `amazon.nova-2-lite` max_output 65536 → 65535 (Bedrock's per-request limit). Earlier fixes already folded into the foundation commits: the `aws-config` sleep-impl (default chain panicked) and AWS error-body decoding (top-level `message`/`Message`/`__type` → proper messages instead of "Unknown error"). ## Manual testing & setup See `docs/integrations/bedrock` — now documents the non-obvious account setup that live testing surfaced: the per-Region Anthropic use-case approval, `aws-marketplace:Subscribe` for third-party models, the Fable 5 / Mythos-class data-sharing opt-in, and the bearer-vs-SigV4 precedence override for running Converse + mantle side by side. ## Open decision / discussion - **Model-id naming** — Bedrock rows use dotted ids mirroring Bedrock's native inference-profile ids (`us.anthropic.claude-sonnet-4-6`, `openai.gpt-5.5`), which also makes them the wire `api_id`. Third scheme alongside bare ids and OpenRouter's `vendor/model` slashes. No collision risk (enforced at catalog build). Open to a uniform scheme if preferred. - **`BEDROCK_API_KEY` alias** — see the comment thread; the AWS console hands some users `export BEDROCK_API_KEY=` while the SDK-standard var is `AWS_BEARER_TOKEN_BEDROCK`. Question of whether to accept both. ## Deferred (named follow-ups) - **`qwen.qwen3-coder-next`** — omitted pending a verified Bedrock model/inference-profile id (its fabro id isn't a valid Bedrock identifier; needs an explicit `api_id`). Re-add once confirmed via `aws bedrock list-inference-profiles`. - **Claude Mythos 5** — Anthropic-Messages-only on `bedrock-mantle` (limited preview). - **Converse structured output** (`response_format` rejected with a clear error). - **`reasoning_effort` on Converse rows** via `additionalModelRequestFields` (the `bedrock-openai` GPT rows already accept effort levels). - **CountTokens** route (`count_input_tokens` returns `None`). ## Verification `cargo nextest run --workspace`: green except the pre-existing environment-dependent fabro-workflow failures (identical on main). clippy `-D warnings` + pinned-nightly fmt clean. Codec unit tests + adapter httpmock tests + frame-decoder/signer locks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Scott Werner <scott@sublayer.com> Co-authored-by: Scott Werner <stwerner@vt.edu> |
||
|---|---|---|
| .. | ||
| src | ||
| tests/it | ||
| Cargo.toml | ||