A trusted hostname over plain http would expose the bearer token or a
replayable SigV4 signature to network observers. Credentials now only ride
https, with localhost exempt so local MCP stubs keep working.
Web-search connector 1.1.0 and later return the machine-readable results in result.structuredContent and may leave the text block as prose, which the parser dropped.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Refuse to SigV4-sign requests to hosts that are neither an AgentCore gateway
hostname nor AGENTCORE_GATEWAY_URL's host, match gateway hostnames on the URL
host instead of anywhere in the URL, accept the env token when api_base is a
real gateway, raise on tools/call responses with result.isError, and split
CRLF-framed SSE events.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
litellm.completion(thinking=True) crashed pre-network in is_thinking_enabled
with a retryable APIConnectionError ('bool' object has no attribute 'get'),
so the router burned retries on a deterministic failure and proxy clients got
a traceback instead of a usable response.
validate_and_fix_thinking_param now coerces thinking=True to the enabled dict
with the default medium budget and drops thinking=False, and the remaining
dict-assuming thinking accessors (base config, bedrock converse, deepseek)
guard with isinstance so raw bools can never crash a transform.
The field itself landed on staging via 0c5c9c79d7; these are the regression tests from PR #31435 for the retrieval-facing half.
(cherry picked from commit a9a322d63f6d4658b1f28d1622335775e94736a4)
Bedrock batch jobs write their results to s3_output_bucket_name when it differs
from the input bucket, but the file-content retrieval path validated the file id
only against the input bucket (s3_bucket_name). A deployment that configures a
separate output bucket therefore could not retrieve its own batch outputs: the
id validated against the input bucket and was rejected as a foreign bucket.
Resolve the trusted output bucket alongside the input bucket from the immutable
credential snapshot (or AWS_S3_OUTPUT_BUCKET_NAME), and try the file id against
each configured bucket, returning the first that validates. The SSRF guard is
preserved: only server-configured buckets are tried, never a request param, and
an id outside both is still rejected.
(cherry picked from commit 1d407c2f26)
Bedrock invoke /v1/messages streaming reports cache_read_input_tokens and
cache_creation_input_tokens on message_stop.usage while attaching
amazon-bedrock-invocationMetrics to the same chunk. The stream decoder
rebuilt that chunk's usage block from inputTokenCount/outputTokenCount
alone, which exclude cache reads and writes, so the cache breakdown was
destroyed before _promote_message_stop_usage could surface it and cache
tokens were billed at $0. Merge instead of replace, and also map
cacheReadInputTokenCount/cacheWriteInputTokenCount when Bedrock reports
the cache itemization inside the invocation metrics.
Co-authored-by: Brian Cox <3924351+brian5021@users.noreply.github.com>
Adds an opt-in operator allow-list, litellm_settings::bedrock_request_metadata_fields, that forwards LiteLLM key, team and end-user identity plus client spend_logs_metadata into Bedrock request metadata so Bedrock spend can be grouped in AWS Cost Explorer.
Covers all three Bedrock surfaces: the Converse body requestMetadata field, and a signed X-Amzn-Bedrock-Request-Metadata header on Invoke chat completions and on Invoke /v1/messages, where the header is the only viable leg.
The resolver reads both metadata variable names, reserves the whole user_api_key_ prefix against caller-supplied keys, caps the client slot budget explicitly at 16 minus the reserved count, and drops rather than rejects auto-injected values that violate Bedrock constraints. Caller-supplied requestMetadata keeps its existing 400 semantics.
The request-metadata field and header are proxy-owned whenever forwarding is enabled. A caller-supplied value, reachable through the generic extra_headers passthrough, is dropped unconditionally and compared case-insensitively, and is replaced only by the proxy's own value, so identity in the AWS billing record cannot be forged. Absence of a resolved value still means absence on the wire rather than a fallback to the caller's. The guardrail headers keep their existing no-displace behaviour.
Shape detection and block normalization sat in the generic batch layer, which
let batch and live parsing of the same wire format drift apart. Both now live on
AmazonConverseConfig as is_converse_usage_shape and usage_from_batch_output, so
batch_utils asks the provider adapter rather than knowing Bedrock's field names.
Adds direct coverage for the shape predicate, the completion of an incomplete
block, cache-count inflation, and the streaming usage event that shares the
public transform. Drops the narrative banner from the batch tests.
Every bedrock batch output line went through the Anthropic usage parser, which
reads snake_case input_tokens/output_tokens. Converse-family models (Nova and
friends) report camelCase inputTokens/outputTokens, so their usage came back
0/0/0 and the batch billed $0 despite real token consumption.
Usage is now selected by the shape of the payload: a Converse-shaped block goes
through the same transform the live Converse path uses, so a batch and an
equivalent non-batch call agree on tokens, including cache reads and writes.
Anthropic-shaped bedrock output is unchanged.
A shape neither parser understands (an InvokeModel-native payload from Titan,
Cohere, or Llama, which name their counts differently again) still reads zero,
but now warns with the keys it saw instead of silently billing $0.
Exposes the Converse usage transform as public, since batch parsing is a second
legitimate caller; that also removes the private-member access invoke_handler
was already making.
Resolves the transform_create_file_response conflict by keeping the
_uploaded_object_size handoff over the response Content-Length read,
and adds the rebind-ok justification LIT011 now requires for the
upload-size litellm_params handoff after the base budget ratcheted.
Three groups, all verified by running the suite rather than by inspection.
18 files whose every test function carries an unconditional @pytest.mark.skip,
39 test functions in total. They are collected on every CI run and always skip,
so they advertise coverage the suite does not have. Reasons on the marks include
"AWS Suspended Account", "lakera deprecated their v1 endpoint" and "moved to
using 'otel' for logging"; 26 of the marks predate 2025.
30 test functions with a byte-identical body and identical decorators to a
sibling in the same file and class, differing only in name. Deleting one of each
pair removes no coverage. Four further candidates were excluded because they
override an inherited test, where deleting the override un-shadows the base
class implementation instead of removing a duplicate.
9 test functions that a later definition of the same name shadows, so Python
never binds them and pytest cannot collect them.
One file that is a demo script rather than a test; its own docstring says to run
it with python.
Verification: collecting the 26 edited files gives 2,492 node IDs before and
2,462 after. The 30 duplicate deletions account for exactly 30 removals, the 9
shadowed deletions account for 0 (confirming at runtime that they were never
collectable), nothing unexplained disappeared, and nothing new appeared. No
other test or module imports any deleted symbol.
Record supports_tool_search on the Bedrock Claude entries in both cost
map files and have _supports_tool_search_on_bedrock read it first via
the provider-resolved capability lookup, keeping the name patterns as a
fallback for ARNs and ids the map cannot resolve. Threads the flag
through ModelInfoBase and drops a dated remark from the pattern list
Claude Code drives Opus 4.7 with thinking {"type": "adaptive"} plus
output_config {"effort": "max"}. The anthropic-to-openai adapter
forwarded thinking verbatim for Claude models but dropped output_config,
and Bedrock Converse streams zero reasoningContent blocks for adaptive
thinking without an effort tier. Forward the effort subset of
output_config for Bedrock targets, accept it in the converse supported
params, and map it with the model's effort ceiling applied. Re-enable
the skipped e2e compat cell that catches this
Bedrock InvokeModel rejects tool_search_tool_* tool types unless the
request body carries the tool-search-tool-2025-10-19 beta. The model
allowlist gating that beta omitted Haiku 4.5 (and Opus 4.7, supported
since launch per live verification), so every tool-search request on
those models got a Bedrock 400. Add both to the allowlist and re-enable
the e2e compat cell that caught it.
* fix(bedrock): reject Anthropic server-side web_search tool with actionable error
Bedrock's Anthropic Messages endpoints cannot execute Anthropic's
server-side web_search tool, so forwarding it returns an opaque
"The provided request is not valid" 400 from Bedrock. Fail fast in the
invoke transform with an error that names the unsupported tool, the
model, and links the web search interception docs as the fix.
* refactor(bedrock): address review nits on web_search guard typing
Adds 'agentcore' as a search provider backed by an AgentCore Gateway MCP web-search target, usable from litellm.search()/`/search` and as a websearch_interception backend. Supports SigV4 (AWS_IAM gateways) and bearer tokens (CUSTOM_JWT gateways) via a new BaseSearchConfig.sign_request hook.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315)
The build_web_search_tool_result_block method copied url/title/page_age but
hardcoded encrypted_content to empty string, never reading SearchResult.snippet.
This left every native block content-free, forcing clients to web_fetch each
result to recover evidence—the reported symptom.
The Anthropic spec carries page text only in encrypted_content (an opaque
server-issued blob we cannot mint), so snippet is emitted as an additive key
alongside the spec fields. encrypted_content stays empty rather than holding
plaintext, which would assert encryption semantics that don't hold.
The anthropic SDK's BaseModel sets extra='allow', so the additive snippet key
survives SDK parsing. litellm has no typed model for web_search_result at all,
so nothing drops it internally. Turn-2 replay behavior is unaffected: the
empty encrypted_content already exists today.
Tests:
- Updated test_shape_with_results to assert snippet present
- Added test_snippet_carried_for_every_result to cover multi-result ordering
- Added test_missing_snippet_degrades_to_empty_string for edge case
- Mutation check: reverting source-only yields 3 test failures, restored to 117 passed
Fixes: LIT-5315
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(websearch): make synthesized web_search blocks replayable by native clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(websearch): flatten a resultless replayed search block so Bedrock accepts the next turn
The flatten added for LIT-5315 bails when the replayed web_search_tool_result
carries an empty content list, but that is exactly what the interceptor emits
when a search legitimately returns nothing and when a search raises. The block
survived into the outbound body, Bedrock rejected the tag, and the conversation
died on the following turn just as it did before the flatten existed.
An empty content list has no encrypted_content to respect and no evidence to
preserve, so it flattens safely, and its paired server_tool_use goes with it.
The rendered text now says so explicitly rather than emitting a bare header.
Adds the multi-turn replay coverage that existed nowhere: the outbound Bedrock
invoke body is asserted free of both block types, parametrized over the
results-present and resultless cases, and built from the interceptor's own
builder so the fixture cannot drift from what it emits.
Resolves LIT-5320
* test(websearch): pin flatten idempotency for the agentic-loop re-entry
The agentic loop re-enters the same /v1/messages entry point for its follow-up
call and hands it the original client history, so the flatten runs again over
already-flattened messages once per iteration. Bedrock always takes that path,
since its config reports web search as natively handled and the short-circuit
is skipped.
A pass that appended the rendered text instead of replacing the block would
duplicate the evidence on every iteration and re-ship the unsupported tag, and
no existing single-pass test sees it. Mutation checked: keeping the original
block alongside the rendered text fails this test on its own.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
S3 rebuilds the canonical request from the wire path with single
percent-encoding, which botocore models as S3SigV4Auth. Generic SigV4Auth
quotes the already encoded path a second time, so an object key holding any
character that percent-encodes was signed over %2520 while the request
carried %20, and S3 answered 403 SignatureDoesNotMatch.
The S3 logger was corrected in #35726; the Bedrock managed-files upload and
retrieval paths copied that same pre-fix pattern and were left behind, so a
configured bucket prefix with a space 403s every file upload and every file
content read.
Converse rejects a request that carries both toolConfig.toolChoice and an
additionalModelRequestFields.tool_choice.type, so any request that pairs
parallel_tool_calls with an explicit tool_choice 400s with "The additional field
tool_choice/type conflicts with the existing field toolConfig.toolChoice.auto".
That pairing is what agentic clients send by default; Codex CLI sends
tool_choice "auto" and parallel_tool_calls false on every turn, so tool calling
was broken outright on Bedrock models that advertise
supports_parallel_tool_use_config.
Drop the type from the Anthropic passthrough once toolChoice carries it, and keep
disable_parallel_tool_use, which has no toolConfig equivalent and is accepted
alongside toolChoice. Measured against Bedrock directly: toolChoice plus
{disable_parallel_tool_use} succeeds for auto, any and tool, while an empty
tool_choice with no toolChoice is rejected for a missing type, so the type still
has to be emitted when the caller sends no tool_choice.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
