Record supports_tool_search on the Bedrock Claude entries in both cost
map files and have _supports_tool_search_on_bedrock read it first via
the provider-resolved capability lookup, keeping the name patterns as a
fallback for ARNs and ids the map cannot resolve. Threads the flag
through ModelInfoBase and drops a dated remark from the pattern list
Claude Code drives Opus 4.7 with thinking {"type": "adaptive"} plus
output_config {"effort": "max"}. The anthropic-to-openai adapter
forwarded thinking verbatim for Claude models but dropped output_config,
and Bedrock Converse streams zero reasoningContent blocks for adaptive
thinking without an effort tier. Forward the effort subset of
output_config for Bedrock targets, accept it in the converse supported
params, and map it with the model's effort ceiling applied. Re-enable
the skipped e2e compat cell that catches this
Bedrock InvokeModel rejects tool_search_tool_* tool types unless the
request body carries the tool-search-tool-2025-10-19 beta. The model
allowlist gating that beta omitted Haiku 4.5 (and Opus 4.7, supported
since launch per live verification), so every tool-search request on
those models got a Bedrock 400. Add both to the allowlist and re-enable
the e2e compat cell that caught it.
* fix(bedrock): reject Anthropic server-side web_search tool with actionable error
Bedrock's Anthropic Messages endpoints cannot execute Anthropic's
server-side web_search tool, so forwarding it returns an opaque
"The provided request is not valid" 400 from Bedrock. Fail fast in the
invoke transform with an error that names the unsupported tool, the
model, and links the web search interception docs as the fix.
* refactor(bedrock): address review nits on web_search guard typing
HTTPHandler.post and AsyncHTTPHandler.post call raise_for_status before returning, so the status_code != 200 branches after the create and cancel POSTs could never run. Non-2xx already surfaces as httpx.HTTPStatusError from inside the client. The checks after GETs stay: the get helpers return without raising. Tests that faked a non-raising POST response are replaced by HTTPStatusError propagation coverage.
* fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315)
The build_web_search_tool_result_block method copied url/title/page_age but
hardcoded encrypted_content to empty string, never reading SearchResult.snippet.
This left every native block content-free, forcing clients to web_fetch each
result to recover evidence—the reported symptom.
The Anthropic spec carries page text only in encrypted_content (an opaque
server-issued blob we cannot mint), so snippet is emitted as an additive key
alongside the spec fields. encrypted_content stays empty rather than holding
plaintext, which would assert encryption semantics that don't hold.
The anthropic SDK's BaseModel sets extra='allow', so the additive snippet key
survives SDK parsing. litellm has no typed model for web_search_result at all,
so nothing drops it internally. Turn-2 replay behavior is unaffected: the
empty encrypted_content already exists today.
Tests:
- Updated test_shape_with_results to assert snippet present
- Added test_snippet_carried_for_every_result to cover multi-result ordering
- Added test_missing_snippet_degrades_to_empty_string for edge case
- Mutation check: reverting source-only yields 3 test failures, restored to 117 passed
Fixes: LIT-5315
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(websearch): make synthesized web_search blocks replayable by native clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(websearch): flatten a resultless replayed search block so Bedrock accepts the next turn
The flatten added for LIT-5315 bails when the replayed web_search_tool_result
carries an empty content list, but that is exactly what the interceptor emits
when a search legitimately returns nothing and when a search raises. The block
survived into the outbound body, Bedrock rejected the tag, and the conversation
died on the following turn just as it did before the flatten existed.
An empty content list has no encrypted_content to respect and no evidence to
preserve, so it flattens safely, and its paired server_tool_use goes with it.
The rendered text now says so explicitly rather than emitting a bare header.
Adds the multi-turn replay coverage that existed nowhere: the outbound Bedrock
invoke body is asserted free of both block types, parametrized over the
results-present and resultless cases, and built from the interceptor's own
builder so the fixture cannot drift from what it emits.
Resolves LIT-5320
* test(websearch): pin flatten idempotency for the agentic-loop re-entry
The agentic loop re-enters the same /v1/messages entry point for its follow-up
call and hands it the original client history, so the flatten runs again over
already-flattened messages once per iteration. Bedrock always takes that path,
since its config reports web search as natively handled and the short-circuit
is skipped.
A pass that appended the rendered text instead of replacing the block would
duplicate the evidence on every iteration and re-ship the unsupported tag, and
no existing single-pass test sees it. Mutation checked: keeping the original
block alongside the rendered text fails this test on its own.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
#35978 stopped the pooled A2A client replaying one upstream's Set-Cookie to
another by installing a blocking policy on that client's httpx cookie jar. That
covers only one of the two jars on the request path. AiohttpTransport is the
default transport unless it is explicitly disabled, and the aiohttp ClientSession
behind it keeps its own cookie jar which no httpx-level assertion can observe, so
the leak is still live on the default path: a live proxy on that commit still
delivers agent-alpha's session cookie to agent-beta's card fetch and JSON-RPC
call.
The reason it looked fixed is that aiohttp's default CookieJar is built with
unsafe=False and refuses to store cookies for IP hosts, so a proof addressed to
127.0.0.1 comes back clean whether or not that jar is blocked.
Cookie persistence is now blocked where the clients are built rather than at one
call site: blocked_cookie_jar() gives every httpx client, async and sync, a jar
whose DefaultCookiePolicy(allowed_domains=()) rejects every domain in both
directions, and both ClientSession constructions litellm owns, the transport's
session factory and the proxy's shared startup session, get a DummyCookieJar.
LiteLLM reads a response cookie nowhere, and an explicitly supplied Cookie header
still goes out, so passthrough forwarding and an agent's extra_headers are
unaffected. The A2A-scoped policy #35978 added is removed, since it is now dead.
The two suites that drive the aiohttp session factory synchronously mock
ClientSession because a real one needs a running event loop; DummyCookieJar has
the same requirement, so they mock it for the same reason.
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
Gate the OpenAI handler's tools forwarding behind scan_only_tool_results,
matching the Anthropic handler, so a tool-results-only scan can no longer
evaluate or rewrite trusted function definitions.
When a guardrail returns a replacement structured_messages list, substitute
the returned messages back into the positions their scoped originals came
from instead of installing the scoped list as the whole conversation, so
out-of-scope messages (system prompt, prior turns) survive redaction on
both the OpenAI and Anthropic paths.
A cached HTTPHandler hands its raw httpx.Client out to consumers that keep it
for the process lifetime. When the shared client cache expires the entry on its
TTL or evicts it under the 200-entry cap, nothing references the handler, so it
is collected and its finalizer closed the client those consumers still hold.
Langfuse ingestion then failed silently on the SDK's background flush thread
until the process restarted.
A finalizer running proves only that nothing references the handler; it proves
nothing about the client. Both handlers now close the client during finalization
only when they built it and are still its sole referrer, so an unshared client is
still released promptly and a handed-out one is left alone. That keeps the
pooled-socket reclamation the finalizer was providing, which measures identical
to base over 2000 handler create-and-drop cycles.
Explicit close() stays, now gated on _owns_client so the wrapper never closes a
caller-injected client, and __aexit__ routes through it.
LangFuseLogger also keeps a reference to the handler whose client it hands the
SDK. Previously that handler was a local that went out of scope immediately,
leaving the client reachable only from the SDK. It still shares the cached
client, so no extra clients are created per logger.
The Jina key fallback chain read JINA_AI_API_KEY three times in a row
before falling through to JINA_AI_TOKEN, so two of the four slots were
dead. Jina's own documentation publishes JINA_API_KEY, and litellm's
rerank validate_environment already tells users to set that name, but
nothing ever read it: a user who set only JINA_API_KEY got no key
resolved and Jina answered AUTH_MISSING_API_KEY.
Replace one of the repeats with JINA_API_KEY and drop the other.
JINA_AI_API_KEY stays first so no install that resolves a key today
changes which key it picks.
Guardrails silently skipped three surfaces on the Anthropic Messages
path, so an agent loop driven by /v1/messages ran unguarded:
- The Anthropic input translation never walked tool_result blocks, so
content returned by a local tool (a curl, a file read, an MCP call)
reached the model unscanned in both the string and list content
shapes, images inside a tool_result included.
- tool_permission only understood ModelResponse, so an Anthropic
non-streaming response or a raw SSE stream carrying tool_use blocks
passed through with no rule ever evaluated.
- ContentFilterGuardrail scanned inputs["texts"] but never
inputs["tool_calls"], so the arguments a model proposes for a tool
call went unchecked.
Tool call arguments are parsed as JSON before filtering so a MASK
action rewrites the value and leaves the payload valid JSON; non-JSON
arguments fall back to scanning the raw string. Denied tool_use blocks
are dropped from the Anthropic content array and replaced with a text
block, and stop_reason resets to end_turn when nothing tool-shaped
survives.
S3 rebuilds the canonical request from the wire path with single
percent-encoding, which botocore models as S3SigV4Auth. Generic SigV4Auth
quotes the already encoded path a second time, so an object key holding any
character that percent-encodes was signed over %2520 while the request
carried %20, and S3 answered 403 SignatureDoesNotMatch.
The S3 logger was corrected in #35726; the Bedrock managed-files upload and
retrieval paths copied that same pre-fix pattern and were left behind, so a
configured bucket prefix with a space 403s every file upload and every file
content read.
Resolve litellm/types/google_genai/main.py and litellm/types/utils.py by
keeping this branch's modernized annotations on top of staging's removal
of inert type: ignore comments. Rebuild ruff-strict-budget.json from
measured merged-tree counts where de-excluding litellm/types adds
violations, keeping the stricter of the two sides' limits everywhere
else so no rule gains headroom. Fix the four type-discipline additions
the merge surfaced: freeze GEMINI_1_5_ACCEPTED_FILE_TYPES, drop a
callback_args parameter rebind in guardrails, and give the two remaining
mutations reasoned suppressions
ruff.toml has excluded litellm/types/* since 2024, so no lint rule ever ran
on the types tree. Remove the exclusion, apply ruff --fix and ruff format
across litellm/types, and hand-fix what autofix cannot reach so the
pyupgrade budgets stay at zero: implicit type aliases converted to PEP 604
unions, RootModel[Union[...]] bases, duplicate imports, and a stray print.
Load-bearing import X as X re-exports deleted by preview-mode F401 are
restored, and the six star-imported hub modules keep their re-export
surface via per-file F401 ignores. Star-import consumers that silently
relied on typing names leaking from those hubs are modernized to builtin
generics and PEP 604 unions.
Runtime annotation introspection that only recognized typing.Union is
taught types.UnionType (guardrail UI field schemas, volcengine response
fill), with regression tests for both. Strict budget limits for the rules
the types tree now trips are raised to exact measured totals, so any
net-new violation still fails the gate
The re-landed closer test asserted a reaped handler's client stays
closed; with #35862 the handler heals on next access, so the test now
pins the inner client up front and asserts the heal as the contract.
Also adds an end-to-end regression test that evicts an init-held
handler through LLMClientCache, waits out the grace close, and proves
the next request succeeds.
Since #35492 the evicted-client closer really closes litellm-owned
httpx clients once their cache entry is evicted and the grace window
passes. Objects that fetch get_async_httpx_client once in __init__ and
hold the handler forever (40 guardrail classes, pagerduty and email
callbacks, and more) then fail every request with 'RuntimeError: Cannot
send a request, as the client has been closed.'
AsyncHTTPHandler.client and HTTPHandler.client are now properties that
rebuild the inner client from the constructor's stored config when the
handler owns it and finds it closed. Caller-supplied or assigned
clients are never rebuilt, and close paths use the backing field so
closing a handler does not resurrect it.
