The async-transform path in the shared handler sent every provider's sign_and_log to the AWS pool, so Ollama, Snowflake, and watsonx queued behind Bedrock refreshes there. Only SignsRequestsWithAWS configs take run_aws_signing now, the rest keep the default-executor hop they had. The executor isolation test also runs on its own loop instead of pinning a one-thread default executor on the session-scoped pytest loop
asyncio.to_thread puts every Bedrock signing on the loop's default executor, the same pool every provider's async entry point hops through, so signings parked on botocore's refresh lock queued unrelated providers behind Bedrock. run_aws_signing runs them on a 16-thread pool only AWS signing uses
The encrypted_content_affinity check only read the pin from the Responses
input, which /v1/messages builds after the router has picked a deployment,
so a model group spread across OpenAI orgs sent follow-up turns to the
wrong org and got invalid_encrypted_content back. The check now also
decodes the pin from bridge-tagged thinking and redacted_thinking blocks
in the Anthropic messages. The bridge also keeps a deployment's own
include list next to reasoning.encrypted_content instead of replacing it.
Guardrails that hand back tool_calls in their own shape (vendor JSON, user code output) raised a KeyError on the non-stream Responses write-back. Returned tool calls are now validated before comparison; a shape or count that does not line up leaves every tool-call item unchanged and logs a warning naming the guardrail. A tool call's name is written back only when the guardrail changed it, so a nameless custom_tool_call no longer picks up the custom_tool placeholder.
The converse reasoning gate only matched openai.gpt-5, so gpt-6-astra fell through to
Anthropic's thinking block and Bedrock rejected the call with 400 Unknown parameter:
'thinking'. Match any openai.gpt-<digit> model at the three gate sites instead.
Nova 2 lite and pro accept forced tool_choice on Converse (verified live on
us.amazon.nova-2-lite-v1:0), so the nine Nova 2 registry keys now advertise
supports_tool_choice. The invoke dispatcher also forwards json_mode to Nova like it
already does for Anthropic and TwelveLabs.
initialize_azure_sdk_client now falls back to litellm.constants.DEFAULT_MAX_RETRIES
when litellm_params carries no max_retries, so off-router Azure clients (files,
batches, fine-tuning, assistants, audio) honor the env var like OpenAI clients do.
Router paths already default max_retries to 0 and are unchanged.
Regression tests cover the default, explicit 0/5/None values, and the env var
reaching the SDK client in a fresh interpreter.
Skipping the name write-back in either handler left every test green; a
guardrail that renames a tool call now has a regression test on both the
chat chunk path and the Anthropic SSE path
The bridge now asks for reasoning.encrypted_content whenever the provider's
Responses config lists include, independent of the client's thinking block,
and leaves it out for providers such as Perplexity that reject the param.
Bridge-tagged blocks are stripped on the chat adapter path too, so a mid
session model switch to Gemini or Bedrock no longer forwards them as real
signatures, a bare prefix counts as bridge-tagged, and non-mapping messages
pass through the strip untouched.
Post-call guardrails on /v1/responses only treated function_call output
items as tool calls, so a custom_tool_call item (Codex's exec shell tool
on GPT-5.6 models) was never scanned or masked, non-streaming and
streaming alike. Both item types now flow through the shared
tool_call_dict_from_output_item helper, ended-stream delivery syncs the
custom_tool_call_input delta/done events and the item's input field, and
the completed-response scan key fingerprints both kinds of item.
Non-streaming Responses tool-call MASK rewrites were also never written
back to the output item even for function_call; they are now.
The Databricks chat transformation only parsed reasoning out of FMAPI-style
reasoning content blocks, so external models behind Databricks AI Gateway that
return the OpenAI-style top-level reasoning_content string lost it, both in the
final message and in every streamed delta. Fall back to the shared OpenAI
reasoning helper when no reasoning block exists, and keep the delta's own
reasoning_content when streaming.
Narrows the no-choices guard so a dict, string, or None still raises the APIError while an empty list passes through,
guards the non-stream Anthropic bridge against indexing an empty choices list, and repairs test_completion_missing_role,
whose raw-response mock was patched in as the create() callable itself so the handler only ever saw a MagicMock
Nine cases assert the sts client is built with verify=True, but get_ssl_verify
reads SSL_CERT_FILE and SSL_VERIFY, so the argument depended on the ambient
environment. The published images set SSL_CERT_FILE, so the suite failed there
while passing in CI.
Fixes#40357
Reasoning items the Responses API returned for a /v1/messages turn were rebuilt
from their summary text on every replay, so the prompt the model saw changed
between turns and the prompt cache never matched. The bridge now asks for
reasoning.encrypted_content, carries it in the thinking signature (or as a
redacted_thinking block when there is no summary), and replays it verbatim as
the reasoning item's encrypted_content. Anthropic replay paths drop those
tagged blocks so a cross-model resume never forwards OpenAI bytes to Anthropic
get_optional_params_image_gen only forwarded the per-call drop_params flag
to provider configs, so litellm_settings drop_params: true never dropped
the n the MAI generations endpoint ignores. The MAI edits config also
advertised and forwarded size, which that endpoint ignores. Non-numeric
and non-positive n now surface as a 400 instead of a 500 or a pass-through.
The bridges derived prompt_cache_key as the first 64 chars of metadata.user_id.
Claude Code packs a JSON object into that field whose prefix is the per-install
device_id, so every session and subagent on one machine shared a single key,
and a plain end-user id pinned all of that user's conversations to one slot.
Parse the JSON and use session_id; send no key otherwise so the provider falls
back to its own prompt-prefix hashing. An explicit prompt_cache_key still wins.
Fixes#39145
