fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882)

* fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models

* fix(responses): define the tool_search_call output item locally for older openai SDKs

* fix(responses): gate tool search on the Anthropic request builder and validate deferred tool fields

Bedrock Converse builds its own request and has no tool search, so the bridge now drops the tool_search tool there with a warning instead of sending a tool the transport loses, while Bedrock invoke, Anthropic, and Vertex keep it. A replayed tool_search_call with no stored result is dropped from the Anthropic request so a previous_response_id follow-up no longer answers 400. Only srvtoolu_ ids whose name is one of Anthropic's search tools count as search calls, so a caller function named like one stays a function call. A non-boolean defer_loading or a non-list allowed_callers now answers 400 instead of 500

* fix(responses): keep an answered search call on replay and send tools undeferred when no search tool goes out

A replayed Claude tool search call that the caller answered with a tool message kept its
tool_result in the user turn while its tool_use was dropped from the assistant turn, so
Anthropic answered 400 on the OpenAI-SDK tool loop. The search call is now dropped only when
no tool message answers it.

The Responses bridge now strips defer_loading from every tool whenever no tool search tool is
sent, whether the caller sent none or the route cannot run it, since a deferred tool with no
search tool can never be loaded. Namespace members had started carrying the flag to every
bridge provider, which providers that reject unknown tool keys answered 422.

The audit round-1 integration cells for the bridge wire and chaos paths ride in this commit.

* fix(responses): gate only the OpenAI tool_search shape and keep native Anthropic search tools intact

A native tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119 tool the caller sends through /v1/responses passes through the bridge on every route as before, and the other tools keep defer_loading whenever any Anthropic search tool goes out. The route gate now drops the OpenAI tool_search item before translation instead of filtering the translated regex tool, so bm25 counts as a search tool. answered_tool_call_ids walks the history with islice instead of a slice copy. The tool_search_call output item drops its created_by field, which OpenAI rejects on a replayed input item and the streaming item never carried

* fix(responses): drop malformed tool flags under drop_params and keep stored search calls off non-Anthropic follow-ups

With drop_params on, a non-boolean defer_loading or a non-list allowed_callers on a Responses API tool (or a namespace member) is dropped before the bridge instead of answering 400, so only a request without drop_params gets the 400. A custom tool now carries defer_loading to the chat tool like a function tool does

A previous_response_id follow-up routed to a provider that does not build an Anthropic request no longer replays the stored Claude search call (srvtoolu_ id) as a client tool call. Mistral answered that history 400 "Unexpected tool call id". An assistant turn that held only the search call is dropped there. The Anthropic route keeps the call, since its factory pairs it with the stored result

The tool_search_call output model is renamed OutputToolSearchCall so the proxy's OpenAPI schema keeps the OpenAI SDK's ResponseToolSearchCall name. answered_tool_call_ids reads the history by index, so the lookup is linear in the history length. The streaming done event reads the response it is given, which is the same object it used to read from the iterator

* fix(responses): validate function allowed_callers, send one search tool, and keep replays clean off Anthropic

A function tool whose allowed_callers is not a list of strings now answers 400
instead of 500. The guardrail scan reads tools without malformed flags, so the
route decides between a 400 and a drop under drop_params. When the caller sends
a native Anthropic search tool beside tool_search, only the native one goes out.
Databricks Claude no longer counts as an Anthropic request route.

A previous_response_id follow-up off Anthropic also drops a search call the
client replayed as a function_call with a srvtoolu_ id, and its output. The
answered tool id scan in the Anthropic message factory runs once per assistant
run instead of once per message. A null created_by on tool_search_call and
tool_search_output output items is left out of the serialized response, so a
cache hit returns the same keys as the first call.

* fix(guardrails): keep OpenAI tool_search out of the Responses guardrail tool scan

The scan translated tool_search into Anthropic's regex search tool on every
route, so a guardrail returning only function tools removed tool_search from
native OpenAI requests and OpenAI answered 400 for the deferred tools

* fix(anthropic): drop answered tool search replays beside client calls, place system before dropped search turns, flag tool search on azure_ai

* fix(anthropic): keep answered tool search replays paired with their answers

* fix(anthropic): place mid-conversation system messages in one pass and scan tool_search_call in output guardrails

* fix(guardrails): send empty incomplete tool_search_call arguments when a rewrite is not a JSON object

* fix(responses): keep the caller's defer_loading on Claude routes when no server tool_search is sent

* fix(guardrails): deliver streamed rewrites when OpenAI's tool_search_call has no call_id

* fix(responses): answer 400 to a tool_search execution that is not server or client

OpenAI's own Responses API answers 400 invalid_value to it. With drop_params the malformed
execution is dropped like the other malformed tool flags, which leaves a server search

* test(responses): cover the 400 and drop_params paths for a malformed tool_search execution

* test(responses): cover a Claude search call replayed to Bedrock Converse with cache marks

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

litellm/__init__.py

@@ -2287,6 +2287,7 @@ if TYPE_CHECKING:
     get_response_string: Callable[..., str]
     supports_function_calling: Callable[..., bool]
     supports_web_search: Callable[..., bool]
+    supports_tool_search: Callable[..., bool]
     supports_url_context: Callable[..., bool]
     supports_response_schema: Callable[..., bool]
     supports_parallel_function_calling: Callable[..., bool]

litellm/_lazy_imports_registry.py

@@ -31,6 +31,7 @@ UTILS_NAMES: Final = (
     "create_tokenizer",
     "supports_function_calling",
     "supports_web_search",
+    "supports_tool_search",
     "supports_url_context",
     "supports_response_schema",
     "supports_parallel_function_calling",
@@ -477,6 +478,7 @@ _UTILS_IMPORT_MAP: Final = {
     "create_tokenizer": (".utils", "create_tokenizer"),
     "supports_function_calling": (".utils", "supports_function_calling"),
     "supports_web_search": (".utils", "supports_web_search"),
+    "supports_tool_search": (".utils", "supports_tool_search"),
     "supports_url_context": (".utils", "supports_url_context"),
     "supports_response_schema": (".utils", "supports_response_schema"),
     "supports_parallel_function_calling": (

litellm/litellm_core_utils/prompt_templates/factory.py

@@ -22,6 +22,8 @@ from litellm._uuid import uuid
 from litellm.constants import REDACTED_BY_LITELLM
 from litellm.litellm_core_utils.prompt_templates.mid_conversation_system import (
     anthropic_system_messages,
+    find_anthropic_server_tool_result,
+    is_anthropic_tool_search_use,
     message_field,
     parts_of,
 )
@@ -1822,23 +1824,6 @@ def convert_function_to_anthropic_tool_invoke(
         raise e
 
 
-ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX: Final = "srvtoolu_"
-
-
-def find_anthropic_server_tool_result(
-    tool_id: str,
-    web_search_results: Sequence[object] | None,
-    tool_results: Sequence[object] | None,
-) -> dict[str, object] | None:
-    if not tool_id.startswith(ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX):
-        return None
-    candidates: Final = (*(web_search_results or ()), *(tool_results or ()))
-    return next(
-        (result for result in candidates if isinstance(result, dict) and result.get("tool_use_id") == tool_id),
-        None,
-    )
-
-
 def _tool_call_ids_in_tool_messages(messages: Sequence[object]) -> frozenset[str]:
     return frozenset(
         tool_call_id for message in messages if isinstance(tool_call_id := message_field(message, "tool_call_id"), str)
@@ -1976,6 +1961,8 @@ def convert_to_anthropic_tool_invoke(
                 }
             )
             anthropic_tool_invoke.append(server_tool_result)
+        elif is_anthropic_tool_search_use(tool_id, tool_name) and tool_id not in answered_tool_ids:
+            continue
         else:
             sanitized_tool_id = _sanitize_anthropic_tool_use_id(tool_id)
             _anthropic_tool_use_param = AnthropicMessagesToolUseParam(

litellm/litellm_core_utils/prompt_templates/mid_conversation_system.py

@@ -29,11 +29,17 @@ prompt cache readable across turns. Messages are handled in OpenAI format; the
 Anthropic wire shape is built later by ``anthropic_messages_pt``.
 """
 
-from collections.abc import Iterator, Mapping, Sequence
-from itertools import chain, groupby
+from collections.abc import Iterable, Iterator, Mapping, Sequence
+from itertools import chain, dropwhile, groupby, takewhile
+from types import MappingProxyType
 from typing import Final, Literal, TypeAlias, TypeVar
 
-from litellm.types.llms.anthropic import AnthropicMessagesSystemMessageParam, AnthropicSystemMessageContent
+from litellm.types.llms.anthropic import (
+    ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX,
+    ANTHROPIC_TOOL_SEARCH_TOOL_NAMES,
+    AnthropicMessagesSystemMessageParam,
+    AnthropicSystemMessageContent,
+)
 from litellm.types.llms.openai import (
     AllMessageValues,
     ChatCompletionCachedContent,
@@ -50,6 +56,7 @@ CONVERTED_SYSTEM_NOTE: Final = (
 
 _USER_TYPE_ROLES: Final = frozenset({"user", "tool", "function"})
 _TOOL_ROLES: Final = frozenset({"tool", "function"})
+_NO_IDS: Final[frozenset[str]] = frozenset()
 _RENDERED_PART_TYPES: Final = frozenset({"text", "image_url", "document", "file"})
 _RENDERED_ASSISTANT_PART_TYPES: Final = frozenset({"text", "server_tool_use"})
 _THINKING_BLOCK_TYPES: Final = frozenset({"thinking", "redacted_thinking"})
@@ -283,13 +290,112 @@ def _separate_thinking_blocks_render(message: object, parts: Sequence[object]) -
     return any(_thinking_block_renders(block) for block in parts_of(message_field(message, "thinking_blocks")))
 
 
-def _assistant_renders(message: object) -> bool:
+def is_anthropic_tool_search_use(tool_id: str, tool_name: str) -> bool:
+    return tool_id.startswith(ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX) and tool_name in ANTHROPIC_TOOL_SEARCH_TOOL_NAMES
+
+
+def find_anthropic_server_tool_result(
+    tool_id: str,
+    web_search_results: Sequence[object] | None,
+    tool_results: Sequence[object] | None,
+) -> dict[str, object] | None:
+    if not tool_id.startswith(ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX):
+        return None
+    candidates: Final = (*(web_search_results or ()), *(tool_results or ()))
+    return next(
+        (result for result in candidates if isinstance(result, dict) and result.get("tool_use_id") == tool_id),
+        None,
+    )
+
+
+def _tool_result_ids(answers: Iterable[object]) -> Iterator[str]:
+    for message in answers:
+        if isinstance(tool_call_id := message_field(message, "tool_call_id"), str) and tool_call_id:
+            yield tool_call_id
+
+
+def answered_tool_call_ids(messages: Sequence[object], assistant_index: int) -> frozenset[str]:
+    from_assistant: Final = (messages[index] for index in range(assistant_index, len(messages)))
+    following: Final = dropwhile(lambda message: message_field(message, "role") == "assistant", from_assistant)
+    answers: Final = takewhile(lambda message: message_field(message, "role") in _TOOL_ROLES, following)
+    return frozenset(_tool_result_ids(answers))
+
+
+def _role_group(message: object) -> str:
+    role: Final = message_field(message, "role")
+    if role == "assistant":
+        return "assistant"
+    return "tool" if role in _TOOL_ROLES else "other"
+
+
+def _answered_by_assistant_index(
+    groups: Sequence[tuple[str, tuple[tuple[int, object], ...]]], group_index: int
+) -> Iterator[tuple[int, frozenset[str]]]:
+    following: Final = groups[group_index + 1] if group_index + 1 < len(groups) else ("other", ())
+    answered: Final = (
+        frozenset(_tool_result_ids(message for _, message in following[1])) if following[0] == "tool" else _NO_IDS
+    )
+    return ((index, answered) for index, _ in groups[group_index][1])
+
+
+def _answered_ids_by_index(messages: Sequence[object]) -> Mapping[int, frozenset[str]]:
+    """``answered_tool_call_ids`` for every assistant message, computed in one pass over the messages."""
+    groups: Final = tuple(
+        (role_group, tuple(group))
+        for role_group, group in groupby(enumerate(messages), key=lambda pair: _role_group(pair[1]))
+    )
+    return MappingProxyType(
+        dict(
+            chain.from_iterable(
+                _answered_by_assistant_index(groups, group_index)
+                for group_index, (role_group, _) in enumerate(groups)
+                if role_group == "assistant"
+            )
+        )
+    )
+
+
+def _unresolved_tool_search_id(tool_call: object, message: object) -> str | None:
+    tool_id: Final = message_field(tool_call, "id")
+    tool_name: Final = message_field(message_field(tool_call, "function"), "name")
+    if not isinstance(tool_id, str) or not isinstance(tool_name, str):
+        return None
+    if not is_anthropic_tool_search_use(tool_id, tool_name):
+        return None
+    provider_fields: Final = message_field(message, "provider_specific_fields")
+    server_result: Final = find_anthropic_server_tool_result(
+        tool_id,
+        parts_of(message_field(provider_fields, "web_search_results")),
+        parts_of(message_field(provider_fields, "tool_results")),
+    )
+    return tool_id if server_result is None else None
+
+
+def _unresolved_tool_search_ids(message: object) -> tuple[str, ...]:
+    return tuple(
+        tool_id
+        for tool_call in parts_of(message_field(message, "tool_calls"))
+        if (tool_id := _unresolved_tool_search_id(tool_call, message)) is not None
+    )
+
+
+def _tool_calls_render(message: object, answered: frozenset[str]) -> bool:
+    tool_calls: Final = parts_of(message_field(message, "tool_calls"))
+    search_ids: Final = _unresolved_tool_search_ids(message)
+    if len(search_ids) < len(tool_calls):
+        return True
+    return bool(search_ids) and not answered.isdisjoint(search_ids)
+
+
+def _assistant_renders(message: object, answered: frozenset[str]) -> bool:
     """Whether ``anthropic_messages_pt`` puts a block on the wire for this assistant message.
 
     String content (the converter pads an empty one with a placeholder), a text part,
     a signed thinking part, a server tool part, tool calls, a function call, a kept
-    thinking block and compaction blocks each render. An assistant message with none
-    of them, such as ``content: None`` or an empty list, vanishes from the wire.
+    thinking block and compaction blocks each render. A tool search use with neither a
+    server result nor a caller's answer is dropped, so it renders nothing. An assistant
+    message with none of them, such as ``content: None`` or an empty list, vanishes
+    from the wire.
     """
     content: Final = message_field(message, "content")
     if isinstance(content, str):
@@ -298,13 +404,13 @@ def _assistant_renders(message: object) -> bool:
     return (
         any(_assistant_part_renders(part) for part in parts)
         or _separate_thinking_blocks_render(message, parts)
-        or bool(message_field(message, "tool_calls"))
+        or _tool_calls_render(message, answered)
         or bool(message_field(message, "function_call"))
         or bool(message_field(message_field(message, "provider_specific_fields"), "compaction_blocks"))
     )
 
 
-def _renders(message: object) -> bool:
+def _renders(message: object, answered: frozenset[str]) -> bool:
     """Whether ``anthropic_messages_pt`` puts a block on the wire for this message.
 
     A tool message always becomes a ``tool_result`` and a user message with string
@@ -316,26 +422,31 @@ def _renders(message: object) -> bool:
     if role in _TOOL_ROLES:
         return True
     if role == "assistant":
-        return _assistant_renders(message)
+        return _assistant_renders(message, answered)
     content: Final = message_field(message, "content")
     return isinstance(content, str) or any(
         message_field(part, "type") in _RENDERED_PART_TYPES for part in parts_of(content)
     )
 
 
+def _rendered_indices(messages: Sequence[object]) -> frozenset[int]:
+    answered: Final = _answered_ids_by_index(messages)
+    return frozenset(index for index, message in enumerate(messages) if _renders(message, answered.get(index, _NO_IDS)))
+
+
 def _rendered_block(
     message_index: int,
-    messages: Sequence[AllMessageValues],
+    rendered: frozenset[int],
     blocks: Sequence[tuple[bool, tuple[int, ...]]],
 ) -> int | None:
     block_index: Final = _block_containing(message_index, blocks)
     _, indices = blocks[block_index]
-    return block_index if any(_renders(messages[index]) for index in indices) else None
+    return block_index if not rendered.isdisjoint(indices) else None
 
 
 def _system_may_follow(
     block_index: int,
-    messages: Sequence[AllMessageValues],
+    rendered: frozenset[int],
     blocks: Sequence[tuple[bool, tuple[int, ...]]],
 ) -> bool:
     """Whether a system message behind this block precedes an assistant turn or ends the array on the wire.
@@ -344,11 +455,7 @@ def _system_may_follow(
     first later block that puts anything on the wire is an assistant block.
     """
     return next(
-        (
-            not is_user
-            for is_user, indices in blocks[block_index + 1 :]
-            if any(_renders(messages[index]) for index in indices)
-        ),
+        (not is_user for is_user, indices in blocks[block_index + 1 :] if not rendered.isdisjoint(indices)),
         True,
     )
 
@@ -356,6 +463,7 @@ def _system_may_follow(
 def _anchor_block(
     run: Sequence[int],
     messages: Sequence[AllMessageValues],
+    rendered: frozenset[int],
     blocks: Sequence[tuple[bool, tuple[int, ...]]],
 ) -> int | None:
     """The user-type block a system run must follow, or ``None`` when it converts in place.
@@ -373,8 +481,8 @@ def _anchor_block(
     neighbour: Final = previous if _is_user_type(messages[previous]) else run[-1] + 1
     if neighbour >= len(messages) or not _is_user_type(messages[neighbour]):
         return None
-    block_index: Final = _rendered_block(neighbour, messages, blocks)
-    if block_index is None or not _system_may_follow(block_index, messages, blocks):
+    block_index: Final = _rendered_block(neighbour, rendered, blocks)
+    if block_index is None or not _system_may_follow(block_index, rendered, blocks):
         return None
     return block_index
 
@@ -389,7 +497,8 @@ def _placed_for_flagged_model(messages: Sequence[AllMessageValues]) -> tuple[All
     move it. Runs that share a user turn merge into one system message.
     """
     blocks: Final = _user_type_blocks(messages)
-    anchors: Final = tuple((run, _anchor_block(run, messages, blocks)) for run in _system_runs(messages))
+    rendered: Final = _rendered_indices(messages)
+    anchors: Final = tuple((run, _anchor_block(run, messages, rendered, blocks)) for run in _system_runs(messages))
 
     def messages_of(run: tuple[int, ...]) -> tuple[AllMessageValues, ...]:
         return tuple(messages[index] for index in run)

litellm/llms/anthropic/chat/transformation.py

@@ -146,6 +146,14 @@ else:
 # response side.
 _ANTHROPIC_TOOL_NAME_INVALID_CHARS: Final = re.compile(r"[^a-zA-Z0-9_-]")
 _ANTHROPIC_TOOL_NAME_MAX_LEN: Final = 128
+_DEFER_LOADING_ADAPTER: Final = TypeAdapter(bool)
+
+
+def _validated_defer_loading(value: object) -> bool:
+    try:
+        return _DEFER_LOADING_ADAPTER.validate_python(value, strict=True)
+    except ValidationError as exc:
+        raise ValueError("defer_loading must be a boolean") from exc
 
 
 class _AnthropicUsageIteration(TypedDict, total=False):
@@ -980,9 +988,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
                 "computer_20250124",
             ):
                 if _defer_loading is not None:
-                    if not isinstance(_defer_loading, bool):
-                        raise ValueError("defer_loading must be a boolean")
-                    returned_tool["defer_loading"] = _defer_loading
+                    returned_tool["defer_loading"] = _validated_defer_loading(_defer_loading)
                 elif _defer_loading_function is not None:
                     if not isinstance(_defer_loading_function, bool):
                         raise ValueError("defer_loading must be a boolean")

litellm/llms/anthropic/common_utils.py

@@ -28,11 +28,12 @@ from litellm.litellm_core_utils.prompt_templates.common_utils import (
     is_encrypted_reasoning_block,
     native_redacted_thinking_data,
 )
-from litellm.litellm_core_utils.prompt_templates.factory import (
-    THOUGHT_SIGNATURE_SEPARATOR,
+from litellm.litellm_core_utils.prompt_templates.factory import THOUGHT_SIGNATURE_SEPARATOR
+from litellm.litellm_core_utils.prompt_templates.mid_conversation_system import (
     find_anthropic_server_tool_result,
+    message_field,
+    parts_of,
 )
-from litellm.litellm_core_utils.prompt_templates.mid_conversation_system import message_field, parts_of
 from litellm.llms.anthropic.wif import (
     aget_anthropic_wif_token,
     anthropic_base_without_chat_suffix,

litellm/llms/openai/responses/guardrail_translation/handler.py

@@ -29,9 +29,10 @@ Output: response.output is List[GenericResponseOutputItem] where each has:
 """
 
 import copy
+import json
 import time
 import uuid
-from collections.abc import Mapping, Sequence
+from collections.abc import Mapping, MutableSequence, Sequence
 from dataclasses import dataclass
 from itertools import accumulate, chain, repeat
 from types import MappingProxyType
@@ -63,6 +64,8 @@ from litellm.llms.base_llm.guardrail_translation.utils import (
     unappliable_request_rewrite,
 )
 from litellm.llms.openai.responses.guardrail_translation.tool_merge import merge_guardrailed_tools
+from litellm.responses.litellm_completion_transformation.custom_tools import without_malformed_tool_flags
+from litellm.responses.litellm_completion_transformation.tool_search import is_server_tool_search_tool
 from litellm.responses.litellm_completion_transformation.transformation import (
     LiteLLMCompletionResponsesConfig,
 )
@@ -228,6 +231,10 @@ _TERMINAL_ENVELOPE_EVENT_TYPES: Final = frozenset(
 
 
 _TOOL_CALL_ITEM_TYPES: Final = frozenset({"function_call", "custom_tool_call"})
+_TOOL_SEARCH_CALL_TYPE: Final = "tool_search_call"
+_OUTPUT_TOOL_CALL_ITEM_TYPES: Final = _TOOL_CALL_ITEM_TYPES | {_TOOL_SEARCH_CALL_TYPE}
+_SCANNED_TOOL_SEARCH_NAME: Final = "tool_search"
+_TOOL_SEARCH_ARGUMENTS_ADAPTER: Final = TypeAdapter(dict[str, object])
 _TOOL_CALL_PAYLOAD_FIELDS: Final[Mapping[str, str]] = MappingProxyType(
     {"function_call": "arguments", "custom_tool_call": "input"}
 )
@@ -283,17 +290,61 @@ def _is_tool_call_item(item: object) -> bool:
 
 
 def _tool_call_output_item_mapping(item: object) -> Mapping[str, object] | None:
-    if stream_item_field(item, "type") not in _TOOL_CALL_ITEM_TYPES:
+    if stream_item_field(item, "type") not in _OUTPUT_TOOL_CALL_ITEM_TYPES:
         return None
     if isinstance(item, Mapping):
         return cast("Mapping[str, object]", item)  # cast-ok: output items are str-keyed JSON objects
     return item.model_dump() if isinstance(item, BaseModel) else None
 
 
+def _tool_call_item_key(item: object) -> str | None:
+    call_id: Final = stream_item_field(item, "call_id")
+    if isinstance(call_id, str) and call_id:
+        return call_id
+    item_id: Final = stream_item_field(item, "id")
+    if stream_item_field(item, "type") == _TOOL_SEARCH_CALL_TYPE and isinstance(item_id, str) and item_id:
+        return item_id
+    return None
+
+
 def _is_tool_call_output_item(item: object) -> bool:
     return _tool_call_output_item_mapping(item) is not None
 
 
+def _indexed_tool_call_output_items(outputs: Sequence[object]) -> tuple[tuple[int, object], ...]:
+    return tuple((index, item) for index, item in enumerate(outputs) if _is_tool_call_output_item(item))
+
+
+def _scanned_tool_call_item(item: Mapping[str, object]) -> Mapping[str, object]:
+    """A ``tool_search_call`` reaches guardrails as a ``tool_search`` call whose arguments are JSON text."""
+    if item.get("type") != _TOOL_SEARCH_CALL_TYPE:
+        return item
+    return {**item, "name": _SCANNED_TOOL_SEARCH_NAME, "arguments": json.dumps(item.get("arguments") or {})}
+
+
+def _tool_search_rewrite(payload: str) -> Mapping[str, object]:
+    try:
+        return {"arguments": _TOOL_SEARCH_ARGUMENTS_ADAPTER.validate_json(payload)}
+    except ValidationError:
+        return {"arguments": {}, "status": "incomplete"}
+
+
+def _rewritten_tool_search_call(item: object, payload: str | None) -> object:
+    if payload is None:
+        return item
+    update: Final = _tool_search_rewrite(payload)
+    if isinstance(item, BaseModel):
+        return item.model_copy(update=update)
+    if isinstance(item, Mapping):
+        return {**item, **update}
+    return item
+
+
+def _deliver_into_output(outputs: Sequence[object], index: int, item: object, delivered: object) -> None:
+    if delivered is not item and isinstance(outputs, MutableSequence):
+        outputs[index] = delivered  # rebind-ok: a frozen output item takes a rewrite by being swapped in the list
+
+
 def _released_tool_call_payload(responses_so_far: Sequence[object], item_id: object) -> str | None:
     events: Final = tuple(event for event in responses_so_far if stream_item_field(event, "item_id") == item_id)
     finished: Final = tuple(
@@ -315,7 +366,9 @@ def _released_tool_call_payload(responses_so_far: Sequence[object], item_id: obj
 
 
 def _with_released_payload(item: Mapping[str, object], responses_so_far: Sequence[object]) -> Mapping[str, object]:
-    field: Final = _TOOL_CALL_PAYLOAD_FIELDS[str(item.get("type"))]
+    field: Final = _TOOL_CALL_PAYLOAD_FIELDS.get(str(item.get("type")))
+    if field is None:
+        return item
     payload: Final = _released_tool_call_payload(responses_so_far, item.get("id"))
     return item if payload is None else {**item, field: payload}
 
@@ -567,8 +620,14 @@ class OpenAIResponsesHandler(BaseTranslation):
         original_tools: Final[tuple[Mapping[str, object], ...]] = (
             tuple(raw_tools) if isinstance(raw_tools, list) else ()
         )
+        scanned_tools: Final = tuple(without_malformed_tool_flags(tool) for tool in original_tools)
         flattened_tool_groups: Final = tuple(
-            form.chat_tools for form in LiteLLMCompletionResponsesConfig.responses_tools_to_chat_forms(original_tools)
+            () if is_server_tool_search_tool(tool) else form.chat_tools
+            for tool, form in zip(
+                scanned_tools,
+                LiteLLMCompletionResponsesConfig.responses_tools_to_chat_forms(scanned_tools),
+                strict=True,
+            )
         )
         extracted: Final = self._extract_guardrail_inputs(
             data, input_data, flattened_tool_groups, skip_system=skip_system
@@ -913,7 +972,7 @@ class OpenAIResponsesHandler(BaseTranslation):
                 task_mappings=task_mappings,
             )
             self._write_tool_call_rewrites_to_output(
-                tool_call_items=tuple(item for item in response_output if _is_tool_call_output_item(item)),
+                outputs=response_output,
                 pre_guardrail_tool_calls=pre_guardrail_tool_calls,
                 post_guardrail_tool_calls=post_guardrail_tool_calls,
             )
@@ -1148,7 +1207,7 @@ class OpenAIResponsesHandler(BaseTranslation):
         )
 
     @staticmethod
-    def _write_event_field(event: object, field: str, value: str) -> None:
+    def _write_event_field(event: object, field: str, value: object) -> None:
         if isinstance(event, dict):
             event[field] = value  # rebind-ok: delivering the rewrite means editing the buffered event in place
         else:
@@ -1214,7 +1273,8 @@ class OpenAIResponsesHandler(BaseTranslation):
     ) -> None:
         """Write ended-stream guardrail tool-call rewrites into the completed
         envelope's ``function_call`` and ``custom_tool_call`` items and sync the
-        earlier stream events, keyed by ``call_id``. The guardrail sees the
+        earlier stream events, keyed by ``call_id`` (a ``tool_search_call`` without one, as
+        OpenAI streams its server search, is keyed by its item ``id``). The guardrail sees the
         envelope's tool calls in output order, which is how a rewritten call
         finds its ``call_id``; the stream events find their call through the
         ``call_id`` on ``output_item`` events and the ``item_id`` on argument
@@ -1226,11 +1286,10 @@ class OpenAIResponsesHandler(BaseTranslation):
         pipeline executor discards it and releases the original events."""
         if post_guardrail_tool_calls == pre_guardrail_tool_calls:
             return
-        tool_call_items: Final = tuple(output_item for output_item in outputs if _is_tool_call_output_item(output_item))
+        indexed_tool_call_items: Final = _indexed_tool_call_output_items(outputs)
+        tool_call_items: Final = tuple(output_item for _, output_item in indexed_tool_call_items)
         call_ids: Final = tuple(
-            call_id
-            for output_item in tool_call_items
-            if isinstance(call_id := stream_item_field(output_item, "call_id"), str) and call_id
+            call_id for output_item in tool_call_items if (call_id := _tool_call_item_key(output_item)) is not None
         )
         stream_events: Final = responses_so_far[:-1]
         call_id_by_item_id: Final = self._tool_call_ids_by_item_id(stream_events)
@@ -1260,12 +1319,14 @@ class OpenAIResponsesHandler(BaseTranslation):
             from litellm.proxy.policy_engine.pipeline_executor import UndeliverableStreamRewrite
 
             raise UndeliverableStreamRewrite(guardrail_name, undeliverable_reason)
-        for output_item, rewrite in (
-            (output_item, rewrites_by_call_id[call_id])
-            for output_item, call_id in zip(tool_call_items, call_ids)
+        for (index, output_item), rewrite in (
+            (indexed_item, rewrites_by_call_id[call_id])
+            for indexed_item, call_id in zip(indexed_tool_call_items, call_ids)
             if call_id in rewrites_by_call_id
         ):
-            self._write_tool_call_item(output_item, rewrite.name, rewrite.arguments)
+            _deliver_into_output(
+                outputs, index, output_item, self._write_tool_call_item(output_item, rewrite.name, rewrite.arguments)
+            )
         delta_replacements: Final = MappingProxyType(
             {call_id: chain((rewrite.arguments,), repeat("")) for call_id, rewrite in rewrites_by_call_id.items()}
         )
@@ -1284,28 +1345,34 @@ class OpenAIResponsesHandler(BaseTranslation):
                         stream_item_field(event, "item"), rewrites_by_call_id[call_id].name, None
                     )
                 case "response.output_item.done":
-                    self._write_tool_call_item(
-                        stream_item_field(event, "item"),
-                        rewrites_by_call_id[call_id].name,
-                        rewrites_by_call_id[call_id].arguments,
+                    done_item = stream_item_field(event, "item")
+                    delivered_item = self._write_tool_call_item(
+                        done_item, rewrites_by_call_id[call_id].name, rewrites_by_call_id[call_id].arguments
                     )
+                    if delivered_item is not done_item:
+                        self._write_event_field(event, "item", delivered_item)
                 case _:
                     pass
 
     def _write_tool_call_rewrites_to_output(
         self,
-        tool_call_items: Sequence[object],
+        outputs: Sequence[object],
         pre_guardrail_tool_calls: tuple[_ToolCallShape, ...],
         post_guardrail_tool_calls: tuple[_ToolCallShape, ...],
     ) -> None:
-        if len(tool_call_items) != len(post_guardrail_tool_calls):
+        indexed_tool_call_items: Final = _indexed_tool_call_output_items(outputs)
+        if len(indexed_tool_call_items) != len(post_guardrail_tool_calls):
             return
-        for output_item, rewrite in (
-            (output_item, _tool_call_rewrite(before, after))
-            for output_item, before, after in zip(tool_call_items, pre_guardrail_tool_calls, post_guardrail_tool_calls)
+        for (index, output_item), rewrite in (
+            (indexed_item, _tool_call_rewrite(before, after))
+            for indexed_item, before, after in zip(
+                indexed_tool_call_items, pre_guardrail_tool_calls, post_guardrail_tool_calls
+            )
             if after != before
         ):
-            self._write_tool_call_item(output_item, rewrite.name, rewrite.arguments)
+            _deliver_into_output(
+                outputs, index, output_item, self._write_tool_call_item(output_item, rewrite.name, rewrite.arguments)
+            )
 
     @staticmethod
     def _tool_call_ids_by_item_id(stream_events: Sequence[object]) -> Mapping[str, str]:
@@ -1333,20 +1400,25 @@ class OpenAIResponsesHandler(BaseTranslation):
         if event_type not in _OUTPUT_ITEM_EVENT_TYPES:
             return None
         item: Final = stream_item_field(event, "item")
-        call_id: Final = stream_item_field(item, "call_id")
-        return (
-            call_id if stream_item_field(item, "type") in _TOOL_CALL_ITEM_TYPES and isinstance(call_id, str) else None
-        )
+        return _tool_call_item_key(item) if stream_item_field(item, "type") in _OUTPUT_TOOL_CALL_ITEM_TYPES else None
 
     @staticmethod
-    def _write_tool_call_item(item: object, name: str | None, payload: str | None) -> None:
+    def _write_tool_call_item(item: object, name: str | None, payload: str | None) -> object:
+        """Apply a rewrite and return the item that carries it.
+
+        A ``tool_search_call`` keeps its name and takes the rewritten arguments as a JSON
+        object on a copy, since its item may be frozen. Other tool calls are edited in place.
+        """
+        item_type: Final = stream_item_field(item, "type")
         if item is None:
-            return
+            return item
+        if item_type == _TOOL_SEARCH_CALL_TYPE:
+            return _rewritten_tool_search_call(item, payload)
         if name is not None:
             OpenAIResponsesHandler._write_event_field(item, "name", name)
-        item_type: Final = stream_item_field(item, "type")
         if payload is not None and isinstance(item_type, str) and item_type in _TOOL_CALL_PAYLOAD_FIELDS:
             OpenAIResponsesHandler._write_event_field(item, _TOOL_CALL_PAYLOAD_FIELDS[item_type], payload)
+        return item
 
     def _check_streaming_has_ended(self, responses_so_far: Sequence[object]) -> bool:
         """
@@ -1397,7 +1469,7 @@ class OpenAIResponsesHandler(BaseTranslation):
             stream_item_field(event, "type") in _TOOL_CALL_PAYLOAD_EVENT_TYPES
             or (
                 stream_item_field(event, "type") in _OUTPUT_ITEM_EVENT_TYPES
-                and stream_item_field(stream_item_field(event, "item"), "type") in _TOOL_CALL_ITEM_TYPES
+                and stream_item_field(stream_item_field(event, "item"), "type") in _OUTPUT_TOOL_CALL_ITEM_TYPES
             )
             for event in responses_so_far
         )
@@ -1406,7 +1478,7 @@ class OpenAIResponsesHandler(BaseTranslation):
     def _completed_response_scan_key(response: object) -> StreamingScanKey:
         output_items: Final = stream_item_items(response, "output")
         message_items: Final = tuple(
-            item for item in output_items if stream_item_field(item, "type") not in _TOOL_CALL_ITEM_TYPES
+            item for item in output_items if stream_item_field(item, "type") not in _OUTPUT_TOOL_CALL_ITEM_TYPES
         )
         return StreamingScanKey(
             texts=tuple(
@@ -1418,7 +1490,7 @@ class OpenAIResponsesHandler(BaseTranslation):
             tool_calls=tuple(
                 stream_item_fingerprint(item)
                 for item in output_items
-                if stream_item_field(item, "type") in _TOOL_CALL_ITEM_TYPES
+                if stream_item_field(item, "type") in _OUTPUT_TOOL_CALL_ITEM_TYPES
             ),
             stream_ended=True,
         )
@@ -1532,7 +1604,9 @@ class OpenAIResponsesHandler(BaseTranslation):
         tool_call_item: Final = _tool_call_output_item_mapping(output_item)
         if tool_call_item is not None:
             if tool_calls_to_check is not None:
-                tool_calls_to_check.append(tool_call_dict_from_output_item(tool_call_item, output_idx))
+                tool_calls_to_check.append(
+                    tool_call_dict_from_output_item(_scanned_tool_call_item(tool_call_item), output_idx)
+                )
             return
 
         # Handle both GenericResponseOutputItem and dict

litellm/model_prices_and_context_window_backup.json

@@ -61352,8 +61352,8 @@
             {
                 "name": "claude-tool-search",
                 "pattern": "claude-[a-z]+-(?:4[-._](?:[5-9]|[1-9]\\d)(?!\\d)|[5-9](?!\\d)(?:[-._]\\d{1,2}(?!\\d))?)",
-                "fill_missing_for_providers": ["anthropic", "bedrock", "bedrock_converse", "vertex_ai-anthropic_models"],
-                "description": "Claude at version 4.5 or higher, in any id shape that contains claude-<family>-: minors 4.5 through 4.99, any later major-minor, and bare 5+ majors so a new family like claude-fable-5 matches. Two-digit majors are deliberately not matched so ids like claude-opus-41 (4.1) are not read as major 41. Anthropic's tool search docs list every Claude 4.5 and newer model as supported and Opus 4.1 and earlier as unsupported, so the flag follows the version instead of a per-model list. azure_ai is left out on purpose: Anthropic documents tool search as unavailable on Azure-hosted Foundry deployments, and the azure_ai/ key cannot tell those from Anthropic-hosted ones.",
+                "fill_missing_for_providers": ["anthropic", "azure_ai", "bedrock", "bedrock_converse", "vertex_ai-anthropic_models"],
+                "description": "Claude at version 4.5 or higher, in any id shape that contains claude-<family>-: minors 4.5 through 4.99, any later major-minor, and bare 5+ majors so a new family like claude-fable-5 matches. Two-digit majors are deliberately not matched so ids like claude-opus-41 (4.1) are not read as major 41. Anthropic's tool search docs list every Claude 4.5 and newer model as supported and Opus 4.1 and earlier as unsupported, so the flag follows the version instead of a per-model list. azure_ai is included because Microsoft Foundry runs tool search for Claude on both Anthropic-hosted and Azure-hosted deployments.",
                 "model_info": {
                     "supports_tool_search": true
                 }

litellm/proxy/_lazy_openapi_snapshot.json

@@ -24368,6 +24368,60 @@
           "title": "OutputTokensDetails",
           "type": "object"
         },
+        "OutputToolSearchCall": {
+          "additionalProperties": true,
+          "properties": {
+            "arguments": {
+              "title": "Arguments"
+            },
+            "call_id": {
+              "anyOf": [
+                {
+                  "type": "string"
+                },
+                {
+                  "type": "null"
+                }
+              ],
+              "title": "Call Id"
+            },
+            "execution": {
+              "enum": [
+                "server",
+                "client"
+              ],
+              "title": "Execution",
+              "type": "string"
+            },
+            "id": {
+              "title": "Id",
+              "type": "string"
+            },
+            "status": {
+              "enum": [
+                "in_progress",
+                "completed",
+                "incomplete"
+              ],
+              "title": "Status",
+              "type": "string"
+            },
+            "type": {
+              "const": "tool_search_call",
+              "title": "Type",
+              "type": "string"
+            }
+          },
+          "required": [
+            "id",
+            "arguments",
+            "execution",
+            "status",
+            "type"
+          ],
+          "title": "OutputToolSearchCall",
+          "type": "object"
+        },
         "PendingSafetyCheck": {
           "additionalProperties": true,
           "description": "A pending safety check for the computer call.",
@@ -26734,6 +26788,9 @@
                       },
                       {
                         "$ref": "#/components/schemas/CustomToolCallOutputItem"
+                      },
+                      {
+                        "$ref": "#/components/schemas/OutputToolSearchCall"
                       }
                     ]
                   },
@@ -46523,6 +46580,10 @@
             "cache_control": {
               "$ref": "#/components/schemas/ChatCompletionCachedContent"
             },
+            "defer_loading": {
+              "title": "Defer Loading",
+              "type": "boolean"
+            },
             "eager_input_streaming": {
               "title": "Eager Input Streaming",
               "type": "boolean"

litellm/responses/litellm_completion_transformation/custom_tools.py

@@ -18,10 +18,11 @@ logic.
 import json
 from collections.abc import Mapping, Sequence
 from types import MappingProxyType
-from typing import Final
+from typing import Final, Literal, TypeAlias
 
 from pydantic import TypeAdapter, ValidationError
 
+from litellm.exceptions import BadRequestError
 from litellm.types.llms.base import LiteLLMBaseModel
 from litellm.types.llms.openai import (
     ChatCompletionToolParam,
@@ -30,7 +31,9 @@ from litellm.types.llms.openai import (
 
 _MAX_ARGUMENTS_LEN: Final = 1_000_000
 
-TOOL_CALL_ITEM_ID_PREFIX_BY_TYPE: Final = MappingProxyType({"function_call": "fc", "custom_tool_call": "ctc"})
+TOOL_CALL_ITEM_ID_PREFIX_BY_TYPE: Final = MappingProxyType(
+    {"function_call": "fc", "custom_tool_call": "ctc", "tool_search_call": "tsc"}
+)
 
 
 def openai_shaped_tool_call_item_id(item_type: str, tool_id: str) -> str:
@@ -157,14 +160,97 @@ class _CustomToolFormat(LiteLLMBaseModel):
     definition: str = ""
 
 
-_ALLOWED_CALLERS_ADAPTER: Final = TypeAdapter(list[str] | None)
+_ALLOWED_CALLERS_ADAPTER: Final = TypeAdapter[list[str] | None](list[str] | None)
 
 
 def validated_allowed_callers(value: object) -> list[str] | None:
     try:
         return _ALLOWED_CALLERS_ADAPTER.validate_python(value, strict=True)
     except ValidationError as exc:
-        raise ValueError("allowed_callers must be a list of strings") from exc
+        raise BadRequestError(message="allowed_callers must be a list of strings", model="", llm_provider="") from exc
+
+
+_DEFER_LOADING_ADAPTER: Final = TypeAdapter[bool | None](bool | None)
+
+
+def validated_defer_loading(value: object) -> bool:
+    try:
+        return bool(_DEFER_LOADING_ADAPTER.validate_python(value, strict=True))
+    except ValidationError as exc:
+        raise BadRequestError(message="defer_loading must be a boolean", model="", llm_provider="") from exc
+
+
+ToolSearchExecution: TypeAlias = Literal["server", "client"]
+_TOOL_SEARCH_EXECUTION_ADAPTER: Final = TypeAdapter[ToolSearchExecution | None](ToolSearchExecution | None)
+
+
+def validated_tool_search_execution(value: object) -> ToolSearchExecution | None:
+    try:
+        return _TOOL_SEARCH_EXECUTION_ADAPTER.validate_python(value, strict=True)
+    except ValidationError as exc:
+        raise BadRequestError(
+            message="tool_search execution must be 'server' or 'client'", model="", llm_provider=""
+        ) from exc
+
+
+_TOOL_FLAG_ADAPTERS: Final = MappingProxyType(
+    {
+        "defer_loading": _DEFER_LOADING_ADAPTER,
+        "allowed_callers": _ALLOWED_CALLERS_ADAPTER,
+        "execution": _TOOL_SEARCH_EXECUTION_ADAPTER,
+    }
+)
+
+
+def _is_well_formed_tool_flag(key: str, value: object) -> bool:
+    adapter: Final = _TOOL_FLAG_ADAPTERS.get(key)
+    if adapter is None:
+        return True
+    try:
+        adapter.validate_python(value, strict=True)
+    except ValidationError:
+        return False
+    return True
+
+
+_NAMESPACE_MEMBERS_ADAPTER: Final = TypeAdapter[list[object]](list[object])
+_TOOL_ADAPTER: Final = TypeAdapter[dict[str, object]](dict[str, object])
+
+
+def _with_well_formed_flags(tool: Mapping[str, object]) -> Mapping[str, object]:
+    return {key: value for key, value in tool.items() if _is_well_formed_tool_flag(key, value)}
+
+
+def _member_with_well_formed_flags(member: object) -> object:
+    try:
+        return _with_well_formed_flags(_TOOL_ADAPTER.validate_python(member))
+    except ValidationError:
+        return member
+
+
+def without_malformed_tool_flags(tool: Mapping[str, object]) -> Mapping[str, object]:
+    kept: Final = _with_well_formed_flags(tool)
+    if tool.get("type") != "namespace":
+        return kept
+    try:
+        members: Final = _NAMESPACE_MEMBERS_ADAPTER.validate_python(tool.get("tools"))
+    except ValidationError:
+        return kept
+    return {**kept, "tools": [_member_with_well_formed_flags(member) for member in members]}
+
+
+def function_chat_tool(
+    function: ChatCompletionToolParamFunctionChunk, allowed_callers: Sequence[str] | None, deferred: bool
+) -> ChatCompletionToolParam:
+    if allowed_callers is None and not deferred:
+        return ChatCompletionToolParam(type="function", function=function)
+    if allowed_callers is None:
+        return ChatCompletionToolParam(type="function", function=function, defer_loading=True)
+    if not deferred:
+        return ChatCompletionToolParam(type="function", function=function, allowed_callers=list(allowed_callers))
+    return ChatCompletionToolParam(
+        type="function", function=function, allowed_callers=list(allowed_callers), defer_loading=True
+    )
 
 
 def custom_tool_grammar_suffix(fmt: object) -> str:
@@ -183,8 +269,8 @@ def convert_custom_tool_to_function_tool(tool: Mapping[str, object]) -> ChatComp
 
     The grammar definition is embedded in the description so the model can
     produce correctly-formatted output. Returns ``None`` if the tool is not a
-    custom tool. Raises ``ValueError`` if ``allowed_callers`` is not a list of
-    strings.
+    custom tool. Raises ``BadRequestError`` if ``allowed_callers`` is not a list
+    of strings or ``defer_loading`` is not a boolean.
     """
     if tool.get("type") != "custom":
         return None
@@ -195,6 +281,7 @@ def convert_custom_tool_to_function_tool(tool: Mapping[str, object]) -> ChatComp
         tool.get("format")
     )
     allowed_callers: Final = validated_allowed_callers(tool.get("allowed_callers"))
+    deferred: Final = validated_defer_loading(tool.get("defer_loading"))
     function_chunk: Final = ChatCompletionToolParamFunctionChunk(
         name=name,
         description=description,
@@ -209,6 +296,4 @@ def convert_custom_tool_to_function_tool(tool: Mapping[str, object]) -> ChatComp
             "required": ["content"],
         },
     )
-    if allowed_callers is None:
-        return ChatCompletionToolParam(type="function", function=function_chunk)
-    return ChatCompletionToolParam(type="function", function=function_chunk, allowed_callers=allowed_callers)
+    return function_chat_tool(function_chunk, allowed_callers, deferred)

litellm/responses/litellm_completion_transformation/streaming_iterator.py

@@ -14,6 +14,11 @@ from litellm.responses.litellm_completion_transformation.custom_tools import (
     serialize_tool_call_arguments,
 )
 from litellm.responses.litellm_completion_transformation.reasoning_items import mint_reasoning_item_id
+from litellm.responses.litellm_completion_transformation.tool_search import (
+    build_tool_search_call,
+    has_server_tool_search,
+    is_anthropic_tool_search_use,
+)
 from litellm.responses.litellm_completion_transformation.transformation import (
     LiteLLMCompletionResponsesConfig,
 )
@@ -159,12 +164,54 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
         self._reasoning_item_id: str | None = None
         self._accumulated_reasoning_content_parts: list[str] = []
         self._accumulated_provider_specific_fields: dict[str, object] = {}
-        self._custom_tool_names: set[str] = extract_custom_tool_names(self.responses_api_request.get("tools"))
-        self._namespace_tool_names = LiteLLMCompletionResponsesConfig.namespace_tool_name_map(
-            self.responses_api_request.get("tools")
-        )
+        request_tools: Final = self.responses_api_request.get("tools")
+        self._custom_tool_names: set[str] = extract_custom_tool_names(request_tools)
+        self._namespace_tool_names = LiteLLMCompletionResponsesConfig.namespace_tool_name_map(request_tools)
         self._web_search_calls: dict[str, object] = {}  # mutable-ok: latest call by provider id
         self._queued_web_search_call_ids: set[str] = set()  # mutable-ok: emitted call ids
+        self._tool_search_requested: bool = has_server_tool_search(request_tools)
+        self._tool_search_call_ids: frozenset[str] = frozenset()
+
+    def _is_tool_search_call(self, call_id: str, fn_name: str) -> bool:
+        if call_id in self._tool_search_call_ids:
+            return True
+        if not self._tool_search_requested or not is_anthropic_tool_search_use(call_id, fn_name):
+            return False
+        self._tool_search_call_ids = self._tool_search_call_ids | {call_id}
+        return True
+
+    def _queue_output_item_added(self, output_index: int, item: BaseLiteLLMOpenAIResponseObject) -> None:
+        self._sequence_number += 1
+        added: Final = OutputItemAddedEvent(
+            type=ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED,
+            output_index=output_index,
+            item=item,
+        )
+        added.__dict__["sequence_number"] = self._sequence_number
+        self._pending_tool_events.append(added)
+
+    def _queue_tool_search_call_added(self, call_id: str, output_index: int, args_delta: str) -> None:
+        if call_id not in self._tool_args_by_call_id:
+            self._tool_args_by_call_id[call_id] = ""
+            item: Final = build_tool_search_call(call_id, "", "in_progress")
+            self._tool_item_id_by_call_id[call_id] = item.id
+            self._queue_output_item_added(output_index, BaseLiteLLMOpenAIResponseObject(**item.model_dump()))
+        self._tool_args_by_call_id[call_id] += args_delta
+
+    def _queue_tool_search_call_done(self, call_id: str, output_index: int, final_args: str) -> None:
+        self._queue_tool_search_call_added(call_id, output_index, "")
+        item: Final = build_tool_search_call(
+            call_id, final_args or self._tool_args_by_call_id.get(call_id, ""), "completed"
+        )
+        self._sequence_number += 1
+        self._pending_tool_events.append(
+            OutputItemDoneEvent(
+                type=ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE,
+                output_index=output_index,
+                sequence_number=self._sequence_number,
+                item=BaseLiteLLMOpenAIResponseObject(**item.model_dump()),
+            )
+        )
 
     def _get_or_assign_tool_output_index(self, call_id: str) -> int:
         existing: Final = self._tool_output_index_by_call_id.get(call_id)
@@ -309,18 +356,15 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
                 fn_args_delta = serialize_tool_call_arguments(getattr(fn, "arguments", ""))
             output_index = self._get_or_assign_tool_output_index(call_id)
 
+            if self._is_tool_search_call(call_id, fn_name):
+                self._queue_tool_search_call_added(call_id, output_index, fn_args_delta)
+                continue
+
             if call_id not in self._tool_args_by_call_id:
                 self._tool_args_by_call_id[call_id] = ""
-                self._sequence_number += 1
                 item_kwargs = self._tool_call_item_kwargs(call_id, fn_name, "", "in_progress")
                 self._tool_item_id_by_call_id[call_id] = item_kwargs["id"]
-                event = OutputItemAddedEvent(
-                    type=ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED,
-                    output_index=output_index,
-                    item=BaseLiteLLMOpenAIResponseObject(**item_kwargs),
-                )
-                event.__dict__["sequence_number"] = self._sequence_number
-                self._pending_tool_events.append(event)
+                self._queue_output_item_added(output_index, BaseLiteLLMOpenAIResponseObject(**item_kwargs))
 
             if fn_args_delta:
                 self._tool_args_by_call_id[call_id] += fn_args_delta
@@ -380,6 +424,9 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
                     self._queue_web_search_events(call_id, web_search_call)
                     self._queued_web_search_call_ids.add(call_id)
                 continue
+            if self._is_tool_search_call(call_id, fn_name):
+                self._queue_tool_search_call_done(call_id, output_index, fn_args)
+                continue
 
             # Track if this is a new tool call that wasn't streamed
             is_new_tool_call = call_id not in self._tool_item_id_by_call_id
@@ -387,16 +434,9 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
             # If we never sent output_item.added for this call_id, emit it now.
             if is_new_tool_call:
                 self._tool_args_by_call_id[call_id] = ""
-                self._sequence_number += 1
                 item_kwargs = self._tool_call_item_kwargs(call_id, fn_name, "", "in_progress")
                 self._tool_item_id_by_call_id[call_id] = item_kwargs["id"]
-                event = OutputItemAddedEvent(
-                    type=ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED,
-                    output_index=output_index,
-                    item=BaseLiteLLMOpenAIResponseObject(**item_kwargs),
-                )
-                event.__dict__["sequence_number"] = self._sequence_number
-                self._pending_tool_events.append(event)
+                self._queue_output_item_added(output_index, BaseLiteLLMOpenAIResponseObject(**item_kwargs))
 
             final_args = fn_args or self._tool_args_by_call_id.get(call_id, "")
 
@@ -450,11 +490,9 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
             else ResponseFunctionWebSearch.model_validate(web_search_call)
         )
         output_index: Final = self._get_or_assign_tool_output_index(call_id)
-        self._sequence_number += 1
-        added: Final = OutputItemAddedEvent(
-            type=ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED,
-            output_index=output_index,
-            item=BaseLiteLLMOpenAIResponseObject(
+        self._queue_output_item_added(
+            output_index,
+            BaseLiteLLMOpenAIResponseObject(
                 **{
                     "id": item.id,
                     "type": item.type,
@@ -463,8 +501,6 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
                 }
             ),
         )
-        added.__dict__["sequence_number"] = self._sequence_number
-        self._pending_tool_events.append(added)
         for event_type, event_class in (
             (ResponsesAPIStreamEvents.WEB_SEARCH_CALL_IN_PROGRESS, WebSearchCallInProgressEvent),
             (ResponsesAPIStreamEvents.WEB_SEARCH_CALL_SEARCHING, WebSearchCallSearchingEvent),
@@ -822,8 +858,8 @@ class LiteLLMCompletionStreamingIterator(ResponsesAPIStreamingIterator):
         if self._cached_item_id is None:
             self._cached_item_id = f"msg_{uuid.uuid4()}"
 
-        text: Final = self.litellm_model_response.choices[0].message.content or ""
-        annotations = getattr(self.litellm_model_response.choices[0].message, "annotations", None)
+        text: Final = litellm_complete_object.choices[0].message.content or ""
+        annotations = getattr(litellm_complete_object.choices[0].message, "annotations", None)
 
         response_annotations: Final = (
             LiteLLMCompletionResponsesConfig.transform_chat_completion_annotations_to_response_output_annotations(

litellm/responses/litellm_completion_transformation/tool_search.py

@@ -0,0 +1,186 @@
+from collections.abc import Iterable, Iterator, Mapping, Sequence, Set
+from typing import Final, Literal, TypeAlias
+
+from pydantic import ConfigDict, TypeAdapter, ValidationError
+
+from litellm.litellm_core_utils.prompt_templates.mid_conversation_system import is_anthropic_tool_search_use
+from litellm.types.llms.anthropic import ANTHROPIC_TOOL_SEARCH_TOOL_TYPES, AnthropicToolSearchToolRegex
+from litellm.types.llms.base import LiteLLMBaseModel
+from litellm.types.responses.main import OutputToolSearchCall
+from litellm.types.utils import LlmProviders, Message
+
+from .custom_tools import openai_shaped_tool_call_item_id
+
+__all__ = [
+    "ANTHROPIC_TOOL_SEARCH_TOOL_NAME",
+    "ANTHROPIC_TOOL_SEARCH_TOOL_TYPE",
+    "CompletionRoute",
+    "ToolSearchCallStatus",
+    "answers_tool_search",
+    "anthropic_tool_search_chat_tool",
+    "build_tool_search_call",
+    "has_server_tool_search",
+    "holds_tool_search_use",
+    "is_anthropic_tool_search_chat_tool",
+    "is_anthropic_tool_search_use",
+    "is_server_tool_search_tool",
+    "provider_builds_anthropic_request",
+    "tool_search_call_ids",
+    "without_tool_search_calls",
+    "without_tool_search_uses",
+]
+
+ANTHROPIC_TOOL_SEARCH_TOOL_TYPE: Final = "tool_search_tool_regex_20251119"
+ANTHROPIC_TOOL_SEARCH_TOOL_NAME: Final = "tool_search_tool_regex"
+ToolSearchCallStatus: TypeAlias = Literal["in_progress", "completed", "incomplete"]
+
+_ARGUMENTS_ADAPTER: Final = TypeAdapter(dict[str, object])
+
+
+class _ToolSearchFields(LiteLLMBaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    type: str = ""
+    execution: object = None
+
+
+def is_server_tool_search_tool(tool: object) -> bool:
+    try:
+        parsed: Final = _ToolSearchFields.model_validate(tool)
+    except ValidationError:
+        return False
+    return parsed.type == "tool_search" and parsed.execution != "client"
+
+
+def has_server_tool_search(tools: Sequence[object] | None) -> bool:
+    return any(is_server_tool_search_tool(tool) for tool in tools or ())
+
+
+def anthropic_tool_search_chat_tool() -> AnthropicToolSearchToolRegex:
+    return AnthropicToolSearchToolRegex(type=ANTHROPIC_TOOL_SEARCH_TOOL_TYPE, name=ANTHROPIC_TOOL_SEARCH_TOOL_NAME)
+
+
+def is_anthropic_tool_search_chat_tool(tool: Mapping[str, object]) -> bool:
+    return tool.get("type") in ANTHROPIC_TOOL_SEARCH_TOOL_TYPES
+
+
+def provider_builds_anthropic_request(model: str, custom_llm_provider: str | None) -> bool:
+    from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
+    from litellm.llms.anthropic.chat.transformation import AnthropicConfig
+    from litellm.llms.openai.chat.gpt_transformation import OpenAIGPTConfig
+    from litellm.utils import ProviderConfigManager
+
+    _, provider, _, _ = get_llm_provider(model=model, custom_llm_provider=custom_llm_provider)
+    try:
+        llm_provider: Final = LlmProviders(provider)
+    except ValueError:
+        return False
+    chat_config: Final = ProviderConfigManager.get_provider_chat_config(model=model, provider=llm_provider)
+    return isinstance(chat_config, AnthropicConfig) and not isinstance(chat_config, OpenAIGPTConfig)
+
+
+class CompletionRoute(LiteLLMBaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    model: str
+    custom_llm_provider: str | None = None
+
+
+class _ToolCallFunctionFields(LiteLLMBaseModel):
+    model_config = ConfigDict(frozen=True, from_attributes=True)
+
+    name: str | None = None
+
+
+class _ToolCallFields(LiteLLMBaseModel):
+    model_config = ConfigDict(frozen=True, from_attributes=True)
+
+    id: str | None = None
+    function: _ToolCallFunctionFields | None = None
+
+
+class _SessionMessageFields(LiteLLMBaseModel):
+    model_config = ConfigDict(frozen=True, from_attributes=True)
+
+    role: str | None = None
+    tool_calls: tuple[object, ...] | None = None
+    tool_call_id: str | None = None
+
+
+def _session_message_fields(message: object) -> _SessionMessageFields:
+    try:
+        return _SessionMessageFields.model_validate(message)
+    except ValidationError:
+        return _SessionMessageFields()
+
+
+def _tool_search_call_id(tool_call: object) -> str | None:
+    try:
+        parsed: Final = _ToolCallFields.model_validate(tool_call)
+    except ValidationError:
+        return None
+    tool_id: Final = parsed.id or ""
+    tool_name: Final = (parsed.function.name if parsed.function is not None else None) or ""
+    return tool_id if is_anthropic_tool_search_use(tool_id, tool_name) else None
+
+
+def _is_tool_search_call(tool_call: object) -> bool:
+    return _tool_search_call_id(tool_call) is not None
+
+
+def _assistant_tool_calls(message: object) -> tuple[object, ...]:
+    fields: Final = _session_message_fields(message)
+    return (fields.tool_calls or ()) if fields.role == "assistant" else ()
+
+
+def holds_tool_search_use(message: object) -> bool:
+    return any(_is_tool_search_call(tool_call) for tool_call in _assistant_tool_calls(message))
+
+
+def _tool_search_call_ids_of(messages: Iterable[object]) -> Iterator[str]:
+    for message in messages:
+        for tool_call in _assistant_tool_calls(message):
+            if (tool_id := _tool_search_call_id(tool_call)) is not None:
+                yield tool_id
+
+
+def tool_search_call_ids(messages: Iterable[object]) -> frozenset[str]:
+    return frozenset(_tool_search_call_ids_of(messages))
+
+
+def answers_tool_search(message: object, search_call_ids: Set[str]) -> bool:
+    fields: Final = _session_message_fields(message)
+    return fields.role == "tool" and fields.tool_call_id in search_call_ids
+
+
+def without_tool_search_calls(message: object) -> tuple[object, ...]:
+    return tuple(tool_call for tool_call in _assistant_tool_calls(message) if not _is_tool_search_call(tool_call))
+
+
+def without_tool_search_uses(message: Message) -> Message | None:
+    tool_calls: Final = message.tool_calls or []
+    kept: Final = tuple(tool_call for tool_call in tool_calls if not _is_tool_search_call(tool_call))
+    if len(kept) == len(tool_calls):
+        return message
+    if not kept and not message.content:
+        return None
+    return message.model_copy(update={"tool_calls": list(kept) or None})
+
+
+def _parsed_arguments(arguments_json: str) -> Mapping[str, object] | None:
+    try:
+        return _ARGUMENTS_ADAPTER.validate_json(arguments_json or "{}")
+    except ValidationError:
+        return None
+
+
+def build_tool_search_call(tool_id: str, arguments_json: str, status: ToolSearchCallStatus) -> OutputToolSearchCall:
+    arguments: Final = _parsed_arguments(arguments_json)
+    return OutputToolSearchCall(
+        id=openai_shaped_tool_call_item_id("tool_search_call", tool_id),
+        call_id=tool_id,
+        arguments={} if arguments is None else arguments,
+        execution="server",
+        status="incomplete" if arguments is None else status,
+        type="tool_search_call",
+    )

litellm/responses/litellm_completion_transformation/transformation.py

@@ -59,6 +59,7 @@ from litellm.llms.snowflake.chat.transformation import is_snowflake_claude_model
 from litellm.responses.litellm_completion_transformation.session_handler import (
     ResponsesSessionHandler,
 )
+from litellm.types.llms.anthropic import AnthropicToolSearchToolRegex
 from litellm.types.llms.base import CachedTokensDetails
 from litellm.types.llms.openai import (
     AllMessageValues,
@@ -102,6 +103,7 @@ from litellm.types.responses.main import (
     OutputFunctionToolCall,
     OutputImageGenerationCall,
     OutputText,
+    OutputToolSearchCall,
 )
 from litellm.types.utils import (
     ChatCompletionAnnotation,
@@ -116,18 +118,37 @@ from litellm.types.utils import (
 from .custom_tools import (
     convert_custom_tool_to_function_tool,
     extract_custom_tool_names,
+    function_chat_tool,
     is_custom_tool_call,
     openai_shaped_tool_call_item_id,
     serialize_tool_call_arguments,
     unwrap_custom_tool_arguments,
     validated_allowed_callers,
+    validated_defer_loading,
+    validated_tool_search_execution,
+    without_malformed_tool_flags,
 )
 from .reasoning_items import decode_thinking_blocks, encode_thinking_blocks, mint_reasoning_item_id
+from .tool_search import (
+    CompletionRoute,
+    answers_tool_search,
+    anthropic_tool_search_chat_tool,
+    build_tool_search_call,
+    has_server_tool_search,
+    holds_tool_search_use,
+    is_anthropic_tool_search_chat_tool,
+    is_anthropic_tool_search_use,
+    is_server_tool_search_tool,
+    provider_builds_anthropic_request,
+    tool_search_call_ids,
+    without_tool_search_calls,
+    without_tool_search_uses,
+)
 
 NamespaceNameMap: TypeAlias = Mapping[str, tuple[str, str]]
 NamespaceTool: TypeAlias = Mapping[str, object]
 ResponseTools: TypeAlias = Sequence[Mapping[str, object]] | None
-ChatToolParam: TypeAlias = ChatCompletionToolParam | OpenAIMcpServerTool
+ChatToolParam: TypeAlias = ChatCompletionToolParam | OpenAIMcpServerTool | AnthropicToolSearchToolRegex
 NAMESPACE_DESCRIPTION_SEPARATOR: Final = "\n\n"
 NAMESPACE_MEMBER_TYPES_WITH_CHAT_TOOLS: Final = frozenset({"function", "custom"})
 _INCOMPLETE_REASON_BY_FINISH_REASON: Final[Mapping[str, Literal["max_output_tokens", "content_filter"]]] = (
@@ -251,6 +272,35 @@ def _input_item_role(input_item: Mapping[str, object]) -> str:
     return cast(str, input_item.get("role") or "user")  # cast-ok: client-supplied role forwarded verbatim, unvalidated
 
 
+SessionMessage: TypeAlias = (
+    AllMessageValues
+    | GenericChatCompletionMessage
+    | ChatCompletionMessageToolCall
+    | ChatCompletionResponseMessage
+    | Message
+)
+
+
+def _replayed_assistant_without_tool_search_uses(message: Mapping[str, object]) -> SessionMessage | None:
+    kept_calls: Final = without_tool_search_calls(message)
+    if not kept_calls and not message.get("content"):
+        return None
+    without_calls: Final = {key: value for key, value in message.items() if key != "tool_calls"}
+    return cast(  # cast-ok: the replayed assistant message minus its search calls keeps every other field as stored
+        SessionMessage, {**without_calls, "tool_calls": list(kept_calls)} if kept_calls else without_calls
+    )
+
+
+def _without_stored_tool_search_uses(message: SessionMessage, search_call_ids: frozenset[str]) -> SessionMessage | None:
+    if isinstance(message, Message):
+        return without_tool_search_uses(message)
+    if answers_tool_search(message, search_call_ids):
+        return None
+    if isinstance(message, Mapping) and holds_tool_search_use(message):
+        return _replayed_assistant_without_tool_search_uses(message)
+    return message
+
+
 def _input_item_cache_control(input_item: Mapping[str, object]) -> ChatCompletionCachedContent | None:
     cache_control: Final = input_item.get("cache_control")
     if not cache_control:
@@ -504,13 +554,7 @@ def _replayed_messages_for_cache_points(
 
 
 class ChatCompletionSession(TypedDict, total=False):
-    messages: list[
-        AllMessageValues
-        | GenericChatCompletionMessage
-        | ChatCompletionMessageToolCall
-        | ChatCompletionResponseMessage
-        | Message
-    ]
+    messages: list[SessionMessage]
     litellm_session_id: str | None
     instructions: ReadOnly[str | None]
 
@@ -643,11 +687,60 @@ class LiteLLMCompletionResponsesConfig:
         )
         return supported_params is not None and "web_search_options" not in supported_params
 
+    @staticmethod
+    def _without_malformed_tool_flags_when_dropping_params(tools: ResponseTools, drop_params: object) -> ResponseTools:
+        if litellm.drop_params is not True and normalize_drop_params(drop_params) is not True:
+            return tools
+        return [without_malformed_tool_flags(tool) for tool in tools or ()]
+
+    @staticmethod
+    def _without_unsupported_tool_search(
+        tools: ResponseTools, model: str, custom_llm_provider: str | None
+    ) -> ResponseTools:
+        from litellm.utils import supports_tool_search
+
+        if not has_server_tool_search(tools):
+            return tools
+        if any(is_anthropic_tool_search_chat_tool(tool) for tool in tools or ()):
+            return [tool for tool in tools or () if not is_server_tool_search_tool(tool)]
+        route_runs_tool_search: Final = supports_tool_search(
+            model=model, custom_llm_provider=custom_llm_provider
+        ) and provider_builds_anthropic_request(model, custom_llm_provider)
+        if route_runs_tool_search:
+            return tools
+        verbose_logger.warning(
+            "Dropping Responses API tool of type 'tool_search': model '%s' (provider '%s') cannot run tool search "
+            "on this route, so the target provider would reject or ignore it.",
+            model,
+            custom_llm_provider,
+        )
+        return [tool for tool in tools or () if not is_server_tool_search_tool(tool)]
+
+    @staticmethod
+    def _deferral_the_route_honors(
+        tools: Sequence[ChatToolParam], callers_tools: ResponseTools, model: str, custom_llm_provider: str | None
+    ) -> Sequence[ChatToolParam]:
+        if not any("defer_loading" in tool for tool in tools):
+            return tools
+        if any(is_anthropic_tool_search_chat_tool(tool) for tool in tools):
+            return tools
+        if not has_server_tool_search(callers_tools) and provider_builds_anthropic_request(model, custom_llm_provider):
+            return tools
+        return [LiteLLMCompletionResponsesConfig._without_deferral(tool) for tool in tools]
+
+    @staticmethod
+    def _without_deferral(tool: ChatToolParam) -> ChatToolParam:
+        if "defer_loading" not in tool:
+            return tool
+        return cast(  # cast-ok: a TypedDict minus one key has no typed construction, and only its deferral changes
+            ChatToolParam, {key: value for key, value in tool.items() if key != "defer_loading"}
+        )
+
     @staticmethod
     def _completion_bridges_back_to_responses_api(
         model: str,
         custom_llm_provider: str | None,
-        tools: Sequence[ChatCompletionToolParam | OpenAIMcpServerTool] | None,
+        tools: Sequence[ChatToolParam] | None,
         web_search_options: OpenAIWebSearchOptions | None,
         reasoning_effort: str | None,
         reasoning_summary: str | None,
@@ -681,7 +774,7 @@ class LiteLLMCompletionResponsesConfig:
         reasoning_param: Reasoning | str | None,
         model: str,
         custom_llm_provider: str | None,
-        tools: Sequence[ChatCompletionToolParam | OpenAIMcpServerTool] | None = None,
+        tools: Sequence[ChatToolParam] | None = None,
         web_search_options: OpenAIWebSearchOptions | None = None,
         api_base: str | None = None,
     ) -> ResponsesReasoningChatForm:
@@ -724,16 +817,25 @@ class LiteLLMCompletionResponsesConfig:
         custom_llm_provider: str | None = None,
         stream: bool | None = None,
         extra_headers: Mapping[str, object] | None = None,
+        drop_params: object = None,
         **kwargs,
     ) -> dict:
         """
         Transform a Responses API request into a Chat Completion request
         """
+        request_tools: Final = LiteLLMCompletionResponsesConfig._without_unsupported_tool_search(
+            LiteLLMCompletionResponsesConfig._without_malformed_tool_flags_when_dropping_params(
+                responses_api_request.get("tools") or [], drop_params
+            ),
+            model=model,
+            custom_llm_provider=custom_llm_provider,
+        )
         (
-            tools,
+            converted_tools,
             web_search_options,
-        ) = LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools(
-            responses_api_request.get("tools") or []
+        ) = LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools(request_tools)
+        tools: Final = LiteLLMCompletionResponsesConfig._deferral_the_route_honors(
+            converted_tools, responses_api_request.get("tools"), model=model, custom_llm_provider=custom_llm_provider
         )
 
         if web_search_options is not None and LiteLLMCompletionResponsesConfig._should_drop_derived_web_search_options(
@@ -755,7 +857,7 @@ class LiteLLMCompletionResponsesConfig:
             api_base=kwargs.get("api_base"),
         )
 
-        request_kwargs: Final = _STR_KEY_DICT_ADAPTER.validate_python(kwargs)
+        request_kwargs: Final = _STR_KEY_DICT_ADAPTER.validate_python({**kwargs, "drop_params": drop_params})
         keeps_item_marks: Final = _target_keeps_item_marks(
             model, custom_llm_provider, drops_unsupported_params(request_kwargs.get("drop_params"))
         )
@@ -870,6 +972,19 @@ class LiteLLMCompletionResponsesConfig:
 
         return messages
 
+    @staticmethod
+    def _without_tool_search_uses_off_anthropic(
+        messages: Sequence[SessionMessage], route: CompletionRoute
+    ) -> tuple[SessionMessage, ...]:
+        if provider_builds_anthropic_request(route.model, route.custom_llm_provider):
+            return tuple(messages)
+        search_call_ids: Final = tool_search_call_ids(messages)
+        return tuple(
+            kept
+            for message in messages
+            if (kept := _without_stored_tool_search_uses(message, search_call_ids)) is not None
+        )
+
     @staticmethod
     async def async_responses_api_session_handler(
         previous_response_id: str,
@@ -898,7 +1013,14 @@ class LiteLLMCompletionResponsesConfig:
             else ChatCompletionSession(messages=[], litellm_session_id=None)
         )
         _messages: Final = litellm_completion_request.get("messages") or []
-        replayed_messages: Final = chat_completion_session.get("messages") or []
+        stored_messages: Final = chat_completion_session.get("messages") or []
+        replayed_messages: Final = (
+            LiteLLMCompletionResponsesConfig._without_tool_search_uses_off_anthropic(
+                stored_messages, CompletionRoute.model_validate(litellm_completion_request)
+            )
+            if any(holds_tool_search_use(message) for message in stored_messages)
+            else stored_messages
+        )
         session_messages: Final = (
             _replayed_messages_for_cache_points(
                 replayed_messages,
@@ -2286,10 +2408,11 @@ class LiteLLMCompletionResponsesConfig:
             parameters=dict(normalized_parameters),
             strict=bool(namespace_tool.get("strict", False)),
         )
-        allowed_callers: Final = validated_allowed_callers(namespace_tool.get("allowed_callers"))
-        if allowed_callers is None:
-            return ChatCompletionToolParam(type="function", function=function)
-        return ChatCompletionToolParam(type="function", function=function, allowed_callers=allowed_callers)
+        return function_chat_tool(
+            function,
+            validated_allowed_callers(namespace_tool.get("allowed_callers")),
+            validated_defer_loading(namespace_tool.get("defer_loading")),
+        )
 
     @staticmethod
     def _namespace_chat_tools(tool: NamespaceTool) -> tuple[ChatCompletionToolParam, ...]:
@@ -2372,10 +2495,11 @@ class LiteLLMCompletionResponsesConfig:
             }
             if tool.get("cache_control"):
                 chat_completion_tool["cache_control"] = tool.get("cache_control")
-            if tool.get("defer_loading"):
-                chat_completion_tool["defer_loading"] = tool.get("defer_loading")
-            if tool.get("allowed_callers"):
-                chat_completion_tool["allowed_callers"] = tool.get("allowed_callers")
+            if validated_defer_loading(tool.get("defer_loading")):
+                chat_completion_tool["defer_loading"] = True
+            function_allowed_callers: Final = validated_allowed_callers(tool.get("allowed_callers"))
+            if function_allowed_callers:
+                chat_completion_tool["allowed_callers"] = function_allowed_callers
             if tool.get("input_examples"):
                 chat_completion_tool["input_examples"] = tool.get("input_examples")
             if tool.get("eager_input_streaming") is not None:
@@ -2390,6 +2514,8 @@ class LiteLLMCompletionResponsesConfig:
         if tool_type == "custom":
             converted: Final = convert_custom_tool_to_function_tool(tool)
             return ResponsesToolChatForm(chat_tools=() if converted is None else (converted,), web_search_options=None)
+        if tool_type == "tool_search" and validated_tool_search_execution(tool.get("execution")) != "client":
+            return ResponsesToolChatForm(chat_tools=(anthropic_tool_search_chat_tool(),), web_search_options=None)
         if tool_type in ("computer_use", "image_generation", "local_shell", "shell", "tool_search"):
             verbose_logger.warning(
                 "Dropping Responses API tool of type '%s': it has no Chat Completions "
@@ -2408,7 +2534,7 @@ class LiteLLMCompletionResponsesConfig:
     def transform_responses_api_tools_to_chat_completion_tools(
         tools: ResponseTools,
     ) -> tuple[
-        list[ChatCompletionToolParam | OpenAIMcpServerTool],
+        list[ChatToolParam],
         OpenAIWebSearchOptions | None,
     ]:
         """
@@ -2506,13 +2632,15 @@ class LiteLLMCompletionResponsesConfig:
     def transform_chat_completion_tools_to_responses_tools(
         chat_completion_response: ModelResponse,
         responses_api_request: ResponsesAPIOptionalRequestParams | None = None,
-    ) -> list[ResponseFunctionToolCall | ResponseFunctionWebSearch | CustomToolCallOutputItem]:
+    ) -> list[ResponseFunctionToolCall | ResponseFunctionWebSearch | CustomToolCallOutputItem | OutputToolSearchCall]:
         """
         Transform a Chat Completion tools into a Responses API tools.
 
         For custom tools (e.g. apply_patch), returns CustomToolCallOutputItem
         with ``type="custom_tool_call"``. For regular function tools, returns
-        ``ResponseFunctionToolCall`` with ``type="function_call"``.
+        ``ResponseFunctionToolCall`` with ``type="function_call"``. A provider-side
+        tool search call made for a request's ``tool_search`` tool returns
+        ``OutputToolSearchCall`` with ``type="tool_search_call"``.
         """
         all_chat_completion_tools: Final[list[ChatCompletionMessageToolCall]] = []
         for choice in chat_completion_response.choices:
@@ -2528,12 +2656,13 @@ class LiteLLMCompletionResponsesConfig:
         request_tools: Final = responses_api_request.get("tools") if responses_api_request is not None else None
         custom_tool_names: Final = extract_custom_tool_names(request_tools)
         namespace_tool_names: Final = LiteLLMCompletionResponsesConfig.namespace_tool_name_map(request_tools)
+        tool_search_requested: Final = has_server_tool_search(request_tools)
 
         web_search_calls: Final = LiteLLMCompletionResponsesConfig._web_search_calls_by_call_id(
             chat_completion_response
         )
         responses_tools: Final[
-            list[ResponseFunctionToolCall | ResponseFunctionWebSearch | CustomToolCallOutputItem]
+            list[ResponseFunctionToolCall | ResponseFunctionWebSearch | CustomToolCallOutputItem | OutputToolSearchCall]
         ] = []
         for tool in all_chat_completion_tools:
             if tool.type == "function":
@@ -2542,9 +2671,12 @@ class LiteLLMCompletionResponsesConfig:
                 tool_id = tool.id or ""
                 tool_arguments = serialize_tool_call_arguments(function_definition.get("arguments"))
 
-                web_search_call = web_search_calls.get(tool_id)
-                if web_search_call is not None:
-                    responses_tools.append(web_search_call)
+                if (
+                    server_tool_call := LiteLLMCompletionResponsesConfig._server_tool_call_item(
+                        tool_id, tool_name, tool_arguments, web_search_calls, tool_search_requested
+                    )
+                ) is not None:
+                    responses_tools.append(server_tool_call)
                 elif is_custom_tool_call(tool_name, custom_tool_names):
                     # Build custom_tool_call output item
                     input_str = unwrap_custom_tool_arguments(tool_arguments)
@@ -2600,6 +2732,21 @@ class LiteLLMCompletionResponsesConfig:
                     responses_tools.append(output_tool_call)
         return responses_tools
 
+    @staticmethod
+    def _server_tool_call_item(
+        tool_id: str,
+        tool_name: str,
+        tool_arguments: str,
+        web_search_calls: Mapping[str, ResponseFunctionWebSearch],
+        tool_search_requested: bool,
+    ) -> ResponseFunctionWebSearch | OutputToolSearchCall | None:
+        web_search_call: Final = web_search_calls.get(tool_id)
+        if web_search_call is not None:
+            return web_search_call
+        if tool_search_requested and is_anthropic_tool_search_use(tool_id, tool_name):
+            return build_tool_search_call(tool_id, tool_arguments, "completed")
+        return None
+
     @staticmethod
     def _web_search_calls_by_call_id(
         chat_completion_response: ModelResponse,
@@ -2867,6 +3014,7 @@ class LiteLLMCompletionResponsesConfig:
         | ResponseFunctionToolCall
         | ResponseFunctionWebSearch
         | CustomToolCallOutputItem
+        | OutputToolSearchCall
     ]:
         responses_output: list[
             GenericResponseOutputItem
@@ -2876,6 +3024,7 @@ class LiteLLMCompletionResponsesConfig:
             | ResponseFunctionToolCall
             | ResponseFunctionWebSearch
             | CustomToolCallOutputItem
+            | OutputToolSearchCall
         ] = []
 
         responses_output.extend(

litellm/types/llms/anthropic.py

@@ -801,6 +801,8 @@ ANTHROPIC_TOOL_SEARCH_BETA_HEADER: Final = "advanced-tool-use-2025-11-20"
 ANTHROPIC_TOOL_SEARCH_TOOL_TYPES: Final = frozenset(
     {"tool_search_tool_regex_20251119", "tool_search_tool_bm25_20251119"}
 )
+ANTHROPIC_TOOL_SEARCH_TOOL_NAMES: Final = frozenset({"tool_search_tool_regex", "tool_search_tool_bm25"})
+ANTHROPIC_SERVER_TOOL_USE_ID_PREFIX: Final = "srvtoolu_"
 
 # Effort beta header constant
 ANTHROPIC_EFFORT_BETA_HEADER: Final = "effort-2025-11-24"

litellm/types/llms/openai.py

@@ -91,6 +91,7 @@ from litellm.types.responses.main import (
     OutputCodeInterpreterCall,
     OutputFunctionToolCall,
     OutputImageGenerationCall,
+    OutputToolSearchCall,
 )
 
 from .base import CachedTokensDetails
@@ -1076,6 +1077,7 @@ class ChatCompletionToolParam(OpenAIChatCompletionToolParam, total=False):
     cache_control: ChatCompletionCachedContent
     allowed_callers: list[str]
     eager_input_streaming: ReadOnly[bool]
+    defer_loading: ReadOnly[bool]
 
 
 class Function(TypedDict, total=False):
@@ -1425,6 +1427,17 @@ One of: completed, failed, in_progress, cancelled, queued, or incomplete.
 """
 
 
+_NULL_DROPPED_OUTPUT_ITEM_KEYS: Final = frozenset(
+    (
+        ("reasoning", "status"),
+        ("reasoning", "content"),
+        ("reasoning", "encrypted_content"),
+        ("tool_search_call", "created_by"),
+        ("tool_search_output", "created_by"),
+    )
+)
+
+
 class ResponsesAPIResponse(BaseLiteLLMOpenAIResponseObject):
     id: str
     created_at: int
@@ -1444,6 +1457,7 @@ class ResponsesAPIResponse(BaseLiteLLMOpenAIResponseObject):
             | ResponseFunctionToolCall
             | ResponseFunctionWebSearch
             | CustomToolCallOutputItem
+            | OutputToolSearchCall
         ]
     )
     parallel_tool_calls: bool | None = None
@@ -1513,8 +1527,12 @@ class ResponsesAPIResponse(BaseLiteLLMOpenAIResponseObject):
             return serialized
         return [
             (
-                {k: v for k, v in item.items() if v is not None or k not in ("status", "content", "encrypted_content")}
-                if isinstance(item, dict) and item.get("type") == "reasoning"
+                {
+                    k: v
+                    for k, v in item.items()
+                    if v is not None or (item.get("type"), k) not in _NULL_DROPPED_OUTPUT_ITEM_KEYS
+                }
+                if isinstance(item, dict)
                 else item
             )
             for item in serialized

litellm/types/responses/main.py

@@ -4,7 +4,7 @@ from typing import Final, Literal, Optional, Union
 
 from openai.types.responses.response_function_tool_call import ResponseFunctionToolCall
 from openai.types.responses.response_function_web_search import ActionSearchSource, ResponseFunctionWebSearch
-from pydantic import PrivateAttr
+from pydantic import ConfigDict, PrivateAttr
 from typing_extensions import Any, TypedDict
 
 from litellm.types.llms.base import BaseLiteLLMOpenAIResponseObject
@@ -133,6 +133,17 @@ class CustomToolCallOutputItem(BaseLiteLLMOpenAIResponseObject):
     status: Literal["in_progress", "completed", "incomplete"] | None = None
 
 
+class OutputToolSearchCall(BaseLiteLLMOpenAIResponseObject):
+    model_config = ConfigDict(extra="allow", frozen=True, protected_namespaces=())
+
+    id: str
+    arguments: object
+    call_id: str | None = None
+    execution: Literal["server", "client"]
+    status: Literal["in_progress", "completed", "incomplete"]
+    type: Literal["tool_search_call"]
+
+
 class GenericResponseOutputItem(BaseLiteLLMOpenAIResponseObject):
     """
     Generic response API output item

litellm/utils.py

@@ -2687,6 +2687,24 @@ def supports_web_search(model: str, custom_llm_provider: str | None = None) -> b
     )
 
 
+def supports_tool_search(model: str, custom_llm_provider: str | None = None) -> bool:
+    """
+    Check if the given model supports provider-side tool search over deferred tools.
+
+    Parameters:
+    model (str): The model name to be checked.
+    custom_llm_provider (str): The provider to be checked.
+
+    Returns:
+    bool: True if the model supports tool search, False otherwise.
+    """
+    return supports_factory(
+        model=model,
+        custom_llm_provider=custom_llm_provider,
+        key="supports_tool_search",
+    )
+
+
 def supports_url_context(model: str, custom_llm_provider: str | None = None) -> bool:
     """
     Check if the given model supports URL context and return a boolean value.

model_prices_and_context_window.json

@@ -61352,8 +61352,8 @@
             {
                 "name": "claude-tool-search",
                 "pattern": "claude-[a-z]+-(?:4[-._](?:[5-9]|[1-9]\\d)(?!\\d)|[5-9](?!\\d)(?:[-._]\\d{1,2}(?!\\d))?)",
-                "fill_missing_for_providers": ["anthropic", "bedrock", "bedrock_converse", "vertex_ai-anthropic_models"],
-                "description": "Claude at version 4.5 or higher, in any id shape that contains claude-<family>-: minors 4.5 through 4.99, any later major-minor, and bare 5+ majors so a new family like claude-fable-5 matches. Two-digit majors are deliberately not matched so ids like claude-opus-41 (4.1) are not read as major 41. Anthropic's tool search docs list every Claude 4.5 and newer model as supported and Opus 4.1 and earlier as unsupported, so the flag follows the version instead of a per-model list. azure_ai is left out on purpose: Anthropic documents tool search as unavailable on Azure-hosted Foundry deployments, and the azure_ai/ key cannot tell those from Anthropic-hosted ones.",
+                "fill_missing_for_providers": ["anthropic", "azure_ai", "bedrock", "bedrock_converse", "vertex_ai-anthropic_models"],
+                "description": "Claude at version 4.5 or higher, in any id shape that contains claude-<family>-: minors 4.5 through 4.99, any later major-minor, and bare 5+ majors so a new family like claude-fable-5 matches. Two-digit majors are deliberately not matched so ids like claude-opus-41 (4.1) are not read as major 41. Anthropic's tool search docs list every Claude 4.5 and newer model as supported and Opus 4.1 and earlier as unsupported, so the flag follows the version instead of a per-model list. azure_ai is included because Microsoft Foundry runs tool search for Claude on both Anthropic-hosted and Azure-hosted deployments.",
                 "model_info": {
                     "supports_tool_search": true
                 }

tests/integration/providers/test_responses_bridge_tool_search_replay_wire.py

@@ -0,0 +1,452 @@
+import asyncio
+import json
+import uuid
+from collections.abc import Iterator, Mapping, Sequence
+from dataclasses import dataclass
+from types import MappingProxyType
+from typing import Final, Literal
+from urllib.parse import unquote
+
+import httpx
+import pytest
+from integration._support import claude_code as cc
+from integration._support import responses_vendor as rv
+from integration._support.anthropic_thinking import BEDROCK_MODEL, EVENT_STREAM, Event, aws_chunks, sse_chunks
+from integration._support.client import Gateway, Scenario
+from integration._support.wire import Reply, Request, Wire, wire_server
+from pydantic import JsonValue, TypeAdapter
+
+_CLAUDE: Final = cc.OPUS
+_REGEX_TOOL_TYPE: Final = "tool_search_tool_regex_20251119"
+_REGEX_TOOL_NAME: Final = "tool_search_tool_regex"
+_REGEX_TOOL: Final[Mapping[str, JsonValue]] = MappingProxyType({"type": _REGEX_TOOL_TYPE, "name": _REGEX_TOOL_NAME})
+_BEDROCK_REGEX_TOOL: Final[Mapping[str, JsonValue]] = MappingProxyType(
+    {"type": _REGEX_TOOL_NAME, "name": _REGEX_TOOL_NAME}
+)
+_SEARCH_ARGUMENTS: Final = '{"pattern": "weather|stock"}'
+_SEARCH_OUTPUT: Final = '{"tools": ["get_weather"]}'
+_CACHE_BUST: Final[Mapping[str, JsonValue]] = MappingProxyType({"cache": {"no-cache": True}})
+_JSON_OBJECT: Final = TypeAdapter(dict[str, JsonValue])
+_ITEMS: Final = TypeAdapter(list[dict[str, JsonValue]])
+_EMPTY_TURN: Final[Mapping[str, JsonValue]] = MappingProxyType({"role": "user", "content": ""})
+_EARLIER_PAIRS: Final = 200
+_WEATHER_PARAMETERS: Final[dict[str, JsonValue]] = {
+    "type": "object",
+    "properties": {"city": {"type": "string"}},
+    "required": ["city"],
+}
+
+Provider = Literal["anthropic", "bedrock_invoke"]
+Answer = Literal["unanswered", "answered"]
+
+
+@dataclass(frozen=True, slots=True)
+class _Call:
+    provider: Provider
+    answer: Answer
+    stream: bool
+    marker: str
+
+
+@dataclass(frozen=True, slots=True)
+class _Served:
+    call: _Call
+    status: int
+    text: str
+
+
+def _prompt(marker: str) -> str:
+    return f"Weather in Paris and the ACME price marker-{marker}"
+
+
+def _message(marker: str) -> dict[str, JsonValue]:
+    return {
+        "id": f"msg_{marker}",
+        "type": "message",
+        "role": "assistant",
+        "model": _CLAUDE,
+        "content": [{"type": "text", "text": f"tomorrow is sunny marker-{marker}"}],
+        "stop_reason": "end_turn",
+        "stop_sequence": None,
+        "usage": {"input_tokens": 20, "output_tokens": 7},
+    }
+
+
+def _events(marker: str) -> tuple[Event, ...]:
+    opened: Final = {**_message(marker), "content": [], "stop_reason": None}
+    return (
+        {"type": "message_start", "message": opened},
+        {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
+        {
+            "type": "content_block_delta",
+            "index": 0,
+            "delta": {"type": "text_delta", "text": f"tomorrow is sunny marker-{marker}"},
+        },
+        {"type": "content_block_stop", "index": 0},
+        {
+            "type": "message_delta",
+            "delta": {"stop_reason": "end_turn", "stop_sequence": None},
+            "usage": {"output_tokens": 7},
+        },
+        {"type": "message_stop"},
+    )
+
+
+def _anthropic_error(status: int, message: str) -> Reply:
+    body: Final = {"type": "error", "error": {"type": "invalid_request_error", "message": message}}
+    return Reply(status=status, body=json.dumps(body).encode())
+
+
+def _block_ids(turn: Mapping[str, JsonValue], block_type: str, key: str) -> frozenset[str]:
+    content: Final = turn.get("content")
+    if not isinstance(content, list):
+        return frozenset()
+    blocks: Final = _ITEMS.validate_python(content)
+    return frozenset(str(block[key]) for block in blocks if block.get("type") == block_type)
+
+
+def _mismatch(messages: Sequence[Mapping[str, JsonValue]], index: int) -> frozenset[str]:
+    following: Final = messages[index + 1] if index + 1 < len(messages) else _EMPTY_TURN
+    return _block_ids(messages[index], "tool_use", "id") ^ _block_ids(following, "tool_result", "tool_use_id")
+
+
+def _unpaired(messages: Sequence[Mapping[str, JsonValue]]) -> frozenset[str]:
+    assistant_turns: Final = [index for index, turn in enumerate(messages) if turn["role"] == "assistant"]
+    return frozenset().union(*(_mismatch(messages, index) for index in assistant_turns))
+
+
+def _vendor(request: Request) -> Reply:
+    if request.method != "POST":
+        return Reply()
+    body: Final = _JSON_OBJECT.validate_json(request.body)
+    marker: Final = rv.newest_marker(request.body.decode()) or "health"
+    unpaired: Final = _unpaired(_ITEMS.validate_python(body.get("messages") or []))
+    if unpaired:
+        return _anthropic_error(400, f"messages: unpaired tool ids {sorted(unpaired)} marker-{marker}")
+    if request.target.endswith("/invoke-with-response-stream"):
+        return Reply(content_type=EVENT_STREAM, chunks=aws_chunks(_events(marker)))
+    if body.get("stream") is True:
+        return Reply(content_type="text/event-stream", chunks=sse_chunks(_events(marker)))
+    return Reply(body=json.dumps(_message(marker)).encode())
+
+
+def _deployment(scenario: Scenario, provider: Provider, wire_url: str) -> str:
+    match provider:
+        case "anthropic":
+            return scenario.model(model=f"anthropic/{_CLAUDE}", api_base=wire_url, api_key=cc.ANTHROPIC_API_KEY)
+        case "bedrock_invoke":
+            return scenario.model(
+                model=f"bedrock/invoke/{BEDROCK_MODEL}",
+                api_base=wire_url,
+                aws_access_key_id="AKIASCRIPTEDPROVIDER",
+                aws_secret_access_key="scripted-secret",
+                aws_region_name="us-east-1",
+                aws_bedrock_runtime_endpoint=wire_url,
+            )
+
+
+def _target(provider: Provider, stream: bool) -> str:
+    match provider:
+        case "anthropic":
+            return "/v1/messages"
+        case "bedrock_invoke":
+            return f"/model/{BEDROCK_MODEL}/invoke-with-response-stream" if stream else f"/model/{BEDROCK_MODEL}/invoke"
+
+
+def _search_call(marker: str) -> dict[str, JsonValue]:
+    return {
+        "id": f"srvtoolu_{marker}",
+        "type": "function",
+        "function": {"name": _REGEX_TOOL_NAME, "arguments": _SEARCH_ARGUMENTS},
+    }
+
+
+def _weather_call(marker: str) -> dict[str, JsonValue]:
+    return {
+        "id": f"toolu_{marker}",
+        "type": "function",
+        "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'},
+    }
+
+
+def _tool_message(tool_call_id: JsonValue, content: str) -> dict[str, JsonValue]:
+    return {"role": "tool", "tool_call_id": tool_call_id, "content": content}
+
+
+def _history(marker: str, answers: Sequence[Mapping[str, JsonValue]]) -> list[JsonValue]:
+    return [
+        {"role": "user", "content": _prompt(marker)},
+        {
+            "role": "assistant",
+            "content": f"searching marker-{marker}",
+            "tool_calls": [_search_call(marker), _weather_call(marker)],
+        },
+        *(dict(answer) for answer in answers),
+        _tool_message(f"toolu_{marker}", "22C"),
+        {"role": "user", "content": f"And tomorrow? marker-{marker}"},
+    ]
+
+
+def _answers(marker: str, answer: Answer) -> tuple[dict[str, JsonValue], ...]:
+    return (_tool_message(f"srvtoolu_{marker}", _SEARCH_OUTPUT),) if answer == "answered" else ()
+
+
+def _chat_tools() -> list[JsonValue]:
+    return [
+        dict(_REGEX_TOOL),
+        {
+            "type": "function",
+            "function": {"name": "get_weather", "description": "Weather for a city", "parameters": _WEATHER_PARAMETERS},
+            "defer_loading": True,
+        },
+    ]
+
+
+def _chat_body(model: str, marker: str, messages: Sequence[JsonValue], stream: bool) -> dict[str, JsonValue]:
+    return {"model": model, "messages": list(messages), "tools": _chat_tools(), "stream": stream, **_CACHE_BUST}
+
+
+def _raw(gateway: Gateway, path: str, body: Mapping[str, JsonValue]) -> httpx.Response:
+    with httpx.Client(base_url=str(gateway.client.base_url).rstrip("/"), trust_env=False, timeout=60) as client:
+        headers: Final = {"Authorization": f"Bearer {gateway.key}"}
+        with client.stream("POST", path, json=dict(body), headers=headers) as response:
+            response.read()
+            return response
+
+
+def _assert_served(response: httpx.Response, marker: str, stream: bool) -> None:
+    assert response.status_code == 200, response.text
+    if stream:
+        assert "data: [DONE]" in response.text and f"marker-{marker}" in response.text, response.text
+        return
+    choice: Final = _ITEMS.validate_python(_JSON_OBJECT.validate_json(response.content)["choices"])[0]
+    assert f"marker-{marker}" in str(_JSON_OBJECT.validate_python(choice["message"])["content"]), response.text
+
+
+def _only_request(wire: Wire) -> tuple[Request, dict[str, JsonValue]]:
+    received: Final = wire.drain()
+    assert len(received) == 1, [(request.method, request.target) for request in received]
+    return received[0], _JSON_OBJECT.validate_json(received[0].body)
+
+
+def _assistant_tool_uses(messages: Sequence[Mapping[str, JsonValue]]) -> list[tuple[str, str]]:
+    (assistant,) = [turn for turn in messages if turn["role"] == "assistant"]
+    blocks: Final = _ITEMS.validate_python(assistant["content"])
+    return sorted((str(block["id"]), str(block["name"])) for block in blocks if block["type"] == "tool_use")
+
+
+def _tool_results(messages: Sequence[Mapping[str, JsonValue]]) -> Iterator[str]:
+    for turn in messages:
+        if turn["role"] != "user" or not isinstance(turn["content"], list):
+            continue
+        yield from (
+            str(block["tool_use_id"])
+            for block in _ITEMS.validate_python(turn["content"])
+            if block["type"] == "tool_result"
+        )
+
+
+def _assert_tools_forwarded(provider: Provider, body: Mapping[str, JsonValue]) -> None:
+    tools: Final = _ITEMS.validate_python(body["tools"])
+    assert tools[0] == dict(_BEDROCK_REGEX_TOOL if provider == "bedrock_invoke" else _REGEX_TOOL), tools
+    assert [(tool["name"], tool.get("defer_loading")) for tool in tools[1:]] == [("get_weather", True)], tools
+
+
+def _assert_wire(call: _Call, request: Request, body: Mapping[str, JsonValue]) -> None:
+    assert unquote(request.target) == _target(call.provider, call.stream), request.target
+    _assert_tools_forwarded(call.provider, body)
+    messages: Final = _ITEMS.validate_python(body["messages"])
+    weather: Final = (f"toolu_{call.marker}", "get_weather")
+    search: Final = (f"srvtoolu_{call.marker}", _REGEX_TOOL_NAME)
+    match call.answer:
+        case "unanswered":
+            assert "srvtoolu_" not in json.dumps(messages), messages
+            assert _assistant_tool_uses(messages) == [weather], messages
+            assert sorted(_tool_results(messages)) == [weather[0]], messages
+        case "answered":
+            assert _assistant_tool_uses(messages) == [search, weather], messages
+            assert sorted(_tool_results(messages)) == [search[0], weather[0]], messages
+
+
+@pytest.mark.parametrize("stream", [False, True], ids=["sync", "stream"])
+@pytest.mark.parametrize("answer", ["unanswered", "answered"])
+@pytest.mark.parametrize("provider", ["anthropic", "bedrock_invoke"])
+def test_a_replayed_search_call_is_dropped_only_when_nothing_answers_it(
+    gateway: Gateway, provider: Provider, answer: Answer, stream: bool
+) -> None:
+    call: Final = _Call(provider, answer, stream, uuid.uuid4().hex)
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, provider, wire.url)
+        history: Final = _history(call.marker, _answers(call.marker, answer))
+        response: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, call.marker, history, stream))
+        _assert_served(response, call.marker, stream)
+        request, body = _only_request(wire)
+        _assert_wire(call, request, body)
+
+
+def _earlier_pairs(marker: str) -> Iterator[JsonValue]:
+    for index in range(_EARLIER_PAIRS):
+        call_id = f"toolu_{marker}_{index}"
+        call: JsonValue = {
+            "id": call_id,
+            "type": "function",
+            "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'},
+        }
+        yield {"role": "assistant", "content": f"step {index}", "tool_calls": [call]}
+        yield _tool_message(call_id, "22C")
+
+
+def _long_history(marker: str, answers: Sequence[Mapping[str, JsonValue]]) -> tuple[JsonValue, ...]:
+    first, *rest = _history(marker, answers)
+    return (first, *_earlier_pairs(marker), *rest)
+
+
+def _last_assistant_tool_uses(messages: Sequence[Mapping[str, JsonValue]]) -> tuple[tuple[str, str], ...]:
+    last: Final = [turn for turn in messages if turn["role"] == "assistant"][-1]
+    blocks: Final = _ITEMS.validate_python(last["content"])
+    return tuple(sorted((str(block["id"]), str(block["name"])) for block in blocks if block["type"] == "tool_use"))
+
+
+@pytest.mark.parametrize("answer", ["unanswered", "answered"])
+def test_a_search_call_after_two_hundred_answered_tool_turns_is_paired_only_by_its_own_answers(
+    gateway: Gateway, answer: Answer
+) -> None:
+    call: Final = _Call("anthropic", answer, False, uuid.uuid4().hex)
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "anthropic", wire.url)
+        history: Final = _long_history(call.marker, _answers(call.marker, answer))
+        response: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, call.marker, history, False))
+        _assert_served(response, call.marker, False)
+        request, body = _only_request(wire)
+        assert request.target == "/v1/messages", request.target
+        _assert_tools_forwarded("anthropic", body)
+        messages: Final = _ITEMS.validate_python(body["messages"])
+        earlier: Final = [f"toolu_{call.marker}_{index}" for index in range(_EARLIER_PAIRS)]
+        weather: Final = (f"toolu_{call.marker}", "get_weather")
+        search: Final = (f"srvtoolu_{call.marker}", _REGEX_TOOL_NAME)
+        match answer:
+            case "unanswered":
+                assert "srvtoolu_" not in json.dumps(messages), messages[-4:]
+                assert _last_assistant_tool_uses(messages) == (weather,), messages[-4:]
+                assert sorted(_tool_results(messages)) == sorted([*earlier, weather[0]]), messages[-4:]
+            case "answered":
+                assert _last_assistant_tool_uses(messages) == (search, weather), messages[-4:]
+                assert sorted(_tool_results(messages)) == sorted([*earlier, search[0], weather[0]]), messages[-4:]
+
+
+def test_a_function_role_answer_keeps_the_replayed_search_call(gateway: Gateway) -> None:
+    call: Final = _Call("anthropic", "answered", False, uuid.uuid4().hex)
+    answer: Final[dict[str, JsonValue]] = {
+        "role": "function",
+        "name": _REGEX_TOOL_NAME,
+        "tool_call_id": f"srvtoolu_{call.marker}",
+        "content": _SEARCH_OUTPUT,
+    }
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "anthropic", wire.url)
+        history: Final = _history(call.marker, (answer,))
+        response: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, call.marker, history, False))
+        _assert_served(response, call.marker, False)
+        request, body = _only_request(wire)
+        _assert_wire(call, request, body)
+
+
+def test_a_duplicated_answer_keeps_the_search_call_with_every_result(gateway: Gateway) -> None:
+    marker: Final = uuid.uuid4().hex
+    repeated: Final = _tool_message(f"srvtoolu_{marker}", _SEARCH_OUTPUT)
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "anthropic", wire.url)
+        history: Final = _history(marker, (repeated, repeated))
+        response: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, marker, history, False))
+        _assert_served(response, marker, False)
+        request, body = _only_request(wire)
+        assert request.target == "/v1/messages", request.target
+        messages: Final = _ITEMS.validate_python(body["messages"])
+        assert _assistant_tool_uses(messages) == [
+            (f"srvtoolu_{marker}", _REGEX_TOOL_NAME),
+            (f"toolu_{marker}", "get_weather"),
+        ]
+        assert sorted(_tool_results(messages)) == [f"srvtoolu_{marker}", f"srvtoolu_{marker}", f"toolu_{marker}"], (
+            messages
+        )
+
+
+@pytest.mark.parametrize("tool_call_id", ["", "x" * 5000], ids=["empty-string", "five-kilobyte-string"])
+def test_an_answer_whose_id_matches_nothing_leaves_the_search_call_dropped(gateway: Gateway, tool_call_id: str) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "anthropic", wire.url)
+        history: Final = _history(marker, (_tool_message(tool_call_id, _SEARCH_OUTPUT),))
+        response: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, marker, history, False))
+        assert response.status_code == 400 and "unpaired tool ids" in response.text, response.text
+        _, body = _only_request(wire)
+        messages: Final = _ITEMS.validate_python(body["messages"])
+        assert "srvtoolu_" not in json.dumps(messages), messages
+        assert _assistant_tool_uses(messages) == [(f"toolu_{marker}", "get_weather")], messages
+
+
+@pytest.mark.parametrize("tool_call_id", [5, ["srvtoolu_"]], ids=["int", "list"])
+def test_a_non_string_answer_id_is_refused_with_400_before_the_wire_and_the_proxy_keeps_serving(
+    gateway: Gateway, tool_call_id: JsonValue
+) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_vendor) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "anthropic", wire.url)
+        history: Final = _history(marker, (_tool_message(tool_call_id, _SEARCH_OUTPUT),))
+        refused: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, marker, history, False))
+        assert refused.status_code == 400, refused.text
+        assert wire.drain() == (), refused.text
+        follow_up: Final = uuid.uuid4().hex
+        history_served: Final = _history(follow_up, _answers(follow_up, "answered"))
+        served: Final = _raw(gateway, "/v1/chat/completions", _chat_body(model, follow_up, history_served, False))
+        _assert_served(served, follow_up, False)
+        assert len(wire.drain()) == 1
+
+
+async def _send(client: httpx.AsyncClient, key: str, model: str, call: _Call) -> _Served:
+    body: Final = _chat_body(model, call.marker, _history(call.marker, _answers(call.marker, call.answer)), call.stream)
+    async with client.stream(
+        "POST", "/v1/chat/completions", json=body, headers={"Authorization": f"Bearer {key}"}
+    ) as response:
+        raw: Final = await response.aread()
+    return _Served(call, response.status_code, raw.decode())
+
+
+async def _burst(
+    base_url: str, key: str, models: Mapping[Provider, str], calls: Sequence[_Call]
+) -> tuple[_Served, ...]:
+    async with httpx.AsyncClient(base_url=base_url, timeout=60, trust_env=False) as client:
+        return tuple(await asyncio.gather(*(_send(client, key, models[call.provider], call) for call in calls)))
+
+
+def _burst_calls(count: int) -> tuple[_Call, ...]:
+    providers: Final[tuple[Provider, ...]] = ("anthropic", "bedrock_invoke")
+    answers: Final[tuple[Answer, ...]] = ("unanswered", "answered")
+    return tuple(
+        _Call(providers[index % 2], answers[(index // 2) % 2], index % 4 >= 2, uuid.uuid4().hex)
+        for index in range(count)
+    )
+
+
+def _marked(received: Sequence[Request], marker: str) -> tuple[Request, dict[str, JsonValue]]:
+    (request,) = [request for request in received if marker in request.body.decode()]
+    return request, _JSON_OBJECT.validate_json(request.body)
+
+
+def test_a_mixed_burst_of_replays_pairs_every_answered_call_and_drops_every_unanswered_one(gateway: Gateway) -> None:
+    calls: Final = _burst_calls(12)
+    with wire_server(_vendor) as anthropic_wire, wire_server(_vendor) as bedrock_wire, gateway.scenario() as scenario:
+        models: Final[dict[Provider, str]] = {
+            "anthropic": _deployment(scenario, "anthropic", anthropic_wire.url),
+            "bedrock_invoke": _deployment(scenario, "bedrock_invoke", bedrock_wire.url),
+        }
+        served: Final = asyncio.run(_burst(str(gateway.client.base_url).rstrip("/"), gateway.key, models, calls))
+        assert [item.status for item in served] == [200] * len(calls), [
+            (item.status, item.text[:200]) for item in served
+        ]
+        received: Final = {"anthropic": anthropic_wire.drain(), "bedrock_invoke": bedrock_wire.drain()}
+        assert {provider: len(posts) for provider, posts in received.items()} == {"anthropic": 6, "bedrock_invoke": 6}
+        for item in served:
+            assert f"marker-{item.call.marker}" in item.text, item.text
+            request, body = _marked(received[item.call.provider], item.call.marker)
+            _assert_wire(item.call, request, body)

tests/integration/providers/test_responses_bridge_tool_search_undeferred_wire.py

@@ -0,0 +1,735 @@
+import asyncio
+import itertools
+import json
+import uuid
+from collections.abc import Callable, Mapping, Sequence
+from dataclasses import dataclass
+from types import MappingProxyType
+from typing import Final, Literal, TypeAlias
+from urllib.parse import unquote
+
+import httpx
+import pytest
+from _pytest.mark.structures import ParameterSet
+from integration._support import claude_code as cc
+from integration._support import responses_vendor as rv
+from integration._support.anthropic_thinking import BEDROCK_MODEL, Event, sse_chunks
+from integration._support.client import Gateway, Scenario
+from integration._support.wire import Reply, Request, Wire, wire_server
+from pydantic import JsonValue, TypeAdapter
+
+_CLAUDE: Final = cc.OPUS
+_CLAUDE_WITHOUT_TOOL_SEARCH: Final = "claude-scripted-unlisted"
+_GEMINI: Final = "gemini-2.5-flash"
+_MISTRAL: Final = "mistral-large-latest"
+_REGEX_TOOL_TYPE: Final = "tool_search_tool_regex_20251119"
+_REGEX_TOOL_NAME: Final = "tool_search_tool_regex"
+_SEARCH_TOOL_TYPES: Final = frozenset({_REGEX_TOOL_TYPE, "tool_search_tool_bm25_20251119"})
+_AZURE_MESSAGES_PATH: Final = "/anthropic/v1/messages"
+_ALL_DEFERRED_ERROR: Final = "At least one tool must have defer_loading=false"
+_DEFERRED_NAMES: Final = ("get_weather", "finance__get_stock_price")
+_CACHE_BUST: Final[Mapping[str, JsonValue]] = MappingProxyType({"cache": {"no-cache": True}})
+_JSON_OBJECT: Final = TypeAdapter(dict[str, JsonValue])
+_ITEMS: Final = TypeAdapter(list[dict[str, JsonValue]])
+_WEATHER_PARAMETERS: Final[dict[str, JsonValue]] = {
+    "type": "object",
+    "properties": {"city": {"type": "string"}},
+    "required": ["city"],
+}
+_TICKER_PARAMETERS: Final[dict[str, JsonValue]] = {
+    "type": "object",
+    "properties": {"ticker": {"type": "string"}},
+    "required": ["ticker"],
+}
+_EMAIL_PARAMETERS: Final[dict[str, JsonValue]] = {
+    "type": "object",
+    "properties": {"to": {"type": "string"}},
+    "required": ["to"],
+}
+
+Route = Literal["gemini", "bedrock_converse", "mistral", "unlisted_claude", "anthropic", "azure_ai"]
+Search = Literal["absent", "dropped"]
+NativeSearch: TypeAlias = Literal["regex", "bm25"]
+
+
+@dataclass(frozen=True, slots=True)
+class _Call:
+    route: Route
+    stream: bool
+    marker: str
+
+
+@dataclass(frozen=True, slots=True)
+class _Served:
+    call: _Call
+    status: int
+    text: str
+
+
+def _prompt(marker: str) -> str:
+    return f"Weather in Paris and the ACME price marker-{marker}"
+
+
+def _tool_search(**fields: JsonValue) -> dict[str, JsonValue]:
+    return {"type": "tool_search", **fields}
+
+
+def _native_search_tool(kind: NativeSearch) -> Mapping[str, JsonValue]:
+    return MappingProxyType({"type": f"tool_search_tool_{kind}_20251119", "name": f"tool_search_tool_{kind}"})
+
+
+def _function_tool(defer_loading: JsonValue = True) -> dict[str, JsonValue]:
+    return {
+        "type": "function",
+        "name": "get_weather",
+        "description": "Weather for a city",
+        "parameters": _WEATHER_PARAMETERS,
+        "defer_loading": defer_loading,
+    }
+
+
+def _loaded_tool() -> dict[str, JsonValue]:
+    return {"type": "function", "name": "send_email", "description": "Send an email", "parameters": _EMAIL_PARAMETERS}
+
+
+def _custom_tool() -> dict[str, JsonValue]:
+    return {"type": "custom", "name": "run_sql", "description": "Run a SQL query", "defer_loading": True}
+
+
+def _namespace_tool() -> dict[str, JsonValue]:
+    return {
+        "type": "namespace",
+        "name": "finance",
+        "description": "Finance tools",
+        "tools": [
+            {
+                "type": "function",
+                "name": "get_stock_price",
+                "description": "Price of a ticker",
+                "parameters": _TICKER_PARAMETERS,
+                "defer_loading": True,
+            }
+        ],
+    }
+
+
+def _search_tools(route: Route, search: Search) -> tuple[dict[str, JsonValue], ...]:
+    if search == "absent":
+        return ()
+    return (_tool_search(execution="client"),) if route == "anthropic" else (_tool_search(),)
+
+
+def _gemini_frame(marker: str) -> dict[str, JsonValue]:
+    return {
+        "candidates": [
+            {"content": {"role": "model", "parts": [{"text": f"loaded marker-{marker}"}]}, "finishReason": "STOP"}
+        ],
+        "usageMetadata": {"promptTokenCount": 5, "candidatesTokenCount": 3, "totalTokenCount": 8},
+        "modelVersion": _GEMINI,
+    }
+
+
+def _chat_completion(marker: str) -> dict[str, JsonValue]:
+    return {
+        "id": f"cmpl-{marker}",
+        "object": "chat.completion",
+        "created": 1,
+        "model": _MISTRAL,
+        "choices": [
+            {
+                "index": 0,
+                "message": {"role": "assistant", "content": f"loaded marker-{marker}"},
+                "finish_reason": "stop",
+            }
+        ],
+        "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},
+    }
+
+
+def _chat_chunks(marker: str) -> tuple[bytes, ...]:
+    head: Final[dict[str, JsonValue]] = {
+        "id": f"cmpl-{marker}",
+        "object": "chat.completion.chunk",
+        "created": 1,
+        "model": _MISTRAL,
+    }
+    return (
+        rv.chat_sse(
+            {
+                **head,
+                "choices": [
+                    {
+                        "index": 0,
+                        "delta": {"role": "assistant", "content": f"loaded marker-{marker}"},
+                        "finish_reason": None,
+                    }
+                ],
+            }
+        ),
+        rv.chat_sse(
+            {
+                **head,
+                "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
+                "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},
+            }
+        ),
+        b"data: [DONE]\n\n",
+    )
+
+
+def _claude_message(marker: str) -> dict[str, JsonValue]:
+    return {
+        "id": f"msg_{marker}",
+        "type": "message",
+        "role": "assistant",
+        "model": _CLAUDE,
+        "content": [{"type": "text", "text": f"loaded marker-{marker}"}],
+        "stop_reason": "end_turn",
+        "stop_sequence": None,
+        "usage": {"input_tokens": 20, "output_tokens": 7},
+    }
+
+
+def _claude_events(marker: str) -> tuple[Event, ...]:
+    opened: Final = {**_claude_message(marker), "content": [], "stop_reason": None}
+    return (
+        {"type": "message_start", "message": opened},
+        {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
+        {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": f"loaded marker-{marker}"}},
+        {"type": "content_block_stop", "index": 0},
+        {
+            "type": "message_delta",
+            "delta": {"stop_reason": "end_turn", "stop_sequence": None},
+            "usage": {"output_tokens": 7},
+        },
+        {"type": "message_stop"},
+    )
+
+
+def _anthropic_error(status: int, message: str) -> Reply:
+    body: Final = {"type": "error", "error": {"type": "invalid_request_error", "message": message}}
+    return Reply(status=status, body=json.dumps(body).encode())
+
+
+def _request_tools(body: Mapping[str, JsonValue]) -> list[dict[str, JsonValue]]:
+    return _ITEMS.validate_python(body.get("tools") or [])
+
+
+def _gemini(request: Request) -> Reply:
+    marker: Final = rv.newest_marker(request.body.decode()) or "health"
+    if f"/models/{_GEMINI}:streamGenerateContent" in request.target:
+        return Reply(content_type="text/event-stream", chunks=(rv.chat_sse(_gemini_frame(marker)),))
+    if f"/models/{_GEMINI}:generateContent" in request.target:
+        return Reply(body=json.dumps(_gemini_frame(marker)).encode())
+    return rv.error(404, f"unexpected target {request.target}", "not_found")
+
+
+def _converse(request: Request) -> Reply:
+    marker: Final = rv.newest_marker(request.body.decode()) or "health"
+    if unquote(request.target) != f"/model/{BEDROCK_MODEL}/converse":
+        return rv.error(404, f"unexpected target {request.target}", "not_found")
+    return Reply(
+        body=json.dumps(
+            {
+                "output": {"message": {"role": "assistant", "content": [{"text": f"converse marker-{marker}"}]}},
+                "stopReason": "end_turn",
+                "usage": {"inputTokens": 11, "outputTokens": 4, "totalTokens": 15},
+                "metrics": {"latencyMs": 1},
+            }
+        ).encode()
+    )
+
+
+def _mistral(request: Request) -> Reply:
+    marker: Final = rv.newest_marker(request.body.decode()) or "health"
+    if request.target != "/v1/chat/completions":
+        return rv.error(404, f"unexpected target {request.target}", "not_found")
+    body: Final = _JSON_OBJECT.validate_json(request.body)
+    deferred: Final = [index for index, tool in enumerate(_request_tools(body)) if "defer_loading" in tool]
+    if deferred:
+        return rv.error(
+            422, f"tools.{deferred[0]}.defer_loading: Extra inputs are not permitted marker-{marker}", "extra_forbidden"
+        )
+    if body.get("stream") is True:
+        return Reply(content_type="text/event-stream", chunks=_chat_chunks(marker))
+    return Reply(body=json.dumps(_chat_completion(marker)).encode())
+
+
+def _claude(request: Request) -> Reply:
+    if request.method != "POST":
+        return Reply()
+    marker: Final = rv.newest_marker(request.body.decode()) or "health"
+    body: Final = _JSON_OBJECT.validate_json(request.body)
+    tools: Final = _request_tools(body)
+    search_tools: Final = [tool for tool in tools if tool.get("type") in _SEARCH_TOOL_TYPES]
+    function_tools: Final = [tool for tool in tools if tool.get("type") not in _SEARCH_TOOL_TYPES]
+    if not search_tools and function_tools and all(tool.get("defer_loading") is True for tool in function_tools):
+        return _anthropic_error(400, f"tools: {_ALL_DEFERRED_ERROR} marker-{marker}")
+    if body.get("stream") is True:
+        return Reply(content_type="text/event-stream", chunks=sse_chunks(_claude_events(marker)))
+    return Reply(body=json.dumps(_claude_message(marker)).encode())
+
+
+def _vendor(route: Route) -> Callable[[Request], Reply]:
+    match route:
+        case "gemini":
+            return _gemini
+        case "bedrock_converse":
+            return _converse
+        case "mistral":
+            return _mistral
+        case "unlisted_claude" | "anthropic" | "azure_ai":
+            return _claude
+
+
+def _deployment(scenario: Scenario, route: Route, wire_url: str) -> str:
+    match route:
+        case "gemini":
+            return scenario.model(model=f"gemini/{_GEMINI}", api_base=wire_url, api_key="synthetic-gemini-key")
+        case "bedrock_converse":
+            return scenario.model(
+                model=f"bedrock/converse/{BEDROCK_MODEL}",
+                aws_access_key_id="AKIASCRIPTEDPROVIDER",
+                aws_secret_access_key="scripted-secret",
+                aws_region_name="us-east-1",
+                aws_bedrock_runtime_endpoint=wire_url,
+            )
+        case "mistral":
+            return scenario.model(model=f"mistral/{_MISTRAL}", api_base=wire_url, api_key="synthetic-mistral-key")
+        case "unlisted_claude":
+            return scenario.model(
+                model=f"anthropic/{_CLAUDE_WITHOUT_TOOL_SEARCH}", api_base=wire_url, api_key=cc.ANTHROPIC_API_KEY
+            )
+        case "anthropic":
+            return scenario.model(model=f"anthropic/{_CLAUDE}", api_base=wire_url, api_key=cc.ANTHROPIC_API_KEY)
+        case "azure_ai":
+            return scenario.model(
+                model=f"azure_ai/{_CLAUDE}", api_base=f"{wire_url}{_AZURE_MESSAGES_PATH}", api_key="synthetic-azure-key"
+            )
+
+
+def _responses_body(
+    model: str, marker: str, tools: Sequence[Mapping[str, JsonValue]], stream: bool
+) -> dict[str, JsonValue]:
+    return {
+        "model": model,
+        "input": _prompt(marker),
+        "tools": [dict(tool) for tool in tools],
+        "stream": stream,
+        **_CACHE_BUST,
+    }
+
+
+def _raw(gateway: Gateway, body: Mapping[str, JsonValue]) -> httpx.Response:
+    with httpx.Client(base_url=str(gateway.client.base_url).rstrip("/"), trust_env=False, timeout=60) as client:
+        headers: Final = {"Authorization": f"Bearer {gateway.key}"}
+        with client.stream("POST", "/v1/responses", json=dict(body), headers=headers) as response:
+            response.read()
+            return response
+
+
+def _assert_served(response: httpx.Response, marker: str, stream: bool) -> None:
+    assert response.status_code == 200, response.text
+    assert f"marker-{marker}" in response.text, response.text
+    if stream:
+        assert "response.completed" in response.text, response.text
+        return
+    types: Final = [
+        item["type"] for item in _ITEMS.validate_python(_JSON_OBJECT.validate_json(response.content)["output"])
+    ]
+    assert "message" in types, response.text
+
+
+def _only_request(wire: Wire) -> tuple[Request, dict[str, JsonValue]]:
+    received: Final = wire.drain()
+    assert len(received) == 1, [(request.method, request.target) for request in received]
+    return received[0], _JSON_OBJECT.validate_json(received[0].body)
+
+
+def _assert_target(route: Route, stream: bool, request: Request, body: Mapping[str, JsonValue]) -> None:
+    match route:
+        case "gemini":
+            method: Final = "streamGenerateContent" if stream else "generateContent"
+            assert f"/models/{_GEMINI}:{method}" in request.target, request.target
+        case "bedrock_converse":
+            assert unquote(request.target) == f"/model/{BEDROCK_MODEL}/converse", request.target
+        case "mistral":
+            assert request.target == "/v1/chat/completions", request.target
+            assert body.get("stream", False) is stream, body
+        case "unlisted_claude" | "anthropic":
+            assert request.target == "/v1/messages", request.target
+            assert body.get("stream", False) is stream, body
+        case "azure_ai":
+            assert request.target == _AZURE_MESSAGES_PATH, request.target
+            assert body.get("stream", False) is stream, body
+
+
+def _wire_tool_names(route: Route, body: Mapping[str, JsonValue]) -> list[str]:
+    match route:
+        case "gemini":
+            declarations: Final = _ITEMS.validate_python(
+                _ITEMS.validate_python(body["tools"])[0]["function_declarations"]
+            )
+            return [str(declaration["name"]) for declaration in declarations]
+        case "bedrock_converse":
+            specs: Final = _ITEMS.validate_python(_JSON_OBJECT.validate_python(body["toolConfig"])["tools"])
+            return [str(_JSON_OBJECT.validate_python(spec["toolSpec"])["name"]) for spec in specs]
+        case "mistral":
+            return [
+                str(_JSON_OBJECT.validate_python(tool["function"])["name"])
+                for tool in _ITEMS.validate_python(body["tools"])
+            ]
+        case "unlisted_claude" | "anthropic" | "azure_ai":
+            return [str(tool["name"]) for tool in _ITEMS.validate_python(body["tools"])]
+
+
+def _assert_undeferred_wire(call: _Call, request: Request, body: Mapping[str, JsonValue]) -> None:
+    _assert_target(call.route, call.stream, request, body)
+    assert _wire_tool_names(call.route, body) == list(_DEFERRED_NAMES), body
+    assert "defer_loading" not in request.body.decode(), body
+    assert "tool_search" not in request.body.decode(), body
+
+
+def _modes(route: Route) -> tuple[bool, ...]:
+    return (False,) if route == "bedrock_converse" else (False, True)
+
+
+_UNDEFERRING: Final[tuple[tuple[Route, Search], ...]] = (
+    ("gemini", "absent"),
+    ("gemini", "dropped"),
+    ("bedrock_converse", "absent"),
+    ("bedrock_converse", "dropped"),
+    ("mistral", "absent"),
+    ("mistral", "dropped"),
+    ("unlisted_claude", "dropped"),
+)
+_KEEPING: Final[tuple[tuple[Route, Search], ...]] = (
+    ("unlisted_claude", "absent"),
+    ("anthropic", "absent"),
+    ("anthropic", "dropped"),
+    ("azure_ai", "absent"),
+)
+
+
+def _cells(pairs: Sequence[tuple[Route, Search]]) -> tuple[ParameterSet, ...]:
+    combos: Final = [
+        (route, search, stream) for route, search in pairs for stream in _modes(route)
+    ]  # comprehension-ok: a parametrize grid
+    return tuple(
+        pytest.param(route, search, stream, id=f"{route}-{search}-{'stream' if stream else 'sync'}")
+        for route, search, stream in combos
+    )
+
+
+@pytest.mark.parametrize(("route", "search", "stream"), _cells(_UNDEFERRING))
+def test_without_a_search_tool_on_the_wire_every_tool_arrives_without_defer_loading(
+    gateway: Gateway, route: Route, search: Search, stream: bool
+) -> None:
+    call: Final = _Call(route, stream, uuid.uuid4().hex)
+    with wire_server(_vendor(route)) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (*_search_tools(route, search), _function_tool(), _namespace_tool())
+        response: Final = _raw(gateway, _responses_body(model, call.marker, tools, stream))
+        _assert_served(response, call.marker, stream)
+        request, body = _only_request(wire)
+        _assert_undeferred_wire(call, request, body)
+
+
+@pytest.mark.parametrize(("route", "search", "stream"), _cells(_KEEPING))
+def test_a_claude_route_sent_no_server_search_tool_keeps_each_tools_own_defer_loading(
+    gateway: Gateway, route: Route, search: Search, stream: bool
+) -> None:
+    call: Final = _Call(route, stream, uuid.uuid4().hex)
+    with wire_server(_vendor(route)) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (*_search_tools(route, search), _function_tool(), _namespace_tool(), _loaded_tool())
+        response: Final = _raw(gateway, _responses_body(model, call.marker, tools, stream))
+        _assert_served(response, call.marker, stream)
+        request, body = _only_request(wire)
+        _assert_target(route, stream, request, body)
+        assert "tool_search" not in request.body.decode(), body
+        sent: Final = _ITEMS.validate_python(body["tools"])
+        assert [(tool["name"], tool.get("defer_loading")) for tool in sent] == [
+            ("get_weather", True),
+            ("finance__get_stock_price", True),
+            ("send_email", None),
+        ], sent
+
+
+def test_a_false_defer_loading_never_reaches_an_openai_compatible_wire_as_a_key(gateway: Gateway) -> None:
+    call: Final = _Call("mistral", False, uuid.uuid4().hex)
+    with wire_server(_mistral) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "mistral", wire.url)
+        tools: Final = (_function_tool(defer_loading=False), _namespace_tool())
+        response: Final = _raw(gateway, _responses_body(model, call.marker, tools, False))
+        _assert_served(response, call.marker, False)
+        request, body = _only_request(wire)
+        _assert_undeferred_wire(call, request, body)
+
+
+@pytest.mark.parametrize("stream", [False, True], ids=["sync", "stream"])
+@pytest.mark.parametrize("route", ["anthropic", "azure_ai"])
+def test_the_supported_anthropic_route_keeps_each_tools_own_defer_loading_with_the_search_tool(
+    gateway: Gateway, route: Route, stream: bool
+) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_claude) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (_tool_search(), _function_tool(), _namespace_tool(), _loaded_tool())
+        response: Final = _raw(gateway, _responses_body(model, marker, tools, stream))
+        _assert_served(response, marker, stream)
+        request, body = _only_request(wire)
+        _assert_target(route, stream, request, body)
+        sent: Final = _ITEMS.validate_python(body["tools"])
+        assert sent[0] == {"type": _REGEX_TOOL_TYPE, "name": _REGEX_TOOL_NAME}, sent
+        assert [(tool["name"], tool.get("defer_loading")) for tool in sent[1:]] == [
+            ("get_weather", True),
+            ("finance__get_stock_price", True),
+            ("send_email", None),
+        ], sent
+        assert "defer_loading" not in json.dumps(sent[3]), sent
+
+
+def _native_cells() -> tuple[ParameterSet, ...]:
+    routes: Final[tuple[tuple[Route, NativeSearch], ...]] = (
+        ("anthropic", "regex"),
+        ("anthropic", "bm25"),
+        ("azure_ai", "regex"),
+    )
+    return tuple(
+        pytest.param(route, kind, stream, id=f"{route}-{kind}-{'stream' if stream else 'sync'}")
+        for (route, kind), stream in itertools.product(routes, (False, True))
+    )
+
+
+@pytest.mark.parametrize(("route", "kind", "stream"), _native_cells())
+def test_a_native_anthropic_search_tool_reaches_the_wire_untouched_and_keeps_every_deferral(
+    gateway: Gateway, route: Route, kind: NativeSearch, stream: bool
+) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_claude) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (_native_search_tool(kind), _function_tool(), _namespace_tool())
+        response: Final = _raw(gateway, _responses_body(model, marker, tools, stream))
+        _assert_served(response, marker, stream)
+        request, body = _only_request(wire)
+        _assert_target(route, stream, request, body)
+        sent: Final = _ITEMS.validate_python(body["tools"])
+        assert sent[0] == dict(_native_search_tool(kind)), sent
+        assert [(tool["name"], tool.get("defer_loading")) for tool in sent[1:]] == [
+            (name, True) for name in _DEFERRED_NAMES
+        ], sent
+
+
+@pytest.mark.parametrize("stream", [False, True], ids=["sync", "stream"])
+@pytest.mark.parametrize("route", ["anthropic", "mistral"])
+def test_a_deferred_custom_tool_keeps_its_deferral_only_while_a_search_tool_goes_out(
+    gateway: Gateway, route: Route, stream: bool
+) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_vendor(route)) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (_tool_search(), _custom_tool(), _loaded_tool())
+        response: Final = _raw(gateway, _responses_body(model, marker, tools, stream))
+        _assert_served(response, marker, stream)
+        request, body = _only_request(wire)
+        _assert_target(route, stream, request, body)
+        if route == "mistral":
+            assert _wire_tool_names(route, body) == ["run_sql", "send_email"], body
+            assert "defer_loading" not in request.body.decode(), body
+            assert "tool_search" not in request.body.decode(), body
+            return
+        sent: Final = _ITEMS.validate_python(body["tools"])
+        assert sent[0] == {"type": _REGEX_TOOL_TYPE, "name": _REGEX_TOOL_NAME}, sent
+        assert [(tool["name"], tool.get("defer_loading")) for tool in sent[1:]] == [
+            ("run_sql", True),
+            ("send_email", None),
+        ], sent
+
+
+def test_bedrock_converse_skips_a_native_search_tool_and_serves_the_deferred_tools(gateway: Gateway) -> None:
+    call: Final = _Call("bedrock_converse", False, uuid.uuid4().hex)
+    with wire_server(_converse) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, "bedrock_converse", wire.url)
+        tools: Final = (_native_search_tool("regex"), _function_tool(), _namespace_tool())
+        response: Final = _raw(gateway, _responses_body(model, call.marker, tools, False))
+        _assert_served(response, call.marker, False)
+        request, body = _only_request(wire)
+        _assert_undeferred_wire(call, request, body)
+
+
+@pytest.mark.parametrize("route", ["gemini", "mistral"])
+def test_a_non_boolean_defer_loading_is_refused_with_400_on_a_route_without_tool_search(
+    gateway: Gateway, route: Route
+) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_vendor(route)) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        refused: Final = _raw(
+            gateway, _responses_body(model, marker, (_function_tool("true"), _namespace_tool()), False)
+        )
+        assert refused.status_code == 400 and "defer_loading must be a boolean" in refused.text, refused.text
+        assert wire.drain() == (), refused.text
+        follow_up: Final = _Call(route, False, uuid.uuid4().hex)
+        served: Final = _raw(
+            gateway, _responses_body(model, follow_up.marker, (_function_tool(), _namespace_tool()), False)
+        )
+        _assert_served(served, follow_up.marker, False)
+        request, body = _only_request(wire)
+        _assert_undeferred_wire(follow_up, request, body)
+
+
+def _text_parts(marker: str) -> tuple[str, ...]:
+    return ("part one ", "part two ", f"marker-{marker}")
+
+
+def _split_claude_events(marker: str) -> tuple[Event, ...]:
+    opened: Final = {**_claude_message(marker), "content": [], "stop_reason": None}
+    deltas: Final[tuple[Event, ...]] = tuple(
+        {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": part}}
+        for part in _text_parts(marker)
+    )
+    return (
+        {"type": "message_start", "message": opened},
+        {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
+        *deltas,
+        {"type": "content_block_stop", "index": 0},
+        {
+            "type": "message_delta",
+            "delta": {"stop_reason": "end_turn", "stop_sequence": None},
+            "usage": {"output_tokens": 7},
+        },
+        {"type": "message_stop"},
+    )
+
+
+def _split_chat_chunks(marker: str) -> tuple[bytes, ...]:
+    head: Final[Mapping[str, JsonValue]] = {
+        "id": f"cmpl-{marker}",
+        "object": "chat.completion.chunk",
+        "created": 1,
+        "model": _MISTRAL,
+    }
+    parts: Final = tuple(
+        rv.chat_sse(
+            {
+                **head,
+                "choices": [{"index": 0, "delta": {"role": "assistant", "content": part}, "finish_reason": None}],
+            }
+        )
+        for part in _text_parts(marker)
+    )
+    return (
+        *parts,
+        rv.chat_sse(
+            {
+                **head,
+                "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
+                "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},
+            }
+        ),
+        b"data: [DONE]\n\n",
+    )
+
+
+def _split_vendor(route: Route) -> Callable[[Request], Reply]:
+    def respond(request: Request) -> Reply:
+        if request.method != "POST":
+            return Reply()
+        marker: Final = rv.newest_marker(request.body.decode()) or "health"
+        if route == "mistral":
+            return Reply(content_type="text/event-stream", chunks=_split_chat_chunks(marker))
+        return Reply(content_type="text/event-stream", chunks=sse_chunks(_split_claude_events(marker)))
+
+    return respond
+
+
+def _frames(text: str) -> tuple[dict[str, JsonValue], ...]:
+    return tuple(_JSON_OBJECT.validate_json(line[6:]) for line in text.splitlines() if line.startswith("data: {"))
+
+
+def _message_texts(items: Sequence[Mapping[str, JsonValue]]) -> tuple[str, ...]:
+    return tuple(
+        "".join(str(part["text"]) for part in _ITEMS.validate_python(item["content"]))
+        for item in items
+        if item["type"] == "message"
+    )
+
+
+@pytest.mark.parametrize("route", ["anthropic", "mistral"])
+def test_a_streamed_answer_closes_its_message_item_with_every_streamed_delta(gateway: Gateway, route: Route) -> None:
+    marker: Final = uuid.uuid4().hex
+    with wire_server(_split_vendor(route)) as wire, gateway.scenario() as scenario:
+        model: Final = _deployment(scenario, route, wire.url)
+        tools: Final = (_tool_search(), _function_tool(), _loaded_tool())
+        response: Final = _raw(gateway, _responses_body(model, marker, tools, True))
+        assert response.status_code == 200, response.text
+        frames: Final = _frames(response.text)
+        expected: Final = "".join(_text_parts(marker))
+        deltas: Final = [str(frame["delta"]) for frame in frames if frame.get("type") == "response.output_text.delta"]
+        assert "".join(deltas) == expected, frames
+        done: Final = [
+            _JSON_OBJECT.validate_python(frame["item"])
+            for frame in frames
+            if frame.get("type") == "response.output_item.done"
+        ]
+        assert _message_texts(done) == (expected,), done
+        (completed,) = [frame for frame in frames if frame.get("type") == "response.completed"]
+        output: Final = _ITEMS.validate_python(_JSON_OBJECT.validate_python(completed["response"])["output"])
+        assert _message_texts(output) == (expected,), output
+        request, body = _only_request(wire)
+        _assert_target(route, True, request, body)
+
+
+async def _send(client: httpx.AsyncClient, key: str, model: str, call: _Call) -> _Served:
+    tools: Final = (*_search_tools(call.route, "dropped"), _function_tool(), _namespace_tool())
+    body: Final = _responses_body(model, call.marker, tools, call.stream)
+    async with client.stream(
+        "POST", "/v1/responses", json=body, headers={"Authorization": f"Bearer {key}"}
+    ) as response:
+        raw: Final = await response.aread()
+    return _Served(call, response.status_code, raw.decode())
+
+
+async def _burst(base_url: str, key: str, models: Mapping[Route, str], calls: Sequence[_Call]) -> tuple[_Served, ...]:
+    async with httpx.AsyncClient(base_url=base_url, timeout=60, trust_env=False) as client:
+        return tuple(await asyncio.gather(*(_send(client, key, models[call.route], call) for call in calls)))
+
+
+def _burst_calls() -> tuple[_Call, ...]:
+    routes: Final[tuple[Route, ...]] = ("gemini", "bedrock_converse", "mistral", "unlisted_claude")
+    return tuple(
+        _Call(route, stream, uuid.uuid4().hex) for route in routes for stream in _modes(route)
+    )  # comprehension-ok: a two-axis burst grid
+
+
+def _marked(received: Sequence[Request], marker: str) -> tuple[Request, dict[str, JsonValue]]:
+    (request,) = [request for request in received if marker in request.body.decode()]
+    return request, _JSON_OBJECT.validate_json(request.body)
+
+
+def test_a_burst_across_four_routes_strips_every_deferral(gateway: Gateway) -> None:
+    calls: Final = _burst_calls()
+    with (
+        wire_server(_gemini) as gemini_wire,
+        wire_server(_converse) as converse_wire,
+        wire_server(_mistral) as mistral_wire,
+        wire_server(_claude) as claude_wire,
+        gateway.scenario() as scenario,
+    ):
+        wires: Final[dict[Route, Wire]] = {
+            "gemini": gemini_wire,
+            "bedrock_converse": converse_wire,
+            "mistral": mistral_wire,
+            "unlisted_claude": claude_wire,
+        }
+        models: Final = {route: _deployment(scenario, route, wire.url) for route, wire in wires.items()}
+        served: Final = asyncio.run(_burst(str(gateway.client.base_url).rstrip("/"), gateway.key, models, calls))
+        assert [item.status for item in served] == [200] * len(calls), [
+            (item.status, item.text[:200]) for item in served
+        ]
+        received: Final = {route: wire.drain() for route, wire in wires.items()}
+        for item in served:
+            assert f"marker-{item.call.marker}" in item.text, item.text
+            request, body = _marked(received[item.call.route], item.call.marker)
+            _assert_undeferred_wire(item.call, request, body)

tests/unit/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py

@@ -3,6 +3,7 @@ import json
 import logging
 import os
 import re
+import time
 from typing import Final, List
 from unittest.mock import MagicMock, patch
 
@@ -7308,3 +7309,90 @@ def test_completion_bedrock_invalid_role_exception(monkeypatch):
         str(exc_info.value)
         == "litellm.BadRequestError: Invalid Message passed in {'role': 'very-bad-role', 'content': 'hello'}"
     )
+
+
+def test_anthropic_messages_pt_drops_replayed_tool_search_call_without_its_result():
+    messages: Final = [
+        {"role": "user", "content": "What is the weather in Paris?"},
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [
+                {
+                    "id": "srvtoolu_01Search",
+                    "type": "function",
+                    "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+                },
+                {
+                    "id": "toolu_01Weather",
+                    "type": "function",
+                    "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'},
+                },
+            ],
+        },
+        {"role": "tool", "tool_call_id": "toolu_01Weather", "content": "18C and cloudy"},
+        {"role": "user", "content": "And tomorrow?"},
+    ]
+
+    result: Final = anthropic_messages_pt(messages=messages, model="claude-sonnet-5-5", llm_provider="anthropic")
+
+    assert [(block["type"], block["id"]) for block in result[1]["content"]] == [("tool_use", "toolu_01Weather")]
+    assert [block["tool_use_id"] for block in result[2]["content"] if block["type"] == "tool_result"] == [
+        "toolu_01Weather"
+    ]
+
+
+def test_anthropic_messages_pt_keeps_a_replayed_tool_search_call_the_caller_answered():
+    messages: Final = [
+        {"role": "user", "content": "What is the weather in Paris?"},
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [
+                {
+                    "id": "srvtoolu_01Search",
+                    "type": "function",
+                    "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+                },
+                {
+                    "id": "toolu_01Weather",
+                    "type": "function",
+                    "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'},
+                },
+            ],
+        },
+        {"role": "tool", "tool_call_id": "srvtoolu_01Search", "content": '[{"name": "get_weather"}]'},
+        {"role": "tool", "tool_call_id": "toolu_01Weather", "content": "18C and cloudy"},
+    ]
+
+    result: Final = anthropic_messages_pt(messages=messages, model="claude-sonnet-5-5", llm_provider="anthropic")
+
+    assert [(block["type"], block["id"]) for block in result[1]["content"]] == [
+        ("tool_use", "srvtoolu_01Search"),
+        ("tool_use", "toolu_01Weather"),
+    ]
+    assert [block["tool_use_id"] for block in result[2]["content"] if block["type"] == "tool_result"] == [
+        "srvtoolu_01Search",
+        "toolu_01Weather",
+    ]
+
+
+def _merged_assistant_run_cost(run_length: int) -> float:
+    messages: Final = [
+        {"role": "user", "content": "Plan the trip"},
+        *({"role": "assistant", "content": "Working on it", "tool_calls": []} for _ in range(run_length)),
+        {"role": "user", "content": "Go on"},
+    ]
+
+    def once() -> float:
+        started: Final = time.perf_counter()
+        anthropic_messages_pt(messages=messages, model="claude-sonnet-5-5", llm_provider="anthropic")
+        return time.perf_counter() - started
+
+    return min(once() for _ in range(3))
+
+
+def test_anthropic_messages_pt_merges_a_long_assistant_run_in_linear_time():
+    growth: Final = _merged_assistant_run_cost(4000) / _merged_assistant_run_cost(1000)
+
+    assert growth < 8.0, f"cost grew {growth:.1f}x for 4x the assistant run"

tests/unit/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_mid_conversation_system.py

@@ -5,6 +5,10 @@ Anthropic, Vertex, Azure AI and Bedrock Invoke transformation tests; these pin
 the pure placement rules on the OpenAI-format message list.
 """
 
+import time
+from itertools import chain
+from typing import Final
+
 import pytest
 
 import litellm
@@ -255,6 +259,13 @@ def test_flagged_placement_keeps_a_system_behind_a_user_turn_merged_with_an_empt
     assert _roles(placed) == ["user", "user", "system", "assistant"]
 
 
+_TOOL_SEARCH_USE = {
+    "id": "srvtoolu_01Search",
+    "type": "function",
+    "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+}
+
+
 EMPTY_ASSISTANT = pytest.mark.parametrize(
     "empty_assistant",
     [
@@ -281,6 +292,7 @@ EMPTY_ASSISTANT = pytest.mark.parametrize(
             "thinking_blocks": [{"type": "redacted_thinking", "data": "x"}],
         },
         litellm.Message(role="assistant", content=None),
+        {"role": "assistant", "content": None, "tool_calls": [_TOOL_SEARCH_USE]},
     ],
     ids=[
         "none",
@@ -291,10 +303,55 @@ EMPTY_ASSISTANT = pytest.mark.parametrize(
         "unsigned-inline-part-hides-signed-block",
         "inline-redacted-part-hides-redacted-block",
         "pydantic-none",
+        "unanswered-tool-search-use",
     ],
 )
 
 
+def test_flagged_placement_handles_a_long_run_of_unanswered_tool_search_uses_in_linear_time():
+    search_uses: Final = tuple(
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [{**_TOOL_SEARCH_USE, "id": f"srvtoolu_{index:06d}"}],
+        }
+        for index in range(3000)
+    )
+    reminders: Final = tuple(
+        chain.from_iterable(
+            ({"role": "system", "content": f"reminder {index}"}, {"role": "user", "content": f"q{index}"})
+            for index in range(10)
+        )
+    )
+    messages: Final = [
+        {"role": "user", "content": "start"},
+        *reminders,
+        *search_uses,
+        {"role": "user", "content": "end"},
+    ]
+
+    started: Final = time.perf_counter()
+    placed: Final = place_mid_conversation_system(messages, supports_mid_conversation_system=True)
+    elapsed: Final = time.perf_counter() - started
+
+    assert set(_roles(placed)[:21]) == {"user"}
+    assert elapsed < 1.0
+
+
+def test_flagged_placement_keeps_a_system_before_a_tool_search_use_the_caller_answered():
+    placed = place_mid_conversation_system(
+        [
+            {"role": "user", "content": "q1"},
+            {"role": "system", "content": "reminder"},
+            {"role": "assistant", "content": None, "tool_calls": [_TOOL_SEARCH_USE]},
+            {"role": "tool", "tool_call_id": "srvtoolu_01Search", "content": "[]"},
+        ],
+        supports_mid_conversation_system=True,
+    )
+
+    assert _roles(placed) == ["user", "system", "assistant", "tool"]
+
+
 @EMPTY_ASSISTANT
 def test_flagged_placement_converts_a_run_when_the_assistant_turn_after_its_anchor_sends_nothing(empty_assistant):
     placed = place_mid_conversation_system(
@@ -343,6 +400,14 @@ def test_flagged_placement_keeps_a_system_whose_empty_assistant_follower_ends_th
             content="",
             tool_calls=[{"id": "toolu_1", "type": "function", "function": {"name": "f", "arguments": "{}"}}],
         ),
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [
+                _TOOL_SEARCH_USE,
+                {"id": "toolu_1", "type": "function", "function": {"name": "f", "arguments": "{}"}},
+            ],
+        },
     ],
     ids=[
         "tool-calls",
@@ -351,6 +416,7 @@ def test_flagged_placement_keeps_a_system_whose_empty_assistant_follower_ends_th
         "signed-thinking-part",
         "function-call",
         "pydantic-tool-calls",
+        "unanswered-tool-search-use-beside-a-client-call",
     ],
 )
 def test_flagged_placement_keeps_a_system_before_an_assistant_turn_that_renders_without_text(assistant_turn):

tests/unit/litellm_core_utils/test_fallback_generalizations.py

@@ -970,19 +970,16 @@ def test_shipped_tool_search_rule_version_boundaries(shipped_cost_map, model, pr
 
 def test_shipped_tool_search_rule_fills_mapped_claude_entries_without_flag(shipped_cost_map):
     """A mapped Claude 4.5+ entry with no supports_tool_search key gets it from the rule
-    on Anthropic direct, Vertex and Bedrock, a mapped pre-4.5 entry stays without one,
-    and Azure Foundry and reseller copies of the same model are not touched."""
+    on Anthropic direct, Azure Foundry, Vertex and Bedrock, a mapped pre-4.5 entry stays
+    without one, and reseller copies of the same model are not touched."""
     for key, model, provider in (
         ("claude-opus-4-7", "claude-opus-4-7", "anthropic"),
+        ("azure_ai/claude-opus-5", "claude-opus-5", "azure_ai"),
         ("vertex_ai/claude-opus-5", "claude-opus-5", "vertex_ai"),
     ):
         assert "supports_tool_search" not in litellm.model_cost[key]
         assert litellm.get_model_info(model, custom_llm_provider=provider)["supports_tool_search"] is True
 
-    assert "supports_tool_search" not in litellm.model_cost["azure_ai/claude-opus-5"]
-    azure_opus_5_info = litellm.get_model_info("claude-opus-5", custom_llm_provider="azure_ai")
-    assert azure_opus_5_info.get("supports_tool_search") is None
-
     assert match_fill_missing_generalizations("claude-opus-5", "bedrock")["supports_tool_search"] is True
-    assert "supports_tool_search" not in match_fill_missing_generalizations("claude-opus-5", "azure_ai")
+    assert match_fill_missing_generalizations("claude-opus-5", "azure_ai")["supports_tool_search"] is True
     assert match_fill_missing_generalizations("claude-opus-5", "perplexity") is None

tests/unit/llms/anthropic/chat/test_anthropic_chat_transformation.py

@@ -1611,6 +1611,19 @@ def test_defer_loading_preserved_in_transformation():
     assert mcp_server is None
 
 
+def test_defer_loading_rejects_non_boolean_values():
+    config = AnthropicConfig()
+
+    tool = {
+        "type": "function",
+        "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {}}},
+        "defer_loading": "true",
+    }
+
+    with pytest.raises(ValueError, match="defer_loading must be a boolean"):
+        config.map_tool_helper(tool)
+
+
 def test_tool_search_complete_response_parsing():
     """Test parsing a complete tool search response with server_tool_use and tool_search_tool_result blocks"""
     config = AnthropicConfig()

tests/unit/llms/openai/responses/test_openai_responses_guardrail_handler.py

@@ -38,7 +38,13 @@ from litellm.responses.litellm_completion_transformation.transformation import (
     LiteLLMCompletionResponsesConfig,
 )
 from litellm.types.llms.openai import ResponsesAPIResponse
-from litellm.types.responses.main import CustomToolCallOutputItem, GenericResponseOutputItem, OutputText
+from litellm.responses.litellm_completion_transformation.tool_search import build_tool_search_call
+from litellm.types.responses.main import (
+    CustomToolCallOutputItem,
+    GenericResponseOutputItem,
+    OutputText,
+    OutputToolSearchCall,
+)
 from litellm.types.utils import CallTypes, GenericGuardrailAPIInputs
 
 
@@ -138,6 +144,40 @@ class PersimmonMaskingGuardrail(CustomGuardrail):
         return {**inputs, "tool_calls": tool_calls}
 
 
+class RecordingPersimmonMaskingGuardrail(PersimmonMaskingGuardrail):
+    def __init__(self, guardrail_name: str) -> None:
+        super().__init__(guardrail_name=guardrail_name)
+        self.seen_tool_calls: list[tuple[str, str]] = []
+
+    async def apply_guardrail(
+        self,
+        inputs: GenericGuardrailAPIInputs,
+        request_data: dict,
+        input_type: Literal["request", "response"],
+        logging_obj: LiteLLMLoggingObj | None = None,
+    ) -> GenericGuardrailAPIInputs:
+        self.seen_tool_calls.extend(
+            (tool_call["function"]["name"], tool_call["function"]["arguments"])
+            for tool_call in inputs.get("tool_calls", [])
+        )
+        return await super().apply_guardrail(inputs, request_data, input_type, logging_obj)
+
+
+class ArgumentsRedactingGuardrail(CustomGuardrail):
+    async def apply_guardrail(
+        self,
+        inputs: GenericGuardrailAPIInputs,
+        request_data: dict,
+        input_type: Literal["request", "response"],
+        logging_obj: LiteLLMLoggingObj | None = None,
+    ) -> GenericGuardrailAPIInputs:
+        tool_calls = [
+            {**tool_call, "function": {**tool_call["function"], "arguments": "[REDACTED]"}}
+            for tool_call in inputs.get("tool_calls", [])
+        ]
+        return {**inputs, "tool_calls": tool_calls}
+
+
 class FlatShapeGuardrail(CustomGuardrail):
     async def apply_guardrail(
         self,
@@ -1058,6 +1098,49 @@ class TestOpenAIResponsesHandlerToolCallExtraction:
         assert result["output"][0]["input"] == "echo [MASKED]"
         assert "name" not in result["output"][0]
 
+    @pytest.mark.asyncio
+    async def test_process_output_response_scans_and_rewrites_a_tool_search_call(self):
+        handler = OpenAIResponsesHandler()
+        guardrail = RecordingPersimmonMaskingGuardrail(guardrail_name="mask")
+        response = ResponsesAPIResponse(
+            id="resp_1",
+            created_at=1,
+            model="claude-sonnet-5-5",
+            object="response",
+            status="completed",
+            output=[build_tool_search_call("srvtoolu_1", '{"query": "persimmon"}', "completed")],
+        )
+
+        result = await handler.process_output_response(response, guardrail)
+
+        assert guardrail.seen_tool_calls == [("tool_search", '{"query": "persimmon"}')]
+        search_call = result.output[0]
+        assert isinstance(search_call, OutputToolSearchCall)
+        assert search_call.arguments == {"query": "[MASKED]"}
+        assert search_call.call_id == "srvtoolu_1"
+        assert "name" not in search_call.model_dump()
+
+    @pytest.mark.asyncio
+    async def test_process_output_response_never_returns_original_search_arguments_after_a_non_json_rewrite(self):
+        response = ResponsesAPIResponse(
+            id="resp_1",
+            created_at=1,
+            model="claude-sonnet-5-5",
+            object="response",
+            status="completed",
+            output=[build_tool_search_call("srvtoolu_1", '{"query": "persimmon"}', "completed")],
+        )
+
+        result = await OpenAIResponsesHandler().process_output_response(
+            response, ArgumentsRedactingGuardrail(guardrail_name="redact")
+        )
+
+        search_call = result.output[0]
+        assert isinstance(search_call, OutputToolSearchCall)
+        assert search_call.arguments == {}
+        assert search_call.status == "incomplete"
+        assert "persimmon" not in result.model_dump_json()
+
     @pytest.mark.asyncio
     async def test_process_output_response_with_tool_calls(self):
         """Test processing output response containing function tool calls"""
@@ -1873,6 +1956,131 @@ class TestOpenAIResponsesHandlerStreamingOutputProcessing:
         assert completed_event.response.output[0].input == "echo [MASKED]"
         assert completed_event.response.output[0].name == "exec"
 
+    @pytest.mark.asyncio
+    async def test_deliver_ended_stream_rewrites_a_tool_search_call(self):
+        from litellm.types.llms.base import BaseLiteLLMOpenAIResponseObject
+        from litellm.types.llms.openai import (
+            OutputItemAddedEvent,
+            OutputItemDoneEvent,
+            ResponseCompletedEvent,
+            ResponsesAPIStreamEvents,
+        )
+
+        def search_call(arguments: str, status: Literal["in_progress", "completed"]) -> OutputToolSearchCall:
+            return build_tool_search_call("srvtoolu_1", arguments, status)
+
+        handler = OpenAIResponsesHandler()
+        added = OutputItemAddedEvent(
+            type=ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED,
+            output_index=0,
+            item=BaseLiteLLMOpenAIResponseObject(**search_call("", "in_progress").model_dump()),
+        )
+        done = OutputItemDoneEvent(
+            type=ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE,
+            output_index=0,
+            sequence_number=2,
+            item=BaseLiteLLMOpenAIResponseObject(**search_call('{"query": "persimmon"}', "completed").model_dump()),
+        )
+        completed = ResponseCompletedEvent(
+            type=ResponsesAPIStreamEvents.RESPONSE_COMPLETED,
+            response=ResponsesAPIResponse(
+                id="resp_1",
+                created_at=1,
+                model="claude-sonnet-5-5",
+                object="response",
+                status="completed",
+                output=[search_call('{"query": "persimmon"}', "completed")],
+            ),
+        )
+
+        await handler.process_output_streaming_response(
+            responses_so_far=[added, done, completed],
+            guardrail_to_apply=PersimmonMaskingGuardrail(guardrail_name="mask"),
+            litellm_logging_obj=None,
+            deliver_ended_stream_rewrites=True,
+        )
+
+        assert added.item.model_dump()["arguments"] == {}
+        assert done.item.model_dump()["arguments"] == {"query": "[MASKED]"}
+        assert isinstance(completed.response.output[0], OutputToolSearchCall)
+        assert completed.response.output[0].arguments == {"query": "[MASKED]"}
+        assert "name" not in done.item.model_dump()
+
+    @pytest.mark.asyncio
+    async def test_deliver_ended_stream_rewrites_a_native_tool_search_call_that_has_no_call_id(self):
+        def search_call(arguments: dict, status: str) -> dict:
+            return {
+                "type": "tool_search_call",
+                "id": "tsc_1",
+                "call_id": None,
+                "arguments": arguments,
+                "execution": "server",
+                "status": status,
+            }
+
+        def function_call(arguments: str, status: str) -> dict:
+            return {
+                "type": "function_call",
+                "id": "fc_1",
+                "call_id": "call_1",
+                "name": "lookup_fruit",
+                "arguments": arguments,
+                "status": status,
+            }
+
+        handler = OpenAIResponsesHandler()
+        events = [
+            {"type": "response.output_item.added", "output_index": 0, "item": search_call({}, "in_progress")},
+            {
+                "type": "response.output_item.done",
+                "output_index": 0,
+                "item": search_call({"paths": ["persimmon"]}, "completed"),
+            },
+            {"type": "response.output_item.added", "output_index": 1, "item": function_call("", "in_progress")},
+            {
+                "type": "response.function_call_arguments.delta",
+                "item_id": "fc_1",
+                "output_index": 1,
+                "delta": '{"fruit": "persimmon"}',
+            },
+            {
+                "type": "response.function_call_arguments.done",
+                "item_id": "fc_1",
+                "output_index": 1,
+                "arguments": '{"fruit": "persimmon"}',
+            },
+            {
+                "type": "response.output_item.done",
+                "output_index": 1,
+                "item": function_call('{"fruit": "persimmon"}', "completed"),
+            },
+            {
+                "type": "response.completed",
+                "response": {
+                    "id": "resp_1",
+                    "model": "gpt-5.6",
+                    "output": [
+                        search_call({"paths": ["persimmon"]}, "completed"),
+                        function_call('{"fruit": "persimmon"}', "completed"),
+                    ],
+                },
+            },
+        ]
+
+        await handler.process_output_streaming_response(
+            responses_so_far=events,
+            guardrail_to_apply=PersimmonMaskingGuardrail(guardrail_name="mask"),
+            litellm_logging_obj=None,
+            deliver_ended_stream_rewrites=True,
+        )
+
+        assert events[1]["item"]["arguments"] == {"paths": ["[MASKED]"]}
+        assert events[3]["delta"] == '{"fruit": "[MASKED]"}'
+        assert events[4]["arguments"] == '{"fruit": "[MASKED]"}'
+        assert events[5]["item"]["arguments"] == '{"fruit": "[MASKED]"}'
+        assert events[6]["response"]["output"][0]["arguments"] == {"paths": ["[MASKED]"]}
+        assert events[6]["response"]["output"][1]["arguments"] == '{"fruit": "[MASKED]"}'
+
     @pytest.mark.asyncio
     async def test_deliver_ended_stream_custom_tool_call_rewrite_without_matching_events_fails_closed(self):
         from litellm.proxy.policy_engine.pipeline_executor import UndeliverableStreamRewrite
@@ -3256,6 +3464,44 @@ class TestOpenAIResponsesHandlerNamespaceTools:
 
         assert result["tools"] == expected_tools
 
+    @pytest.mark.asyncio
+    async def test_scanning_tools_with_malformed_flags_leaves_them_for_the_route_to_judge(self):
+        data: Final = _codex_request("hi")
+        data["tools"][0]["defer_loading"] = "false"
+        data["tools"][1]["tools"][0]["allowed_callers"] = "direct"
+        data["tools"][2]["defer_loading"] = 1
+        data["tools"].append({"type": "tool_search", "execution": ""})
+        expected_tools: Final = copy.deepcopy(data["tools"])
+        guardrail: Final = MockRecordingGuardrail(guardrail_name="test")
+
+        result: Final = await OpenAIResponsesHandler().process_input_messages(data, guardrail)
+
+        assert result["tools"] == expected_tools
+        assert [tool["function"]["name"] for tool in guardrail.seen_inputs[0]["tools"]] == [
+            "get_weather",
+            "mcp__confluence__confluence_get_page",
+            "mcp__confluence__confluence_search",
+            "apply_patch",
+        ]
+
+    @pytest.mark.asyncio
+    async def test_a_function_only_guardrail_keeps_the_tool_search_tool_it_never_scans(self):
+        data: Final = _codex_request("hi")
+        data["tools"] = [{"type": "tool_search"}, {**data["tools"][0], "defer_loading": True}]
+        expected_tools: Final = copy.deepcopy(data["tools"])
+        scanned: Final[list[list[dict]]] = []
+
+        def keep_functions(tools):
+            scanned.append(copy.deepcopy(tools))
+            return [tool for tool in tools if tool["type"] == "function"]
+
+        result: Final = await OpenAIResponsesHandler().process_input_messages(
+            data, ToolEditingGuardrail(edit=keep_functions, guardrail_name="test")
+        )
+
+        assert result["tools"] == expected_tools
+        assert [tool["type"] for tool in scanned[0]] == ["function"]
+
     @pytest.mark.asyncio
     async def test_appending_guardrail_keeps_namespace_and_adds_tool(self):
         data = _codex_request("hi")

tests/unit/responses/litellm_completion_transformation/test_anthropic_responses_bridge.py

@@ -1,5 +1,6 @@
 import json
 from collections.abc import Mapping, Sequence
+from itertools import chain
 from typing import Final
 from unittest.mock import AsyncMock, MagicMock, patch
 
@@ -17,7 +18,7 @@ from litellm.responses.litellm_completion_transformation.transformation import (
     LiteLLMCompletionResponsesConfig,
 )
 from litellm.types.llms.openai import ResponsesAPIStreamEvents
-from litellm.types.utils import ModelResponse
+from litellm.types.utils import Message, ModelResponse
 
 
 def test_response_api_handler_merges_metadata_and_service_tier_without_error():
@@ -307,6 +308,179 @@ async def test_previous_response_id_without_instructions_carries_the_latest_ones
     assert [block.text for block in anthropic.request.system] == [expected_system]
 
 
+_SEARCH_CALL: Final = {
+    "id": "srvtoolu_search",
+    "type": "function",
+    "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+}
+_WEATHER_CALL: Final = {
+    "id": "toolu_weather",
+    "type": "function",
+    "function": {"name": "get_weather", "arguments": '{"city": "Tokyo"}'},
+}
+
+
+def _search_turn(tool_calls: Sequence[Mapping[str, object]]) -> Mapping[str, object]:
+    return {
+        "request_id": "chatcmpl-search-turn",
+        "call_type": "aresponses",
+        "session_id": "session-search",
+        "proxy_server_request": {"model": _MODEL, "input": "What is the weather in Tokyo?"},
+        "response": {
+            "id": "chatcmpl-search-turn",
+            "object": "chat.completion",
+            "created": 0,
+            "model": _MODEL,
+            "choices": [
+                {
+                    "index": 0,
+                    "finish_reason": "tool_calls",
+                    "message": {"role": "assistant", "content": None, "tool_calls": list(tool_calls)},
+                }
+            ],
+        },
+    }
+
+
+async def _follow_up_messages(
+    tool_calls: Sequence[Mapping[str, object]], model: str, custom_llm_provider: str
+) -> Sequence[object]:
+    return await _follow_up_messages_after(_search_turn(tool_calls), model, custom_llm_provider)
+
+
+async def _follow_up_messages_after(
+    spend_log: Mapping[str, object], model: str, custom_llm_provider: str
+) -> Sequence[object]:
+    with patch("litellm.proxy.proxy_server.prisma_client", _FakePrismaClient([spend_log])):
+        request: Final = await LiteLLMCompletionResponsesConfig.async_responses_api_session_handler(
+            previous_response_id="chatcmpl-search-turn",
+            litellm_completion_request={
+                "model": model,
+                "custom_llm_provider": custom_llm_provider,
+                "messages": [{"role": "user", "content": "And in Osaka?"}],
+            },
+            instructions=None,
+        )
+    return request["messages"]
+
+
+def _replayed_tool_call_ids(messages: Sequence[object]) -> list[str]:
+    assistant_turns: Final = [message for message in messages if isinstance(message, Message)]
+    return [tool_call.id for turn in assistant_turns for tool_call in turn.tool_calls or ()]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("model", "custom_llm_provider", "expected_tool_call_ids"),
+    [
+        ("mistral/mistral-small-latest", "mistral", ["toolu_weather"]),
+        ("gemini/gemini-2.5-flash", "gemini", ["toolu_weather"]),
+        ("databricks/databricks-claude-sonnet-4-5", "databricks", ["toolu_weather"]),
+        (_MODEL, "anthropic", ["srvtoolu_search", "toolu_weather"]),
+    ],
+)
+async def test_previous_response_id_keeps_a_stored_claude_search_call_only_on_an_anthropic_route(
+    model: str, custom_llm_provider: str, expected_tool_call_ids: list[str]
+) -> None:
+    """
+    A follow-up that routes a Claude turn's history elsewhere must not replay Anthropic's server-side
+    search call as a client tool call, which Mistral answers 400 "Unexpected tool call id"
+    """
+    messages: Final = await _follow_up_messages([_SEARCH_CALL, _WEATHER_CALL], model, custom_llm_provider)
+
+    assert _replayed_tool_call_ids(messages) == expected_tool_call_ids
+
+
+@pytest.mark.asyncio
+async def test_previous_response_id_drops_a_claude_turn_that_held_only_a_search_call_off_anthropic() -> None:
+    messages: Final = await _follow_up_messages([_SEARCH_CALL], "mistral/mistral-small-latest", "mistral")
+
+    assert [message["role"] for message in messages] == ["user", "user"]
+
+
+def _replayed_search_turn(search_answered: bool) -> Mapping[str, object]:
+    search_result: Final = (
+        [{"type": "function_call_output", "call_id": "srvtoolu_search", "output": "get_weather"}]
+        if search_answered
+        else []
+    )
+    return {
+        **_search_turn([]),
+        "proxy_server_request": {
+            "model": _MODEL,
+            "input": [
+                {"role": "user", "content": "What is the weather in Tokyo?"},
+                {
+                    "type": "function_call",
+                    "call_id": "srvtoolu_search",
+                    "name": "tool_search_tool_regex",
+                    "arguments": '{"pattern": "weather"}',
+                },
+                *search_result,
+                {
+                    "type": "function_call",
+                    "call_id": "toolu_weather",
+                    "name": "get_weather",
+                    "arguments": '{"city": "Tokyo"}',
+                },
+                {"type": "function_call_output", "call_id": "toolu_weather", "output": '{"temp_c": 18}'},
+            ],
+        },
+        "response": {
+            "id": "chatcmpl-search-turn",
+            "object": "chat.completion",
+            "created": 0,
+            "model": _MODEL,
+            "choices": [
+                {
+                    "index": 0,
+                    "finish_reason": "stop",
+                    "message": {"role": "assistant", "content": "It is 18C in Tokyo."},
+                }
+            ],
+        },
+    }
+
+
+def _tool_call_ids_of(message: object) -> Sequence[str]:
+    if isinstance(message, Message):
+        return [tool_call.id or "" for tool_call in message.tool_calls or ()]
+    if isinstance(message, dict):
+        return [tool_call["id"] for tool_call in message.get("tool_calls") or ()]
+    return []
+
+
+def _history_tool_ids(messages: Sequence[object]) -> tuple[Sequence[str], Sequence[str]]:
+    called: Final = list(chain.from_iterable(map(_tool_call_ids_of, messages)))
+    answered: Final = [
+        message["tool_call_id"] for message in messages if isinstance(message, dict) and message["role"] == "tool"
+    ]
+    return called, answered
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("search_answered", [False, True])
+@pytest.mark.parametrize(
+    ("model", "custom_llm_provider", "expect_search"),
+    [("mistral/mistral-small-latest", "mistral", False), (_MODEL, "anthropic", True)],
+)
+async def test_previous_response_id_drops_a_client_replayed_search_call_only_off_anthropic(
+    model: str, custom_llm_provider: str, expect_search: bool, search_answered: bool
+) -> None:
+    """
+    A client can replay Claude's search call as a ``function_call`` input item with its ``srvtoolu_`` id. A later
+    turn sent elsewhere must not carry it, which OpenAI-compatible providers answer 400 "did not have response messages"
+    """
+    messages: Final = await _follow_up_messages_after(
+        _replayed_search_turn(search_answered), model, custom_llm_provider
+    )
+
+    called, answered = _history_tool_ids(messages)
+    search_ids: Final = ["srvtoolu_search"] if expect_search else []
+    assert called == [*search_ids, "toolu_weather"]
+    assert answered == [*(search_ids if search_answered else []), "toolu_weather"]
+
+
 def _anthropic_sse(*events: Mapping[str, object]) -> bytes:
     return b"".join(f"event: {event['type']}\ndata: {json.dumps(event)}\n\n".encode() for event in events)
 

tests/unit/responses/litellm_completion_transformation/test_litellm_completion_responses.py

@@ -30,7 +30,7 @@ from litellm.responses.litellm_completion_transformation.transformation import (
 )
 from litellm.types.llms.base import HiddenParams
 from litellm.types.llms.openai import ResponsesAPIOptionalRequestParams
-from litellm.types.responses.main import build_web_search_call
+from litellm.types.responses.main import OutputToolSearchCall, build_web_search_call
 from litellm.types.utils import (
     ChatCompletionMessageToolCall,
     Choices,
@@ -1597,6 +1597,64 @@ class TestContentTypeTransformation:
         ]
 
 
+def _deferred_tools_with_server_tool_search() -> list[dict[str, object]]:
+    return [
+        {"type": "tool_search", "execution": "server"},
+        {
+            "type": "function",
+            "name": "get_weather",
+            "description": "Get the current weather for a city",
+            "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
+            "defer_loading": True,
+        },
+        {
+            "type": "namespace",
+            "name": "finance",
+            "tools": [
+                {
+                    "type": "function",
+                    "name": "get_stock_price",
+                    "parameters": {"type": "object", "properties": {"ticker": {"type": "string"}}},
+                    "defer_loading": True,
+                }
+            ],
+        },
+    ]
+
+
+def _chat_response_with_tool_search_call(
+    search_arguments: str, search_call_id: str = "srvtoolu_01Search"
+) -> ModelResponse:
+    return ModelResponse(
+        id="chatcmpl-tool-search",
+        created=1748575031,
+        model="claude-sonnet-5-5",
+        object="chat.completion",
+        choices=[
+            {
+                "index": 0,
+                "finish_reason": "tool_calls",
+                "message": {
+                    "role": "assistant",
+                    "content": None,
+                    "tool_calls": [
+                        {
+                            "id": search_call_id,
+                            "type": "function",
+                            "function": {"name": "tool_search_tool_regex", "arguments": search_arguments},
+                        },
+                        {
+                            "id": "toolu_01Weather",
+                            "type": "function",
+                            "function": {"name": "get_weather", "arguments": '{"city": "Paris"}'},
+                        },
+                    ],
+                },
+            }
+        ],
+    )
+
+
 class TestToolTransformation:
     """Test cases for tool transformation from Responses API to Chat Completion format"""
 
@@ -1734,6 +1792,420 @@ class TestToolTransformation:
         assert len(result_tools) == function_and_custom_count
         assert web_search_options is not None
 
+    @pytest.mark.parametrize("provider", ["anthropic", "azure_ai"])
+    def test_server_tool_search_becomes_anthropic_tool_search_for_claude(self, provider: str) -> None:
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request
+        result: Final = transform(
+            model=f"{provider}/claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+            custom_llm_provider=provider,
+        )
+
+        chat_tools: Final = result["tools"]
+        assert chat_tools[0] == {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"}
+        assert [tool["function"]["name"] for tool in chat_tools[1:]] == ["get_weather", "finance__get_stock_price"]
+        assert [tool["defer_loading"] for tool in chat_tools[1:]] == [True, True]
+
+    def test_server_tool_search_dropped_for_models_without_tool_search(self) -> None:
+        model: Final = "openai/model-with-no-cost-map-entry"
+        assert litellm.supports_tool_search(model) is False
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model=model,
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+            custom_llm_provider="openai",
+        )
+
+        assert [tool["type"] for tool in result["tools"]] == ["function", "function"]
+
+    def test_dropping_the_search_tool_undefers_the_remaining_tools(self) -> None:
+        model: Final = "azure_ai/claude-opus-4-1"
+        assert litellm.supports_tool_search(model, "azure_ai") is False
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model=model,
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+            custom_llm_provider="azure_ai",
+        )
+
+        assert [tool["type"] for tool in result["tools"]] == ["function", "function"]
+        assert all("defer_loading" not in tool for tool in result["tools"])
+
+    @pytest.mark.parametrize(
+        "search_tools", [[], [{"type": "tool_search", "execution": "client"}]], ids=["none", "client"]
+    )
+    @pytest.mark.parametrize(
+        ("model", "provider"),
+        [
+            ("anthropic/claude-sonnet-5-5", "anthropic"),
+            ("vertex_ai/claude-sonnet-5-5", "vertex_ai"),
+            ("bedrock/invoke/us.anthropic.claude-sonnet-5-5-v1:0", "bedrock"),
+            ("azure_ai/claude-sonnet-5-5", "azure_ai"),
+        ],
+    )
+    def test_claude_routes_keep_the_callers_deferral_when_no_server_search_tool_is_sent(
+        self, model: str, provider: str, search_tools: list[dict[str, object]]
+    ) -> None:
+        function_tools: Final = [
+            tool for tool in _deferred_tools_with_server_tool_search() if tool["type"] != "tool_search"
+        ]
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model=model,
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": [*search_tools, *function_tools]},
+            custom_llm_provider=provider,
+        )
+
+        assert [tool["function"]["name"] for tool in result["tools"]] == ["get_weather", "finance__get_stock_price"]
+        assert [tool["defer_loading"] for tool in result["tools"]] == [True, True]
+
+    @pytest.mark.parametrize(
+        ("model", "provider"),
+        [
+            ("mistral/mistral-large-latest", "mistral"),
+            ("gemini/gemini-2.5-flash", "gemini"),
+            ("bedrock/converse/us.anthropic.claude-sonnet-5-5-v1:0", "bedrock"),
+        ],
+    )
+    def test_routes_without_anthropic_deferral_get_deferred_tools_undeferred(self, model: str, provider: str) -> None:
+        tools: Final = [tool for tool in _deferred_tools_with_server_tool_search() if tool["type"] != "tool_search"]
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model=model,
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": tools},
+            custom_llm_provider=provider,
+        )
+
+        assert [tool["function"]["name"] for tool in result["tools"]] == ["get_weather", "finance__get_stock_price"]
+        assert all("defer_loading" not in tool for tool in result["tools"])
+
+    def test_native_bm25_search_tool_keeps_the_other_tools_deferred(self) -> None:
+        tools: Final = [
+            {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},
+            *_deferred_tools_with_server_tool_search()[1:],
+        ]
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model="anthropic/claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": tools},
+            custom_llm_provider="anthropic",
+        )
+
+        chat_tools: Final = result["tools"]
+        assert chat_tools[0] == {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"}
+        assert [tool["defer_loading"] for tool in chat_tools[1:]] == [True, True]
+
+    @pytest.mark.parametrize(
+        "native_search_tool",
+        [
+            {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},
+            {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},
+        ],
+    )
+    def test_server_tool_search_beside_a_native_search_tool_sends_only_the_native_one(
+        self, native_search_tool: Mapping[str, str]
+    ) -> None:
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model="anthropic/claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": [native_search_tool, *_deferred_tools_with_server_tool_search()]},
+            custom_llm_provider="anthropic",
+        )
+
+        chat_tools: Final = result["tools"]
+        assert [tool for tool in chat_tools if tool["type"] != "function"] == [native_search_tool]
+        assert [tool["defer_loading"] for tool in chat_tools if tool["type"] == "function"] == [True, True]
+
+    def test_native_search_tool_passes_through_routes_without_tool_search(self) -> None:
+        model: Final = "azure_ai/claude-opus-4-1"
+        assert litellm.supports_tool_search(model, "azure_ai") is False
+        tools: Final = [
+            {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},
+            *_deferred_tools_with_server_tool_search()[1:],
+        ]
+
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model=model,
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": tools},
+            custom_llm_provider="azure_ai",
+        )
+
+        chat_tools: Final = result["tools"]
+        assert chat_tools[0] == {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"}
+        assert [tool["defer_loading"] for tool in chat_tools[1:]] == [True, True]
+
+    def test_server_tool_search_follows_the_bedrock_route(self) -> None:
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request
+        converse: Final = transform(
+            model="bedrock/anthropic.claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+            custom_llm_provider="bedrock",
+        )
+        invoke: Final = transform(
+            model="bedrock/invoke/anthropic.claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+            custom_llm_provider="bedrock",
+        )
+
+        assert [tool["type"] for tool in converse["tools"]] == ["function", "function"]
+        assert invoke["tools"][0] == {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"}
+
+    def test_caller_function_named_like_the_search_tool_stays_a_function_call(self) -> None:
+        items: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_tools_to_responses_tools(
+            chat_completion_response=_chat_response_with_tool_search_call(
+                '{"pattern": "weather"}', search_call_id="call_01Mine"
+            ),
+            responses_api_request={
+                "tools": [
+                    *_deferred_tools_with_server_tool_search(),
+                    {"type": "function", "name": "tool_search_tool_regex", "parameters": {"type": "object"}},
+                ]
+            },
+        )
+
+        assert [(item.type, item.name) for item in items] == [
+            ("function_call", "tool_search_tool_regex"),
+            ("function_call", "get_weather"),
+        ]
+
+    def test_defer_loading_must_be_a_boolean(self) -> None:
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools
+        namespace_tool: Final = {
+            "type": "namespace",
+            "name": "finance",
+            "tools": [{"type": "function", "name": "get_stock_price", "parameters": {"type": "object"}}],
+        }
+
+        with pytest.raises(litellm.BadRequestError, match="defer_loading must be a boolean"):
+            transform(tools=[{**namespace_tool, "tools": [{**namespace_tool["tools"][0], "defer_loading": "false"}]}])
+        with pytest.raises(litellm.BadRequestError, match="defer_loading must be a boolean"):
+            transform(tools=[{"type": "function", "name": "get_weather", "parameters": {}, "defer_loading": "false"}])
+        not_deferred, _ = transform(
+            tools=[
+                {**namespace_tool, "tools": [{**namespace_tool["tools"][0], "defer_loading": False}]},
+                {"type": "function", "name": "get_weather", "parameters": {}, "defer_loading": False},
+            ]
+        )
+        assert [tool["function"]["name"] for tool in not_deferred] == ["finance__get_stock_price", "get_weather"]
+        assert all("defer_loading" not in tool for tool in not_deferred)
+
+    @pytest.mark.parametrize("global_drop_params", [False, True])
+    def test_malformed_tool_flags_answer_400_unless_drop_params_drops_them(
+        self, monkeypatch: pytest.MonkeyPatch, global_drop_params: bool
+    ) -> None:
+        monkeypatch.setattr(litellm, "drop_params", global_drop_params)
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request
+        tools: Final = [
+            {"type": "tool_search", "execution": "server"},
+            {
+                "type": "function",
+                "name": "get_weather",
+                "parameters": {"type": "object"},
+                "defer_loading": "false",
+                "allowed_callers": "direct",
+            },
+            {
+                "type": "namespace",
+                "name": "finance",
+                "tools": [
+                    "not a tool",
+                    {
+                        "type": "function",
+                        "name": "get_stock_price",
+                        "parameters": {"type": "object"},
+                        "defer_loading": 1,
+                        "allowed_callers": "direct",
+                    },
+                    {
+                        "type": "function",
+                        "name": "get_fx_rate",
+                        "parameters": {"type": "object"},
+                        "defer_loading": True,
+                    },
+                ],
+            },
+            {"type": "custom", "name": "apply_patch", "defer_loading": "yes"},
+        ]
+
+        if not global_drop_params:
+            with pytest.raises(litellm.BadRequestError):
+                transform(
+                    model="anthropic/claude-sonnet-5-5",
+                    input="What is the weather in Paris right now?",
+                    responses_api_request={"tools": tools},
+                    custom_llm_provider="anthropic",
+                )
+        result: Final = transform(
+            model="anthropic/claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": tools},
+            custom_llm_provider="anthropic",
+            drop_params=None if global_drop_params else True,
+        )
+
+        function_tools: Final = [tool for tool in result["tools"] if tool["type"] == "function"]
+        assert {tool["function"]["name"]: tool.get("defer_loading", False) for tool in function_tools} == {
+            "get_weather": False,
+            "finance__get_stock_price": False,
+            "finance__get_fx_rate": True,
+            "apply_patch": False,
+        }
+        assert all("allowed_callers" not in tool for tool in function_tools)
+
+    @pytest.mark.parametrize("execution", ["", "bogus", 5, ["server"]])
+    def test_malformed_tool_search_execution_answers_400_unless_drop_params_drops_it(self, execution: object) -> None:
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request
+        tools: Final = [{"type": "tool_search", "execution": execution}, *_deferred_tools_with_server_tool_search()[1:]]
+
+        with pytest.raises(litellm.BadRequestError, match="tool_search execution must be 'server' or 'client'"):
+            transform(
+                model="anthropic/claude-sonnet-5-5",
+                input="What is the weather in Paris right now?",
+                responses_api_request={"tools": tools},
+                custom_llm_provider="anthropic",
+            )
+        dropped: Final = transform(
+            model="anthropic/claude-sonnet-5-5",
+            input="What is the weather in Paris right now?",
+            responses_api_request={"tools": tools},
+            custom_llm_provider="anthropic",
+            drop_params=True,
+        )
+        items: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_tools_to_responses_tools(
+            chat_completion_response=_chat_response_with_tool_search_call('{"pattern": "weather"}'),
+            responses_api_request={"tools": tools},
+        )
+
+        assert dropped["tools"][0] == {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"}
+        assert all(tool.get("defer_loading") is True for tool in dropped["tools"][1:])
+        assert [item.type for item in items] == ["tool_search_call", "function_call"]
+
+    def test_top_level_function_allowed_callers_must_be_a_list_of_strings(self) -> None:
+        transform: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools
+        weather_tool: Final = {"type": "function", "name": "get_weather", "parameters": {"type": "object"}}
+
+        with pytest.raises(litellm.BadRequestError, match="allowed_callers must be a list of strings"):
+            transform(tools=[{**weather_tool, "allowed_callers": "direct"}])
+        converted, _ = transform(tools=[{**weather_tool, "allowed_callers": ["direct"]}])
+        assert converted[0]["allowed_callers"] == ["direct"]
+
+    def test_custom_tools_stay_deferred_when_tool_search_goes_out(self) -> None:
+        result: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_request_to_chat_completion_request(
+            model="anthropic/claude-sonnet-5-5",
+            input="Patch the README",
+            responses_api_request={
+                "tools": [
+                    {"type": "tool_search", "execution": "server"},
+                    {"type": "custom", "name": "apply_patch", "defer_loading": True},
+                    {
+                        "type": "namespace",
+                        "name": "editor",
+                        "tools": [{"type": "custom", "name": "insert", "defer_loading": True}],
+                    },
+                    {"type": "function", "name": "get_weather", "parameters": {"type": "object"}},
+                ]
+            },
+            custom_llm_provider="anthropic",
+        )
+
+        function_tools: Final = [tool for tool in result["tools"] if tool["type"] == "function"]
+        assert {tool["function"]["name"]: tool.get("defer_loading", False) for tool in function_tools} == {
+            "apply_patch": True,
+            "insert": True,
+            "get_weather": False,
+        }
+
+    def test_anthropic_tool_search_call_becomes_tool_search_call_item(self) -> None:
+        response: Final = _chat_response_with_tool_search_call('{"pattern": "weather|forecast"}')
+        with_tool_search: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_tools_to_responses_tools(
+            chat_completion_response=response,
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+        )
+        without_tool_search: Final = (
+            LiteLLMCompletionResponsesConfig.transform_chat_completion_tools_to_responses_tools(
+                chat_completion_response=response,
+                responses_api_request={"tools": _deferred_tools_with_server_tool_search()[1:]},
+            )
+        )
+
+        assert [item.type for item in with_tool_search] == ["tool_search_call", "function_call"]
+        search_call: Final = with_tool_search[0]
+        assert isinstance(search_call, OutputToolSearchCall)
+        assert search_call.id == "tsc_srvtoolu_01Search"
+        assert search_call.call_id == "srvtoolu_01Search"
+        assert search_call.arguments == {"pattern": "weather|forecast"}
+        assert search_call.execution == "server"
+        assert search_call.status == "completed"
+        assert with_tool_search[1].name == "get_weather"
+        assert [item.type for item in without_tool_search] == ["function_call", "function_call"]
+
+        full_response: Final = (
+            LiteLLMCompletionResponsesConfig.transform_chat_completion_response_to_responses_api_response(
+                request_input="What is the weather in Paris right now?",
+                responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+                chat_completion_response=response,
+            )
+        )
+        assert [item.type for item in full_response.output] == ["tool_search_call", "function_call"]
+
+    def test_tool_search_call_with_unparseable_arguments_is_incomplete(self) -> None:
+        items: Final = LiteLLMCompletionResponsesConfig.transform_chat_completion_tools_to_responses_tools(
+            chat_completion_response=_chat_response_with_tool_search_call('{"pattern": '),
+            responses_api_request={"tools": _deferred_tools_with_server_tool_search()},
+        )
+
+        search_call: Final = items[0]
+        assert isinstance(search_call, OutputToolSearchCall)
+        assert search_call.status == "incomplete"
+        assert search_call.arguments == {}
+
+    def test_tool_search_items_are_not_replayed_to_chat_messages(self) -> None:
+        tools: Final = _deferred_tools_with_server_tool_search()
+        messages: Final = LiteLLMCompletionResponsesConfig.transform_responses_api_input_to_messages(
+            input=[
+                {"type": "message", "role": "user", "content": "What is the weather in Paris right now?"},
+                {
+                    "type": "tool_search_call",
+                    "id": "tsc_srvtoolu_01Search",
+                    "call_id": "srvtoolu_01Search",
+                    "arguments": {"pattern": "weather|forecast"},
+                    "execution": "server",
+                    "status": "completed",
+                },
+                {
+                    "type": "tool_search_output",
+                    "id": "tso_srvtoolu_01Search",
+                    "call_id": "srvtoolu_01Search",
+                    "execution": "server",
+                    "status": "completed",
+                    "tools": [tools[1]],
+                },
+                {
+                    "type": "function_call",
+                    "id": "fc_toolu_01Weather",
+                    "call_id": "toolu_01Weather",
+                    "name": "get_weather",
+                    "arguments": '{"city": "Paris"}',
+                    "status": "completed",
+                },
+                {"type": "function_call_output", "call_id": "toolu_01Weather", "output": "18C and sunny"},
+            ],
+            responses_api_request={"tools": tools},
+        )
+
+        assert [message["role"] for message in messages] == ["user", "assistant", "tool"]
+        assert [call["id"] for call in messages[1]["tool_calls"]] == ["toolu_01Weather"]
+        assert "tool_search" not in json.dumps(messages)
+
     def test_transform_local_shell_tools_dropped(self) -> None:
         (
             result_tools,
@@ -1852,7 +2324,7 @@ class TestToolTransformation:
 
         tools = [custom_tool]
 
-        with pytest.raises(ValueError, match="allowed_callers must be a list of strings"):
+        with pytest.raises(litellm.BadRequestError, match="allowed_callers must be a list of strings"):
             LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools(tools=tools)
 
     def test_transform_web_search_tools_to_web_search_options(self):
@@ -2419,7 +2891,7 @@ class TestToolTransformation:
             else {**function_tool, "type": "namespace"}
         )
 
-        with pytest.raises(ValueError, match="allowed_callers must be a list of strings"):
+        with pytest.raises(litellm.BadRequestError, match="allowed_callers must be a list of strings"):
             LiteLLMCompletionResponsesConfig.transform_responses_api_tools_to_chat_completion_tools(
                 tools=[namespace_tool]
             )
@@ -3808,6 +4280,8 @@ class TestEnsureOutputItemContentPartAdded:
         iterator._namespace_tool_names = LiteLLMCompletionResponsesConfig.namespace_tool_name_map(None)
         iterator._web_search_calls = {}
         iterator._queued_web_search_call_ids = set()
+        iterator._tool_search_requested = False
+        iterator._tool_search_call_ids = set()
         return iterator
 
     def _make_text_chunk(self):
@@ -5558,6 +6032,35 @@ def test_replayed_session_turns_yield_their_cache_control_to_the_newest_markers(
     assert messages[-1] == {"role": "user", "content": "newest", "cache_control": _EPHEMERAL}
 
 
+def test_a_replayed_claude_search_call_is_dropped_off_anthropic_while_the_turns_keep_their_cache_marks():
+    search_call: Final = {
+        "id": "srvtoolu_search",
+        "type": "function",
+        "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+    }
+    weather_call: Final = {
+        "id": "toolu_weather",
+        "type": "function",
+        "function": {"name": "get_weather", "arguments": '{"city": "Tokyo"}'},
+    }
+    replayed_turns: Final = [
+        {"role": "user", "content": "turn 0", "cache_control": _EPHEMERAL},
+        {"role": "assistant", "content": None, "tool_calls": [search_call, weather_call]},
+        {"role": "tool", "tool_call_id": "srvtoolu_search", "content": "get_weather"},
+        {"role": "tool", "tool_call_id": "toolu_weather", "content": "18C"},
+        *({"role": "user", "content": f"turn {index}", "cache_control": _EPHEMERAL} for index in range(1, 4)),
+    ]
+
+    messages: Final = _replayed_messages_from(
+        replayed_turns, {"role": "user", "content": "newest", "cache_control": _EPHEMERAL}
+    )
+
+    assert [message["role"] for message in messages] == ["user", "assistant", "tool", "user", "user", "user", "user"]
+    assert messages[1]["tool_calls"] == [weather_call]
+    assert messages[2]["tool_call_id"] == "toolu_weather"
+    assert ["cache_control" in message for message in messages] == [False, False, False, True, True, True, True]
+
+
 def test_a_marked_tool_takes_a_slot_from_the_replayed_turns():
     replayed_turns: Final = [
         {"role": "user", "content": f"turn {index}", "cache_control": _EPHEMERAL} for index in range(3)

tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py

@@ -1326,3 +1326,144 @@ async def test_plain_text_stream_announces_exactly_one_message_item(sync_mode: b
             ResponsesAPIStreamEvents.OUTPUT_TEXT_DONE,
         ):
             assert event.item_id == message_item_adds[0].item.id
+
+
+@pytest.mark.parametrize("sync_mode", [True, False])
+@pytest.mark.asyncio
+async def test_anthropic_tool_search_call_streams_as_tool_search_call_item(sync_mode: bool):
+    deltas: Final = (
+        Delta(
+            content=None,
+            tool_calls=[
+                {
+                    "index": 0,
+                    "id": "srvtoolu_01Search",
+                    "type": "function",
+                    "function": {"name": "tool_search_tool_regex", "arguments": ""},
+                }
+            ],
+        ),
+        Delta(content=None, tool_calls=[{"index": 0, "function": {"arguments": '{"pattern": '}}]),
+        Delta(content=None, tool_calls=[{"index": 0, "function": {"arguments": '"weather|forecast"}'}}]),
+        Delta(
+            content=None,
+            tool_calls=[
+                {
+                    "index": 1,
+                    "id": "toolu_01Weather",
+                    "type": "function",
+                    "function": {"name": "get_weather", "arguments": '{"city":"Paris"}'},
+                }
+            ],
+        ),
+    )
+    chunks: Final = tuple(
+        ModelResponseStream(
+            id=CHAT_COMPLETION_ID,
+            created=1748575031,
+            model="claude-sonnet-5-5",
+            object="chat.completion.chunk",
+            choices=[
+                StreamingChoices(
+                    index=0, delta=delta, finish_reason="tool_calls" if index == len(deltas) - 1 else None
+                )
+            ],
+        )
+        for index, delta in enumerate(deltas)
+    )
+    iterator: Final = LiteLLMCompletionStreamingIterator(
+        model="claude-sonnet-5-5",
+        litellm_custom_stream_wrapper=_FakeStreamWrapper(chunks),
+        request_input="What is the weather in Paris right now?",
+        responses_api_request={
+            "tools": [
+                {"type": "tool_search", "execution": "server"},
+                {"type": "function", "name": "get_weather", "parameters": {"type": "object"}, "defer_loading": True},
+            ]
+        },
+        custom_llm_provider="anthropic",
+    )
+    events: Final = (
+        [event.model_dump(exclude_none=True) for event in iterator]
+        if sync_mode
+        else [event.model_dump(exclude_none=True) async for event in iterator]
+    )
+    completed: Final = events[-1]
+    output: Final = completed["response"]["output"]
+    search_item: Final = next(item for item in output if item["type"] == "tool_search_call")
+    function_item: Final = next(item for item in output if item["type"] == "function_call")
+    search_events: Final = [
+        event for event in events if event.get("item_id", event.get("item", {}).get("id")) == search_item["id"]
+    ]
+    function_events: Final = [
+        event for event in events if event.get("item_id", event.get("item", {}).get("id")) == function_item["id"]
+    ]
+
+    assert completed["type"] == "response.completed"
+    assert search_item["id"] == "tsc_srvtoolu_01Search"
+    assert search_item["call_id"] == "srvtoolu_01Search"
+    assert search_item["arguments"] == {"pattern": "weather|forecast"}
+    assert search_item["execution"] == "server"
+    assert search_item["status"] == "completed"
+    assert [event["type"] for event in search_events] == [
+        "response.output_item.added",
+        "response.output_item.done",
+    ]
+    assert search_events[0]["item"]["status"] == "in_progress"
+    assert search_events[0]["item"]["arguments"] == {}
+    assert search_events[-1]["item"] == search_item
+    assert function_item["name"] == "get_weather"
+    assert json.loads(function_item["arguments"]) == {"city": "Paris"}
+    assert "response.function_call_arguments.done" in [event["type"] for event in function_events]
+    assert {event["output_index"] for event in search_events}.isdisjoint(
+        {event["output_index"] for event in function_events}
+    )
+
+
+@pytest.mark.parametrize("sync_mode", [True, False])
+@pytest.mark.asyncio
+async def test_caller_function_named_like_the_search_tool_streams_as_function_call(sync_mode: bool):
+    chunk: Final = ModelResponseStream(
+        id=CHAT_COMPLETION_ID,
+        created=1748575031,
+        model="claude-sonnet-5-5",
+        object="chat.completion.chunk",
+        choices=[
+            StreamingChoices(
+                index=0,
+                finish_reason="tool_calls",
+                delta=Delta(
+                    content=None,
+                    tool_calls=[
+                        {
+                            "index": 0,
+                            "id": "call_01Mine",
+                            "type": "function",
+                            "function": {"name": "tool_search_tool_regex", "arguments": '{"pattern": "weather"}'},
+                        }
+                    ],
+                ),
+            )
+        ],
+    )
+    iterator: Final = LiteLLMCompletionStreamingIterator(
+        model="claude-sonnet-5-5",
+        litellm_custom_stream_wrapper=_FakeStreamWrapper((chunk,)),
+        request_input="What is the weather in Paris right now?",
+        responses_api_request={
+            "tools": [
+                {"type": "tool_search", "execution": "server"},
+                {"type": "function", "name": "tool_search_tool_regex", "parameters": {"type": "object"}},
+            ]
+        },
+        custom_llm_provider="anthropic",
+    )
+    events: Final = (
+        [event.model_dump(exclude_none=True) for event in iterator]
+        if sync_mode
+        else [event.model_dump(exclude_none=True) async for event in iterator]
+    )
+    output: Final = events[-1]["response"]["output"]
+
+    assert [(item["type"], item["name"]) for item in output] == [("function_call", "tool_search_tool_regex")]
+    assert "response.function_call_arguments.done" in [event["type"] for event in events]

tests/unit/types/llms/test_types_llms_openai.py

@@ -668,3 +668,21 @@ def test_openai_video_object_hidden_params_public_accessor_preserves_identity()
     video.hidden_params = replacement
     assert video._hidden_params is replacement
     assert video.hidden_params["public_key"] == "visible"
+
+
+@pytest.mark.parametrize("extra_keys", ({}, {"created_by": "server"}), ids=("bridge-item", "openai-item"))
+def test_rebuilding_a_dumped_responses_answer_keeps_the_tool_search_call_keys(extra_keys: dict[str, str]) -> None:
+    search_call: Final = {
+        "id": "tsc_1",
+        "arguments": {"query": "weather"},
+        "call_id": None,
+        "execution": "server",
+        "status": "completed",
+        "type": "tool_search_call",
+        **extra_keys,
+    }
+    answer: Final = ResponsesAPIResponse(id="resp_1", created_at=1, output=[search_call])
+
+    rebuilt: Final = ResponsesAPIResponse(**answer.model_dump())
+
+    assert rebuilt.model_dump()["output"] == [search_call]

ui/litellm-dashboard/src/lib/http/schema.d.ts

@@ -28866,6 +28866,8 @@ export interface components {
             /** Allowed Callers */
             allowed_callers?: string[];
             cache_control?: components["schemas"]["ChatCompletionCachedContent"];
+            /** Defer Loading */
+            defer_loading?: boolean;
             /** Eager Input Streaming */
             eager_input_streaming?: boolean;
             function: components["schemas"]["ChatCompletionToolParamFunctionChunk"];
@@ -39876,6 +39878,32 @@ export interface components {
         } & {
             [key: string]: unknown;
         };
+        /** OutputToolSearchCall */
+        OutputToolSearchCall: {
+            /** Arguments */
+            arguments: unknown;
+            /** Call Id */
+            call_id?: string | null;
+            /**
+             * Execution
+             * @enum {string}
+             */
+            execution: "server" | "client";
+            /** Id */
+            id: string;
+            /**
+             * Status
+             * @enum {string}
+             */
+            status: "in_progress" | "completed" | "incomplete";
+            /**
+             * Type
+             * @constant
+             */
+            type: "tool_search_call";
+        } & {
+            [key: string]: unknown;
+        };
         /**
          * PageLinks
          * @description Hypermedia for a paginated list. No `first`/`last`: without a total count the last page is unknown.
@@ -43853,7 +43881,7 @@ export interface components {
             /** Output */
             output: (components["schemas"]["ResponseOutputMessage"] | components["schemas"]["ResponseFileSearchToolCall"] | components["schemas"]["ResponseFunctionToolCall"] | components["schemas"]["ResponseFunctionToolCallOutputItem"] | components["schemas"]["ResponseFunctionWebSearch"] | components["schemas"]["ResponseComputerToolCall"] | components["schemas"]["ResponseComputerToolCallOutputItem"] | components["schemas"]["ResponseReasoningItem"] | components["schemas"]["ResponseToolSearchCall"] | components["schemas"]["ResponseToolSearchOutputItem"] | components["schemas"]["ResponseCompactionItem"] | components["schemas"]["ImageGenerationCall"] | components["schemas"]["ResponseCodeInterpreterToolCall"] | components["schemas"]["LocalShellCall"] | components["schemas"]["LocalShellCallOutput"] | components["schemas"]["ResponseFunctionShellToolCall"] | components["schemas"]["ResponseFunctionShellToolCallOutput"] | components["schemas"]["ResponseApplyPatchToolCall"] | components["schemas"]["ResponseApplyPatchToolCallOutput"] | components["schemas"]["McpCall"] | components["schemas"]["McpListTools"] | components["schemas"]["McpApprovalRequest"] | components["schemas"]["McpApprovalResponse"] | components["schemas"]["ResponseCustomToolCall"] | components["schemas"]["ResponseCustomToolCallOutputItem"] | {
                 [key: string]: unknown;
-            })[] | (components["schemas"]["GenericResponseOutputItem"] | components["schemas"]["OutputCodeInterpreterCall"] | components["schemas"]["OutputFunctionToolCall"] | components["schemas"]["OutputImageGenerationCall"] | components["schemas"]["ResponseFunctionToolCall"] | components["schemas"]["ResponseFunctionWebSearch"] | components["schemas"]["CustomToolCallOutputItem"])[];
+            })[] | (components["schemas"]["GenericResponseOutputItem"] | components["schemas"]["OutputCodeInterpreterCall"] | components["schemas"]["OutputFunctionToolCall"] | components["schemas"]["OutputImageGenerationCall"] | components["schemas"]["ResponseFunctionToolCall"] | components["schemas"]["ResponseFunctionWebSearch"] | components["schemas"]["CustomToolCallOutputItem"] | components["schemas"]["OutputToolSearchCall"])[];
             /** Parallel Tool Calls */
             parallel_tool_calls?: boolean | null;
             /** Previous Response Id */