mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
901f914ba2
* fix(responses): replay Claude tool search on follow-up turns Claude tool search on /v1/responses now returns a tool_search_output after each tool_search_call, streamed and not, listing the tools the search loaded as the caller sent them A stateless follow-up sends the search back to Anthropic as the server_tool_use and tool_search_tool_result Claude produced, and the Anthropic request drops a replayed search whose loaded tools are no longer sent, so a changed tool list no longer answers 400 The guardrail write-back for SDK output items reads part text through a typed helper, so the new output item type adds no attribute errors * fix(responses): emit Claude's tool search output in stream order The streamed tool_search_output item was announced at the end of the stream with an output index reserved earlier, so output_item.added events arrived as 0, 2, 1 and the OpenAI SDK's stream helper raised IndexError. The item is now emitted when Claude's search result arrives. * fix(anthropic): keep the caller's tool names in tool search results A tool name Anthropic cannot take gets rewritten on the request, so Claude's tool_search_tool_result named the rewritten tool. The response now maps those tool references back to the caller's names, streamed and not, and a replayed search result maps them forward again on the next turn. * fix(responses): replay Claude's tool search only on routes that run it A follow-up sent to a route that cannot run tool search, a Claude model without it included, now leaves out the replayed search the way it did before, instead of sending Claude search blocks that model cannot take. The stateless gate and the stored-turn filter now use the same check that already decides whether the tool_search tool itself is kept A streamed tool_search_call now gets its output_item.done when Claude's search result arrives, so it finishes before its tool_search_output. Clients that build history from done events, like Codex, kept the output ahead of its call * fix(responses): keep a replayed search when the caller sent Anthropic's own search tool A Claude model the cost map does not flag for tool search still runs it when the request carries Anthropic's own tool_search_tool_regex or bm25 tool, which is how turn 1 already decides to keep it. Both replay gates now follow that rule, so a stored follow-up on such a model keeps Claude's search as it did before, and a stateless one replays it * fix(responses): replay a BM25 tool search under its own name * fix(responses): keep a replayed tool search visible to guardrails and prune only missing tools A replayed Claude tool search now reaches pre-call guardrails and the Purview DLP scan with its search arguments, a stale search drops only the tool references the follow-up no longer sends, and the stored previous_response_id path keeps the base gate for routes the cost map does not flag * fix(responses): replay Claude's search on a stateless follow-up only when it still sends a search tool A store false follow-up that sent no search tool still got the search replayed, and Claude then said it called a tool it no longer had * fix(responses): replay a stored Claude search only when the follow-up still sends a search tool A previous_response_id follow-up that sent no search tool still got the stored search, the same gap a store false follow-up had. The stale search pruning now logs the tools it leaves out at debug level. handler.py reads the reported speed from the usage chunk, and the guardrail write-back skips tool search outputs, so the branch adds no type errors. * fix(responses): leave out a stored Claude search kept in server_tool_calls Main stores Claude's server-side tool search on the assistant message in provider_specific_fields.server_tool_calls and tool_results, not in tool_calls. The previous_response_id path only looked at tool_calls, so a follow-up that sent no search tool still replayed the stored search. The session handler now finds the search in either place and drops it from both server_tool_calls and tool_results when the follow-up cannot run a search. * test(responses): give the replayed search guardrail fixture its own call id The guardrail test reused toolu_01Weather, an id that streaming and bridge tests already put in the process-wide TOOL_CALLS_CACHE. When one of those ran first in the same worker, the guardrail write-back took its rebuild path and the test failed for a reason unrelated to the replayed search. * fix(anthropic): keep Claude's replayed tool search on renamed tools and stored follow-ups A search-only assistant turn now gets its search result renamed to the sanitized tool names too, so the stale-tool trim no longer drops it. Stored follow-ups on Claude routes replay the search as base did, trimmed to the tools still sent, and Bedrock invoke sends the tool search beta whenever the history holds a search * test(responses): pin that guardrails scan a replayed Claude search on a follow-up without a search tool * fix(anthropic): keep Anthropic's tool names in chat tool search results A chat history whose search result used the request's tool names (weather.get) got Anthropic's 400 when replayed on a proxy still on the old code, during a rolling deploy or a rollback. Chat completions now returns Claude's search result as main does, and the Responses bridge maps a rewritten name back to the request's tool when it builds tool_search_output. That also fills tool_search_output on Bedrock invoke streams, which carry no name map --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| docker-entrypoint.d | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||