mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-08 08:08:37 +00:00
febbe38326
* fix(anthropic): do not surface server_tool_use as client tool_calls Anthropic server-executed tools (web search, tool search, code execution) were returned in chat completions tool_calls with srvtoolu_ ids, so OpenAI-protocol agent clients treated them as pending work and re-invoked the model until their recursion limit. Move them to provider_specific_fields.server_tool_calls on both the non-streaming and streaming paths. History replay, the /v1/responses bridge and the /v1/messages streaming guardrail read that field, so multi-turn reconstruction, web_search_call output items and the guardrail's fail-closed check on server tool blocks keep working. * fix(responses): return Claude web_fetch calls as web_search_call open_page items * fix(responses): keep Claude server web calls without a result as hosted items A non-streamed Claude turn that calls web_search or web_fetch next to a client tool can return the server call before its result. Build an in_progress web_search_call for it, so the Responses bridge no longer returns it as a client function_call * fix(anthropic): keep pending server calls native and hide server calls from client tool paths Replaying a server call that has no result yet now sends it back as server_tool_use, so Anthropic accepts the next turn. The tool permission guardrail checks server calls again and rewrite mode removes a denied one. The /v1/messages adapter keeps Claude's server blocks, and the Responses bridge never returns a server call as a client function_call. * fix(anthropic): let guardrails, tool usage, tracing, and Responses replay see Claude's server calls Claude's server tool calls moved out of tool_calls into provider_specific_fields.server_tool_calls, so every reader of tool_calls stopped seeing them. The chat guardrail handler now passes them to output guardrails and writes rewrites back to server_tool_calls (stream and non-stream), tool usage and registry discovery count them, otel, Datadog and the Logs Tools panel list them, and a pending Claude web search or fetch item replayed to /v1/responses goes back to Anthropic as its own server_tool_use instead of running again * fix(anthropic): let guardrails and loggers read Claude's server tool calls again Guardrails, tool policies, tracing loggers and the Responses guardrail translation read only client tool calls after server calls moved to provider_specific_fields. They now read server calls first, then client calls, through one shared helper, and the Responses guardrail treats Claude's web_search_call items as web_search or web_fetch calls. Non-streaming splits server calls by block type, as streaming does. * fix(anthropic): keep Claude server calls visible to guardrails, redaction and tracing A guardrail rewrite of a Claude server call now also updates its hosted web_search_calls copy, streamed or not. Redaction with message logging off hides server call arguments and hosted web call actions on chat and Responses. Pillar and Pangea scan server calls, transform-mode streaming guardrails keep server call chunks, and OTEL v2 Responses spans list Claude's web calls. * fix(anthropic): keep the legacy hook view of streamed /v1/messages web search after the main merge Main's legacy post-call hooks see Claude's server call as tool_use with its search results first. The streamed /v1/messages response now keeps that view while native scans keep server_tool_use. * fix(guardrails): let Cisco AI Defense scan Claude's web calls on Responses Claude's web search and fetch now come back from /v1/responses as web_search_call items, so Cisco AI Defense reads and clears their query or URL the way it did for the old function_call arguments. The helpers that read and rewrite those items move to the shared guardrail utils so the Responses guardrail handler and Cisco use one copy. * test(guardrails): inject Pillar's HTTP client and type the Cisco web call actions * test(guardrails): type the fake Pillar client like its Protocol * fix(anthropic): keep a pending server call and its thinking when the turn also waits on a client tool Only a turn with no client tool_use is a paused turn. A tool_use turn that also started web_fetch or web_search still needs that call and its thinking replayed, or Claude loses the fetch result * fix(guardrails): scan replayed Claude server calls and hosted web calls Request guardrails now see and rewrite server_tool_calls replayed in chat messages. Cato and Purview scan Claude web_search and web_fetch calls on Responses output, and a function tool call with an extra custom key is no longer dropped from Cisco, Purview, Repello and PANW scans. * fix(guardrails): scan replayed Claude server tool calls in Lasso, Lakera and Qualifire request checks * fix(anthropic): apply streamed guardrail tool call rewrites to their own block when a server tool call follows * test: annotate the new server tool call test locals with Final and type their parameters * test: type the new guardrail test helpers with litellm's tool call types instead of Any * test: narrow the remaining new test helper parameters from bare dict and object * fix(anthropic): keep server calls in Responses streams, guardrail masks, OTEL and Logs Streamed /v1/responses now emits Claude's tool search and paused web search calls, text guardrail masks rewrite replayed server calls, OTEL gen_ai messages and the Logs pretty view list server calls next to client calls, and integration tests read server calls from server_tool_calls * fix(guardrails): keep hosted web call copies in sync on request masks and denied server calls, and accept null tool_calls in Lakera * fix(anthropic): serialize masked hosted web call copies through their adapter so a replay before the OpenAI model is built does not 500 --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| audio_utils | ||
| llm_cost_calc | ||
| llm_response_utils | ||
| prompt_templates | ||
| specialty_caches | ||
| __init__.py | ||
| attribute_forwarder.py | ||
| conftest.py | ||
| event_loop_lag.py | ||
| fake_secret_vault.py | ||
| messages_with_counts.py | ||
| test_agentic_followup_kwargs.py | ||
| test_allowed_tools_tool_choice.py | ||
| test_anthropic_dedup_factory.py | ||
| test_api_route_to_call_types.py | ||
| test_app_crypto.py | ||
| test_audio_utils.py | ||
| test_aws_partition.py | ||
| test_batch_utils.py | ||
| test_bedrock_converse_dedup_factory.py | ||
| test_bug_report.py | ||
| test_chat_completion_agentic_loop.py | ||
| test_classifier_logging.py | ||
| test_cli_token_utils.py | ||
| test_cloud_storage_security.py | ||
| test_codestral_provider_routing.py | ||
| test_core_helpers.py | ||
| test_coroutine_checker.py | ||
| test_cost_discount.py | ||
| test_custom_logger_registry.py | ||
| test_dd_tracing.py | ||
| test_decisions_pricing.py | ||
| test_decode_special_tokens.py | ||
| test_default_encoding.py | ||
| test_dot_notation_indexing.py | ||
| test_duration_parser.py | ||
| test_env_utils.py | ||
| test_error_normalization.py | ||
| test_exception_mapping_utils.py | ||
| test_extract_base64_image.py | ||
| test_fallback_generalizations.py | ||
| test_fallback_utils.py | ||
| test_get_litellm_params.py | ||
| test_get_llm_provider_endpoint_match.py | ||
| test_get_llm_provider_logic.py | ||
| test_get_model_cost_map.py | ||
| test_get_supported_openai_params.py | ||
| test_health_check_helpers.py | ||
| test_hidden_params.py | ||
| test_image_handling.py | ||
| test_initialize_dynamic_callback_params.py | ||
| test_internal_call_metadata.py | ||
| test_json_fragment_accumulator.py | ||
| test_json_schema_validation.py | ||
| test_litellm_logging.py | ||
| test_llm_judge.py | ||
| test_llm_request_utils.py | ||
| test_logging_callback_manager.py | ||
| test_logging_utils.py | ||
| test_logging_worker.py | ||
| test_max_streaming_duration.py | ||
| test_model_latest_aliases.py | ||
| test_model_param_helper.py | ||
| test_model_response_utils.py | ||
| test_moderation_standard_logging.py | ||
| test_oauth_token_exchange.py | ||
| test_offline_mode.py | ||
| test_optional_imports.py | ||
| test_path_utils.py | ||
| test_private_json.py | ||
| test_provider_affinity.py | ||
| test_provider_specific_headers.py | ||
| test_ptu_pricing.py | ||
| test_realtime_errors.py | ||
| test_realtime_streaming.py | ||
| test_redact_messages.py | ||
| test_request_timeout_resolver.py | ||
| test_retry_after_headers.py | ||
| test_safe_divide_seconds.py | ||
| test_safe_json_dumps.py | ||
| test_sensitive_data_masker.py | ||
| test_sentry_scrubbing.py | ||
| test_served_output_texts.py | ||
| test_stream_usage_logging.py | ||
| test_streaming_chunk_builder_cursor.py | ||
| test_streaming_chunk_builder_server_tool_use.py | ||
| test_streaming_chunk_builder_utils.py | ||
| test_streaming_handler.py | ||
| test_streaming_overhead.py | ||
| test_text_completion_conversion.py | ||
| test_thread_pool_executor.py | ||
| test_token_counter.py | ||
| test_token_counter_tool.py | ||
| test_token_counter_tool_data.py | ||
| test_tokenizer.py | ||
| test_tool_fields.py | ||
| test_tool_search_spend_logging.py | ||
| test_url_utils.py | ||
| test_xai_oauth_routing.py | ||