mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
b51b9b7d43
* fix(anthropic): answer 400 for a malformed tool_result text on the Responses bridge and count_tokens A tool_result text block whose text is null, a number, or missing made /v1/messages and /v1/messages/count_tokens answer 500 on OpenAI and Azure models. Both routes now answer 400 naming the block, and drop the block when drop_params is on. * fix(anthropic): keep a missing tool_result text as empty text, bound the error, honor body drop_params on count_tokens * fix(anthropic): apply the tool_result text check on the chat/completions bridge and every count route The chat/completions bridge used for Azure, Gemini and other providers now answers the same 400 for a null or numeric tool_result text and drops the block under drop_params, so a model group that mixes OpenAI with another provider answers the same way on every deployment. Gemini and Vertex count_tokens apply the same check, count_tokens turns on drop_params for Claude Code and Codex the way /v1/messages does, and a tool_result without a tool_use_id is checked too * fix(proxy): answer 400 for a malformed tool_result text on count_tokens models without a count API Providers with no token counting API fall back to the local tokenizer, which counted a null or numeric tool_result text as fine while /v1/messages on the same model answers 400. The fallback now runs the same check, unless drop_params applies to the deployment or the request * fix(proxy): honor request drop_params on the local count fallback when no deployment resolves * test(integration): cover non-string tool_result text on both /v1/messages bridges and count routes * fix(anthropic): keep a dropped tool_result on Mistral and reject it before a guardrail rewrite Under drop_params a tool_result whose only text block was null or a number became an empty tool message, and Mistral's empty-message filter removed it, so Mistral answered 400 for a missing tool reply. The filter now only removes empty assistant messages, which is what Mistral rejects. A guardrail that rewrites structured_messages wrote the dropped text back as "", so a null or numeric text was silently blanked even without drop_params. The rewrite path now answers the same 400 as the bridges unless the request or global drop_params is set. * test(integration): cover the Mistral empty tool reply and the guardrail rewrite tool_result text check * fix(anthropic): keep a malformed tool_result text through a guardrail rewrite and keep Mistral's null-message drop A guardrail that rewrites structured_messages now writes the caller's null or numeric tool_result text back in place, so the route's bridge and the deployment's drop_params decide after routing, the same as with no guardrail. The rewrite path no longer answers its own 400 before routing. Mistral drops a null-content system or user message again, as main does, and only a tool reply is exempt from that filter so a dropped tool_result keeps its tool call. * test(mistral): expect empty system and user turns dropped and the empty tool reply kept * perf(anthropic): restore kept tool_result text parts in one linear pass * test(integration): cover the tool_result text restore across tool_results, the chat bridge, and a batch upload behind a rewriting guardrail * fix(anthropic): let guardrails scan malformed tool_result text and honor drop_params on native Anthropic A rewriting guardrail now sees a non-string tool_result text as its JSON text and the write-back restores the caller's value by the sanitized tool_use_id when the guardrail left it unchanged. Native Anthropic drops the bad block under drop_params on /v1/messages and count_tokens, and Mistral keeps an empty tool reply only when it answers a tool call, sent as empty text. * fix(anthropic): compare a kept tool_result text to its scanned form once and align the wire tests with drop_params * test(integration): cover native Anthropic drop_params resolution, guardrail scanning of numeric tool_result text, and the Mistral orphan tool message * fix(count_tokens): honor the router's default drop_params when counting a malformed tool_result text * test(count_tokens): send the tool_result count tests through a real request body instead of a mocked parser * test(count_tokens): send the native Claude count test through a real request body * fix(anthropic): keep each non-string tool_result text when tool_use_ids sanitize to the same id * fix(anthropic): keep non-ASCII characters in the guardrail's view of a non-string tool_result text * test(gemini): mock the count_tokens upstream with respx instead of patching the client factory * fix(mistral): keep an empty tool reply only for an open call, and forward non-object message items on the native drop An empty tool reply re-answering a call the conversation already moved past made Mistral reject the request, and a tool call id that is not a string crashed the filter. Native Anthropic under drop_params assumed every message item was an object. * fix(mistral): find answering tool replies in one pass instead of copying the open call set per reply * fix(mistral): ignore dropped empty placeholders when tracking open tool calls * fix(mistral): drop an empty tool reply when another reply to the same call carries the result * refactor(anthropic): share tool_result text helpers to shrink the diff * fix(anthropic): answer 400 for a non-string tool_result text on the OpenAI Responses bridge |
||
|---|---|---|
| .. | ||
| audio_utils | ||
| llm_cost_calc | ||
| llm_response_utils | ||
| prompt_templates | ||
| specialty_caches | ||
| __init__.py | ||
| conftest.py | ||
| event_loop_lag.py | ||
| fake_secret_vault.py | ||
| messages_with_counts.py | ||
| test_agentic_followup_kwargs.py | ||
| test_anthropic_dedup_factory.py | ||
| test_api_route_to_call_types.py | ||
| test_app_crypto.py | ||
| test_audio_utils.py | ||
| test_aws_partition.py | ||
| test_bedrock_converse_dedup_factory.py | ||
| test_bug_report.py | ||
| test_chat_completion_agentic_loop.py | ||
| test_classifier_logging.py | ||
| test_cli_token_utils.py | ||
| test_cloud_storage_security.py | ||
| test_codestral_provider_routing.py | ||
| test_core_helpers.py | ||
| test_coroutine_checker.py | ||
| test_cost_discount.py | ||
| test_custom_logger_registry.py | ||
| test_dd_tracing.py | ||
| test_decisions_pricing.py | ||
| test_decode_special_tokens.py | ||
| test_default_encoding.py | ||
| test_dot_notation_indexing.py | ||
| test_duration_parser.py | ||
| test_env_utils.py | ||
| test_error_normalization.py | ||
| test_exception_mapping_utils.py | ||
| test_extract_base64_image.py | ||
| test_fallback_generalizations.py | ||
| test_fallback_utils.py | ||
| test_get_litellm_params.py | ||
| test_get_llm_provider_endpoint_match.py | ||
| test_get_llm_provider_logic.py | ||
| test_get_model_cost_map.py | ||
| test_get_supported_openai_params.py | ||
| test_health_check_helpers.py | ||
| test_hidden_params.py | ||
| test_image_handling.py | ||
| test_initialize_dynamic_callback_params.py | ||
| test_internal_call_metadata.py | ||
| test_json_fragment_accumulator.py | ||
| test_json_schema_validation.py | ||
| test_litellm_logging.py | ||
| test_llm_judge.py | ||
| test_llm_request_utils.py | ||
| test_logging_callback_manager.py | ||
| test_logging_utils.py | ||
| test_logging_worker.py | ||
| test_max_streaming_duration.py | ||
| test_model_param_helper.py | ||
| test_model_response_utils.py | ||
| test_moderation_standard_logging.py | ||
| test_oauth_token_exchange.py | ||
| test_optional_imports.py | ||
| test_path_utils.py | ||
| test_private_json.py | ||
| test_provider_affinity.py | ||
| test_provider_specific_headers.py | ||
| test_ptu_pricing.py | ||
| test_realtime_errors.py | ||
| test_realtime_streaming.py | ||
| test_redact_messages.py | ||
| test_request_timeout_resolver.py | ||
| test_retry_after_headers.py | ||
| test_safe_divide_seconds.py | ||
| test_safe_json_dumps.py | ||
| test_sensitive_data_masker.py | ||
| test_sentry_scrubbing.py | ||
| test_served_output_texts.py | ||
| test_stream_usage_logging.py | ||
| test_streaming_chunk_builder_cursor.py | ||
| test_streaming_chunk_builder_server_tool_use.py | ||
| test_streaming_chunk_builder_utils.py | ||
| test_streaming_handler.py | ||
| test_streaming_overhead.py | ||
| test_text_completion_conversion.py | ||
| test_thread_pool_executor.py | ||
| test_token_counter.py | ||
| test_token_counter_tool.py | ||
| test_token_counter_tool_data.py | ||
| test_tokenizer.py | ||
| test_tool_fields.py | ||
| test_tool_search_spend_logging.py | ||
| test_url_utils.py | ||
| test_xai_oauth_routing.py | ||