mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
8e2e99670d
* fix(router): avoid assistant prefill in Responses fallbacks * fix(router): keep prefill for Claude fallbacks that take it and skip constrained continuations A Responses fallback to a Claude model that takes assistant prefill now gets the partial response as prefill again, so it continues mid-word without the prompt-shape drift. Every other fallback keeps the request to continue. A stream cut after text no longer continues when the request constrains its output (a json_schema or json_object text format, or a forced tool_choice), since a continued JSON document or tool call would not parse. The client gets the primary's error, as on the chat path. The fallback's re-sent leading space at the seam is dropped, and the prefill shaping runs per deployment after the encrypted reasoning strip, so stripped reasoning is never replayed. * fix(router): guard SDK text_format, narrow forced tool_choice, read Claude base_model, and cap Responses continuation output Cut Responses streams that asked for structured output through the SDK's text_format now surface the primary's error instead of stitching two models' JSON. Only a required or named tool_choice counts as forced, so allowed_tools in auto mode and hosted tool choices still continue. A Claude deployment named only through base_model keeps its prefill continuation, and the continuation's max_output_tokens is lowered by what the cut stream already emitted. * fix(router): quote the partial's ending in the continue request and surface the primary's error after a cut tool call * fix(router): stop Responses fallbacks from continuing content-filter cuts and client-run tool calls A content filter that cuts a /v1/responses stream mid-answer now returns its error, as on chat, instead of letting a content_policy_fallbacks backup finish the filtered text. Shell, apply_patch, computer, local_shell, and MCP approval items count as a started tool call. Chat continuation goes back to treating any tool_choice object as constraining, as on main * fix(responses): count a stream of tool arguments as a started tool call * fix(router): let Responses fallbacks continue under allowed_tools auto and after server-run tool calls A Responses tool_choice of allowed_tools in auto mode no longer blocks the continuation, since the model may still answer in text. Hosted MCP calls and the bridge's server-run tool search no longer count as a started client tool call. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| azure_client_usage_test.py | ||
| ban_constant_numbers.py | ||
| ban_copy_deepcopy_kwargs.py | ||
| bedrock_pricing.py | ||
| callback_manager_test.py | ||
| check_data_replace_usage.py | ||
| check_e2e_no_raw_requests.py | ||
| check_endpoint_coverage.py | ||
| check_fastuuid_usage.py | ||
| check_get_model_cost_key_performance.py | ||
| check_guardrail_apply_decorator.py | ||
| check_licenses.py | ||
| check_migrations_lock_timeout.py | ||
| check_migrations_no_data_rewrites.py | ||
| check_no_publicly_known_master_key.py | ||
| check_prisma_binary_cache.py | ||
| check_provider_folders_documented.py | ||
| check_py310_typing_imports.py | ||
| check_spanattributes_value_usage.py | ||
| check_telemetry_imports.py | ||
| check_unbounded_in_lists.py | ||
| check_unsafe_enterprise_import.py | ||
| check_workflow_job_name_collisions.py | ||
| check_workflow_startup_safety.py | ||
| code_qa_check_tests.py | ||
| enforce_llms_folder_style.py | ||
| ensure_async_clients_test.py | ||
| info_log_check.py | ||
| liccheck.ini | ||
| license_cache.json | ||
| litellm_logging_code_coverage.py | ||
| log.txt | ||
| memory_test.py | ||
| prevent_key_leaks_in_exceptions.py | ||
| recursive_detector.py | ||
| router_code_coverage.py | ||
| router_enforce_line_length.py | ||
| test_aio_http_image_conversion.py | ||
| test_ban_set_verbose.py | ||
| test_chat_completion_imports.py | ||
| test_check_no_publicly_known_master_key.py | ||
| test_e2e_idp_stack.py | ||
| test_e2e_junit_report.py | ||
| test_e2e_metadata.py | ||
| test_generate_provider_endpoints_support.py | ||
| test_merge_smoke.py | ||
| test_no_hardcoded_secrets.py | ||
| test_provider_cache.py | ||
| test_provider_replay_harness.py | ||
| test_proxy_types_import.py | ||
| test_router_strategy_async.py | ||
| test_unit_passed_gate.py | ||
| test_workflow_job_name_collisions.py | ||
| unbounded_in_baseline.txt | ||