mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-08 08:08:37 +00:00
5a392b9e5f
* fix(anthropic): stop forcing the response_format json tool when Claude will think A forced tool call skips Claude's thinking. The json_schema emulation forced json_tool_call unless thinking.type was enabled, so adaptive thinking and the models that think by default (Opus 5, Sonnet 5, Haiku 5.5, and the always-on families) lost their thinking on every structured-output request. AnthropicModelInfo.response_format_tool_can_be_forced is now the one check the Anthropic, Bedrock Converse (including the native output fallback) and Databricks paths use. It leaves the tool unforced when thinking is requested in any form or the model thinks by default, and keeps forcing it when the request turns thinking off (thinking disabled or reasoning_effort none) on a model that accepts forced tools. The new thinking_on_by_default cost map key marks Opus 5, Sonnet 5 and Haiku 5.5 on the providers that reach these paths, and a miss-only fallback rule covers unmapped Claude 5 and later ids. * fix(bedrock): route native output and thinking token counts by whether Claude will think Bedrock's native output opt-in now uses the same will-think check as the json tool, so Claude 5 models that think by default with no thinking param go native. Converse also asks for /usage/output_tokens_details on those models, so reasoning tokens are reported for them. * fix(anthropic): tell Claude to answer through the unforced response_format tool Moves the integration tests that check the forced JSON tool from Claude 5 models to Opus 4.8, since Claude 5 models now keep their default thinking and take the unforced tool or native output * fix(anthropic): keep open-object schemas on the json tool when Claude will think Native output closes every object, so an open object came back empty. Also declare thinking_on_by_default in the Rust model catalog and expect the Converse thinking-token field in test_main * fix(anthropic): hold streamed prose until the JSON tool decides the answer In json mode with the unforced JSON tool, Claude can write its working before it calls the tool. The stream now holds that text, drops it when the JSON tool answers, and sends it when Claude calls a caller tool or skips the JSON tool, matching the non-streamed reply. * test(anthropic): build the stream test events without nested comprehensions * fix(anthropic): keep always-on thinking models unforced, open objects native when thinking, and the tool instruction Claude-only * test(anthropic): keep main's open-object integration cases and cover the unforced tool on a default-thinking model * fix(anthropic): treat thinking False as thinking off so the JSON tool stays forced Cover thinking False on the Anthropic and Converse forcing tests, and pin that a Converse json_schema stream with the unforced JSON tool never sends Claude's working text as content * fix(anthropic): drop the thinking False forcing case that completion never reaches litellm.completion turns thinking False into no thinking param before any route sees it, so the case was dead on the request path and False keeps meaning the model's default * test(bedrock): collect the Converse json_schema stream chunks in one assignment --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _prompt_cache_pin_support.py | ||
| _reasoning_effort_levels_support.py | ||
| either_role.json | ||
| test_advisor_failure_cooldown.py | ||
| test_anthropic_overloaded_cooldown_owned_proxy.py | ||
| test_audit_any_sweep_chaos.py | ||
| test_auto_router_management.py | ||
| test_azure_passthrough_spend_row.py | ||
| test_base_model_context_window_owned_proxy.py | ||
| test_batch_mode_deployments_wire.py | ||
| test_complexity_router_codex_compaction.py | ||
| test_complexity_router_databricks_classifier.py | ||
| test_complexity_router_llm_classifier.py | ||
| test_declared_provider_bare_model_routes.py | ||
| test_deployment_mode_routing_wire.py | ||
| test_deployment_num_retries_generic_routes_wire.py | ||
| test_encrypted_content_affinity_no_peer.py | ||
| test_end_user_region_routing.py | ||
| test_enforce_model_rate_limits_daily_limits.py | ||
| test_fallback_hop_foreign_encrypted_reasoning.py | ||
| test_fallback_retry_header_contracts.py | ||
| test_include_fallback_errors_wire.py | ||
| test_key_max_parallel_requests.py | ||
| test_key_router_settings_image_generation.py | ||
| test_key_tpm_reservation.py | ||
| test_least_busy_concurrent_decisions.py | ||
| test_messages_stream_retry_budget_sources_owned_proxy.py | ||
| test_model_capability_flags.py | ||
| test_non_chat_model_endpoint_gate.py | ||
| test_observed_routing.py | ||
| test_otpm_default_reservation_fraction.py | ||
| test_payment_required_cooldown.py | ||
| test_payment_required_mapping.py | ||
| test_pdf_document_pre_call_checks_owned_proxy.py | ||
| test_per_model_rate_limit_fallbacks.py | ||
| test_per_model_rate_limit_fallbacks_lifecycle.py | ||
| test_pre_call_response_format_filter.py | ||
| test_priority_model_tpm_enforcement.py | ||
| test_priority_rate_limit_headers.py | ||
| test_priority_scheduler_queue_cleanup.py | ||
| test_prompt_cache_pin_memory_routes.py | ||
| test_prompt_cache_pin_ttl.py | ||
| test_prompt_cache_pin_ttl_chaos.py | ||
| test_prompt_caching_affinity_tool_call_marks.py | ||
| test_ptu_shares.py | ||
| test_ptu_shares_regex_timing.py | ||
| test_reasoning_effort_levels_chaos.py | ||
| test_reasoning_effort_levels_wire.py | ||
| test_redis_increment_with_floor.py | ||
| test_redis_recovery.py | ||
| test_responses_websocket_turn_admission.py | ||
| test_router_strategies_count_async_requests_once.py | ||
| test_silent_model_shadow_wire.py | ||
| test_stale_cost_map_boot.py | ||
| test_tag_routing_prefix.py | ||
| test_team_tag_routing.py | ||
| test_team_wildcard_precedence.py | ||
| test_usage_based_routing_redis_reads.py | ||
| test_user_config_routing.py | ||
| test_video_poll_routing_wire.py | ||
| test_vllm_passthrough_spend_row.py | ||
| test_vllm_passthrough_stream_relay.py | ||
| test_vllm_passthrough_stream_usage_settings.py | ||
| test_wildcard_model_listing.py | ||