* fix: check for model_response_choices before guardrail input
* test: add tests for responses api translation
* fix: protect other guardrail translations
* refactor: remove type ignores
* anthropic request body got mutated fix
* add warning when extra_body is provided but user is non premium
* fix: resolve mypy union-attr errors in anthropic guardrail handler
Cast choices[0] to Choices type before accessing .message attribute
to satisfy mypy's union type checking for Choices | StreamingChoices.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* add logger when model response has no choices for streaming /response and /messages
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 541a2b075a.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 716ea0caa1.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#18137
Similar to the fix for web_search_tool_result (#17746, #17798), this PR
preserves web_fetch_tool_result blocks in multi-turn conversations.
Changes:
- Add handling for web_fetch_tool_result in transformation.py (non-streaming)
- Add capture of web_fetch_tool_result in handler.py (streaming)
- Fix streaming tool arguments bug where empty input {} was prepended to
actual arguments by using empty string instead of str({})
- Add unit tests for web_fetch_tool_result handling
Claude 3.7 Sonnet's default max_output_tokens is 64000, not 128000.
The 128K output limit requires the beta header 'output-128k-2025-02-19'.
This fixes the integration test failure where requests with max_tokens=128000
were being rejected by the Anthropic API.
Fixes test_multiturn_tool_calls in test_anthropic_responses_api.py
* fix(anthropic): use dynamic max_tokens based on model
When users don't specify max_tokens in requests to Anthropic models,
LiteLLM now uses the correct max_output_tokens value from the model
pricing JSON instead of a hardcoded 4096.
This fixes truncated responses for Claude 3.5+ models which support
higher output limits (8192 for Claude 3.5, 128k for Claude 3.7, etc.)
Fixes#8835
* fix(anthropic): restore env var support for backwards compatibility
Keep DEFAULT_ANTHROPIC_CHAT_MAX_TOKENS as fallback when model is not
found in JSON, allowing users to configure via environment variable.
This fix addresses two issues with Anthropic web search streaming:
1. Fix trailing {} in tool call arguments
- web_search_tool_result blocks have input_json_delta events that were
incorrectly emitted as tool calls
- Added current_content_block_type tracking to only emit tool calls for
tool_use and server_tool_use blocks
2. Capture web_search_tool_result for multi-turn
- The web_search_tool_result content comes ALL AT ONCE in content_block_start
- Now captured in provider_specific_fields.web_search_results
- stream_chunk_builder combines these for final message
- Allows multi-turn conversations to work with streaming web search
- Extract web_search_tool_result blocks in extract_response_content()
- Store web_search_results in provider_specific_fields for round-trip
- Detect srvtoolu_ prefix to reconstruct as server_tool_use (not tool_use)
- Add corresponding web_search_tool_result after server_tool_use blocks
This ensures multi-turn conversations with Anthropic web search + custom
tools work correctly without Anthropic expecting tool_result for server-
side tool executions.
Fixes#17473 - Anthropic streaming fails with JSONDecodeError when
network fragmentation causes SSE data to arrive in partial chunks.
Changes:
- Add accumulated_json buffer and chunk_type to ModelResponseIterator
- Add _handle_accumulated_json_chunk() to accumulate partial JSON
- Add _parse_sse_data() to handle both complete and partial chunks
- Modify __next__ and __anext__ to use accumulation logic
- Add unit tests for partial chunk handling
When translating system messages for the Anthropic API, empty text
content blocks cause the error "messages: text content blocks must be
non-empty". This fix skips empty string content and empty text blocks
in list content to prevent this error.
Fixes issue with Vertex AI Anthropic API calls.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
* Added tool search support for anthropic
* Add programtic tool calling support
* Add tool use input examples support
* Add anthropic effort param support
* Add anthropic effort param support
* Add blog for new features
* fix mypy and lint errors
* fix mypy and lint errors
* fix mypy and lint errors
* fix mypy and lint errors
* Add better handling
* Add better handling
* feat(anthropic/chat/transformations): for claude-4-5-sonnet and opus-4-1 support passing structured output to anthropic api
* docs: document new feature
* fix: fix output format
* fix: cleanup
* fix(transformation.py): conditionally pass in json tool call
* fix: support ARIZE_SPACE_ID instead of ARIZE_SPACE_KEY
* docs(arize_integration.md): cleanup arize docs
* feat(callback_info_helpers.tsx): allow setting arize space id via ui
* fix: fix linting error
* fix(opentelemetry.py): working arize phoenix root span tracing
* add memory tool in anthropic.py
* add memory tool test
* make format
* update transformation
* adding memory to hosted tools
* add test
* make format
- Fixed 'missing finish_reason for choice 1' error with reasoning_effort
- Anthropic sends multiple content blocks with different indices
- OpenAI expects all content in a single choice at index=0
- Added comprehensive tests for text-only, text+tool, and multiple tools
* fix(anthropic): fix streaming + response_format + tools bug
- Fix _handle_json_mode_chunk to only convert response_format tools to content
- Regular user tools now remain as proper tool_calls in streaming mode
- Add comprehensive test for the fix
- Resolves issue where all tools were incorrectly converted to content chunks
Before: All tools converted to content with different indices
After: Only response_format tool converted, regular tools remain as tool_calls
* fix(anthropic): improve streaming + response_format + tools handling
* fix: lint error (too many statements)
* fix(anthropic): correct finish_reason for streaming response_format tools
* [Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage (BerriAI/litellm#11920)
* [Fix] Missed a null check and used a cast instead by error
* build(model_prices_and_context_window.json): mark all gemini-2.5 models as supporting pdf input
Closes https://github.com/BerriAI/litellm/issues/11881
* fix(anthropic_transformation.py): set custom llm provider custom property
Fixes https://github.com/BerriAI/litellm/issues/11861
* test: add unit test for checking supports_reasoning
* test: add test for vertex ai flow
* feat(bedrock/anthropic): ensure thinking param correctly passed for bedrock/invoke
* docs(index.md): add stable pip package
* fix(anthropic/chat/transformation.py): add 'none' tool choice mapping
Allows disabling anthropic tool calling
Maintain parity
* fix(transformation.py): if tool_choice="none" ignore 'disable_parallel_Tool_use'
unsupported param from anthropic - makes sense as the 'none' implies no tool calls are being made
* fix(anthropic/chat/transformation.py): append prefix to start of assistant response, if set
ensures assistant response contains complete response
* fix(anthropic/chat/transformation.py): add flag to allow user to opt out of enabling prefix in prompt
* fix(anthropic/chat/transformation.py): working e2e support for prefix prompt in assistant response
* feat(networking.tsx): always include model access groups on UI
show admin created access groups when giving key/user/team model permissions
* feat(add_model_tab.tsx): initial ui component for adding to an existing model access group
allows user to add model to an access group (simplify giving users/keys/teams model access)
* feat(proxy_server.py): add 'only_model_access_groups' flag support to `/v1/models`
simplifies listing available access groups on UI
* test: add e2e test for new only_model_access_groups param
* feat(add_model_tab.tsx): allow adding+viewing model access groups on models tab
make feature functional on UI
* feat(view_users.tsx): route edit user to user info page
more detailed user edit
* feat(columns.tsx): route edit user to user info page
more detailed user edit
* fix(columns.tsx): fix linting error
* build(ui/): fix linting errors
* feat(anthropic/): initial commit adding working mcp tool call support
pass in mcp tool via `tools` and litellm will handle translating it to the right anthropic param
* feat(anthropic/): map openai mcp tool to anthropic mcp tool
allows usage within responses api
* fix(databricks/transformation.py): fix databricks linting error
* test(test_anthropic_chat_transformation.py): fix test
* test: update test
* fix(anthropic/chat/transformation.py): add dummy tool call
* fix(anthropic/chat/handler.py): Fixes https://github.com/BerriAI/litellm/issues/10328
Adopts changes from https://github.com/BerriAI/litellm/pull/10329
* fix(vertex_and_google_ai_studio.py): don't set 'include thoughts' if thinking budget = 0
VertexAI raises errors
* fix(vertex_llm_base.py): new function for deciding the api base, handles 'global' api base
Fixes https://github.com/BerriAI/litellm/issues/11190
* fix(vertex_ai/partner_models): fix instrumentation for custom api base check
* refactor(vertex_ai/partner): refactor function to keep below 50 LOC
* fix(vertex_ai/gemini): remove parallel tool calls error for >1 tool - just ignore (prevent call from failing)
* fix: fix linting error
