* fix(unified_guardrails.py): send all chunks on completion of final stream
* feat(generic_guardrail_api.py): handle tool call response on streaming LLM responses
* fix(anthropic/chat/guardrail_translation): initial commit adding anthropic tool response streaming guardrails
enables guardrail checks on tool response from llm's to work via `/v1/messages`
* feat(anthropic/): working guardrail checks on tool response from LLMs
ensures guardrail checks on anthropic /v1/messages works as expected
* feat(responses/guardrail_translation): support tool call response guardrails on streaming for /v1/responses
ensures complete coverage of tool call responses
* refactor(openai.py): refactor to use consistent pydantic model for responses api tool response on streaming
enables non-openai model tool call response to work correctly with guardrail checks on /v1/responses
* test: update tests
* fix: fix linting error
* fix: fix failing tests
* fix: fix import errors
* fix(openai/chat/guardrail_transformation): fix final chunk returned on streaming
* docs: add OpenAI-compatible API limitations for Anthropic thinking
Document the fundamental incompatibility between Anthropic extended
thinking and OpenAI-compatible API clients. Explains:
- Why thinking_blocks must be resent (stateless vs stateful APIs)
- OpenAI vs Anthropic architecture differences
- Solutions for client developers
* Update docs
* fix: auto-drop thinking param when thinking_blocks missing
When modify_params=True, LiteLLM now automatically drops the 'thinking'
param if the last assistant message with tool_calls is missing
thinking_blocks. This prevents the Anthropic error:
"Expected thinking or redacted_thinking, but found tool_use"
This workaround addresses the OpenAI-Anthropic API incompatibility where
OpenAI-compatible clients don't preserve thinking_blocks.
When calling Vertex AI Model Garden custom endpoints with service account
credentials (instead of ADC), the credentials were not passed to the
PredictionServiceClient, resulting in "default credentials not found" error.
- Move credential loading before cache check to ensure availability
- Pass credentials to sync/async PredictionServiceClient
- Add vertex_credentials param to async_completion and async_streaming
Closes#8597
* fix(anthropic): use dynamic max_tokens based on model
When users don't specify max_tokens in requests to Anthropic models,
LiteLLM now uses the correct max_output_tokens value from the model
pricing JSON instead of a hardcoded 4096.
This fixes truncated responses for Claude 3.5+ models which support
higher output limits (8192 for Claude 3.5, 128k for Claude 3.7, etc.)
Fixes#8835
* fix(anthropic): restore env var support for backwards compatibility
Keep DEFAULT_ANTHROPIC_CHAT_MAX_TOKENS as fallback when model is not
found in JSON, allowing users to configure via environment variable.
