* fix(anthropic): use dynamic max_tokens based on model
When users don't specify max_tokens in requests to Anthropic models,
LiteLLM now uses the correct max_output_tokens value from the model
pricing JSON instead of a hardcoded 4096.
This fixes truncated responses for Claude 3.5+ models which support
higher output limits (8192 for Claude 3.5, 128k for Claude 3.7, etc.)
Fixes#8835
* fix(anthropic): restore env var support for backwards compatibility
Keep DEFAULT_ANTHROPIC_CHAT_MAX_TOKENS as fallback when model is not
found in JSON, allowing users to configure via environment variable.
Add direct Stability AI REST API support for image generation endpoints.
This enables using Stability's SD3, SD3.5, and Stable Image models via
LiteLLM's OpenAI-compatible interface.
Changes:
- Add STABILITY provider to LlmProviders enum
- Create StabilityImageGenerationConfig with multipart/form-data support
- Add OpenAI size to Stability aspect_ratio mapping
- Register provider in ProviderConfigManager
- Add 9 Stability models to model_prices_and_context_window.json
- Add documentation at docs/providers/stability.md
- Add 25 unit tests
Supported models:
- stability/sd3, sd3-large, sd3-large-turbo, sd3-medium
- stability/sd3.5-large, sd3.5-large-turbo, sd3.5-medium
- stability/stable-image-ultra, stable-image-core
Fixes#15337
Perplexity API returns pre-calculated costs in `usage.cost.total_cost`
that include the `request_cost` (fixed per-request fee). LiteLLM was
ignoring this and calculating costs manually, resulting in ~27x
underreporting (e.g., $0.0002 vs actual $0.006).
Changes:
- Use `usage.cost.total_cost` from Perplexity response when available
- Fall back to manual calculation if cost object not present
- Add tests for both behaviors
Add support for the 'xhigh' reasoning effort level on all gpt-5.2 model
variants, not just gpt-5.2-pro. This enables deeper reasoning capabilities
for the base gpt-5.2 model.
Changes:
- Add is_model_gpt_5_2_model() method to detect gpt-5.2 variants
- Update xhigh validation to allow gpt-5.2 models
- Update documentation with gpt-5.2 reasoning_effort support
- Update tests to reflect new behavior
- Add database and Redis setup to litellm_mapped_tests_proxy job in CircleCI
- Create shared test helpers in tests/test_litellm/proxy/conftest.py for proxy test setup
- Refactor health endpoint tests to use shared helpers from conftest
- Support automatic Redis cache configuration when REDIS_HOST is set
- Ensure minimal config is created when Redis/database is needed
Fixes#17821
The `is_cached_message` function crashed with TypeError when message
content was a string instead of a list of content blocks.
Changes:
- Add explicit `isinstance(content, list)` check before iteration
- Add `isinstance(content_item, dict)` check inside loop to skip non-dict items
- Use `.get()` for safer nested dict access
- Follow same pattern as `extract_ttl_from_cached_messages` (same module)
Tests:
- Add TestIsCachedMessage class with 9 test cases covering:
- String content (the reported bug)
- None content
- Missing content key
- Empty list content
- List with/without cache_control
- Mixed content types (strings + dicts)
- Wrong cache_control type
Moved speechConfig from RequestBody to GenerationConfig TypedDict so that
TTS configuration survives the filtering in _transform_request_body().
This fixes the 400 INVALID_ARGUMENT error when using Gemini TTS models
(gemini-2.5-flash-tts, gemini-2.5-flash-preview-tts, etc.) with both
vertex_ai and gemini providers.
Fixes: speechConfig was being created correctly in map_openai_params()
but then filtered out because GenerationConfig.__annotations__.keys()
didn't include it.
Tested with both preview and non-preview TTS model names and both
vertex_ai and gemini providers.
When using litellm.completion() with model="openai/responses/...", images
in tool message content were not being transformed from Chat Completion
format to Responses API format.
Chat Completion format: {"type": "image_url", "image_url": {"url": "..."}}
Responses API format: {"type": "input_image", "image_url": "..."}
This caused OpenAI to reject the request with error 400 since "image_url"
is not a valid type for function_call_output content.
This fix addresses two issues with Anthropic web search streaming:
1. Fix trailing {} in tool call arguments
- web_search_tool_result blocks have input_json_delta events that were
incorrectly emitted as tool calls
- Added current_content_block_type tracking to only emit tool calls for
tool_use and server_tool_use blocks
2. Capture web_search_tool_result for multi-turn
- The web_search_tool_result content comes ALL AT ONCE in content_block_start
- Now captured in provider_specific_fields.web_search_results
- stream_chunk_builder combines these for final message
- Allows multi-turn conversations to work with streaming web search
Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).
Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage
This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
* fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests
Azure AI Anthropic endpoint rejects max_retries and stream_options parameters
with "Extra inputs are not permitted" error. These are LiteLLM-internal
parameters that should not be sent to the API.
Fixes 400 Bad Request error when using azure_ai/claude-sonnet-4-5 and other
Azure AI Anthropic models.
* test(azure_ai): Add test for unsupported params removal in Azure AI Anthropic
Verifies that max_retries, stream_options, and extra_body are properly
removed from the request before sending to Azure AI Anthropic endpoint.
