Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).
Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage
This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
* fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests
Azure AI Anthropic endpoint rejects max_retries and stream_options parameters
with "Extra inputs are not permitted" error. These are LiteLLM-internal
parameters that should not be sent to the API.
Fixes 400 Bad Request error when using azure_ai/claude-sonnet-4-5 and other
Azure AI Anthropic models.
* test(azure_ai): Add test for unsupported params removal in Azure AI Anthropic
Verifies that max_retries, stream_options, and extra_body are properly
removed from the request before sending to Azure AI Anthropic endpoint.
* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
- Extract web_search_tool_result blocks in extract_response_content()
- Store web_search_results in provider_specific_fields for round-trip
- Detect srvtoolu_ prefix to reconstruct as server_tool_use (not tool_use)
- Add corresponding web_search_tool_result after server_tool_use blocks
This ensures multi-turn conversations with Anthropic web search + custom
tools work correctly without Anthropic expecting tool_result for server-
side tool executions.
* feat(voyage): add rerank API support
Add support for Voyage AI rerank models (rerank-2.5, rerank-2.5-lite,
rerank-2, rerank-2-lite) to the LiteLLM rerank API.
Changes:
- Add VoyageRerankConfig transformation class
- Register voyage provider in rerank_api/main.py
- Add voyage case in utils.py get_provider_rerank_config
- Add rerank-2.5 and rerank-2.5-lite models to pricing JSON
- Add unit tests for transformation logic
- Update documentation for voyage.md and rerank.md
Usage:
```python
from litellm import rerank
response = rerank(
model="voyage/rerank-2.5",
query="What is the capital of France?",
documents=["Paris is...", "London is..."],
top_n=3,
)
```
* refactor(voyage): simplify rerank transformation code
Remove verbose docstrings to align with other providers (jina_ai pattern).
No functional changes - 168 lines vs 169 for jina_ai.
* fix(voyage): remove incorrect input_cost_per_query from rerank models
Voyage AI charges per token, not per query. The input_cost_per_query
field was incorrectly set to the same value as input_cost_per_token
in the existing rerank-2 and rerank-2-lite models.
Removes input_cost_per_query from all Voyage rerank models:
- voyage/rerank-2
- voyage/rerank-2-lite
- voyage/rerank-2.5
- voyage/rerank-2.5-lite
Pricing source: https://docs.voyageai.com/docs/pricing
