* fix(hosted_vllm): route through base_llm_http_handler to support ssl_verify
The hosted_vllm provider was falling through to the OpenAI catch-all path
which doesn't pass ssl_verify to the HTTP client. This adds an explicit
elif branch that routes hosted_vllm through base_llm_http_handler.completion()
which properly passes ssl_verify to the httpx client.
- Add explicit hosted_vllm branch in main.py completion()
- Add ssl_verify tests for sync and async completion
- Update existing audio_url test to mock httpx instead of OpenAI client
* feat(hosted_vllm): add embedding support with ssl_verify
- Add HostedVLLMEmbeddingConfig for embedding transformations
- Register hosted_vllm embedding config in utils.py
- Add lazy import for embedding transformation module
- Add unit test for ssl_verify parameter handling
* fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic
Fixes#19891
Vertex AI Anthropic models don't support URL sources for images. LiteLLM
already converted image URLs to base64 for user messages, but not for tool
messages (role='tool'). This caused errors when using ToolOutputImage with
image_url in tool outputs.
Changes:
- Add force_base64 parameter to convert_to_anthropic_tool_result()
- Pass force_base64 to create_anthropic_image_param() for tool message images
- Calculate force_base64 in anthropic_messages_pt() based on llm_provider
- Add unit tests for tool message image handling
* chore: remove extra comment from test file header
The regex in get_vertex_model_id_from_url() was using [^/:]+
which stopped at the first slash, truncating model names like
'gcp/google/gemini-2.5-flash' to just 'gcp'. This caused
access_groups checks to fail for custom model names.
Changed the pattern to [^:]+ to allow slashes in model names,
only stopping at the colon before the action (e.g., :generateContent).
- Add whitelist-based filtering for anthropic_beta headers
- Only allow Bedrock-supported beta flags (computer-use, tool-search, etc.)
- Filter out unsupported flags like mcp-servers, structured-outputs
- Remove output_format parameter from Bedrock Invoke requests
- Force tool-based structured outputs when response_format is used
Fixes#16726
Extend advanced-tool-use header translation to include Claude Sonnet 4.5
in addition to Opus 4.5 on Bedrock Invoke API.
When Claude Code sends the advanced-tool-use-2025-11-20 header, it now
gets correctly translated to Bedrock-specific headers for both:
- Claude Opus 4.5
- Claude Sonnet 4.5
Headers translated:
- tool-search-tool-2025-10-19
- tool-examples-2025-10-29
Fixes defer_loading validation error on Bedrock with Sonnet 4.5.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
Adds support for Anthropic-style 'thinking' parameter in hosted_vllm,
converting it to OpenAI-style 'reasoning_effort' since vLLM is
OpenAI-compatible.
This enables users to use Claude Code CLI with hosted vLLM models
like GLM-4.6/4.7 through the /v1/messages endpoint.
Mapping (same as Anthropic adapter):
- budget_tokens >= 10000 -> "high"
- budget_tokens >= 5000 -> "medium"
- budget_tokens >= 2000 -> "low"
- budget_tokens < 2000 -> "minimal"
Fixes#19761
Add support for /embeddings endpoint via Vercel AI Gateway.
Closes#19658
Changes:
- Add VercelAIGatewayEmbeddingConfig in litellm/llms/vercel_ai_gateway/embedding/
- Register provider in utils.py and main.py
- Add unit tests for embedding transformation
- Update documentation with embeddings examples
Usage:
```python
from litellm import embedding
response = embedding(
model="vercel_ai_gateway/openai/text-embedding-3-small",
input="Hello world",
api_key="your-api-key"
)
```
The Azure Responses API uses a different schema (flattened) for tools compared to the standard OpenAI/Azure Chat Completions API (nested). This caused a `BadRequestError` when users passed standard tool definitions.
Changes:
- Implemented tool flattening logic in `AzureOpenAIResponsesAPIConfig.transform_responses_api_request`.
- Added comprehensive unit tests in test_azure_transformation.py to verify nested-to-flat transformation, pass-through of flat tools, and immutability.
- Ensures cross-provider compatibility for tool definitions.
Fixes#19523
