* fix(hosted_vllm): route through base_llm_http_handler to support ssl_verify
The hosted_vllm provider was falling through to the OpenAI catch-all path
which doesn't pass ssl_verify to the HTTP client. This adds an explicit
elif branch that routes hosted_vllm through base_llm_http_handler.completion()
which properly passes ssl_verify to the httpx client.
- Add explicit hosted_vllm branch in main.py completion()
- Add ssl_verify tests for sync and async completion
- Update existing audio_url test to mock httpx instead of OpenAI client
* feat(hosted_vllm): add embedding support with ssl_verify
- Add HostedVLLMEmbeddingConfig for embedding transformations
- Register hosted_vllm embedding config in utils.py
- Add lazy import for embedding transformation module
- Add unit test for ssl_verify parameter handling
* fix(proxy): prevent provider-prefixed model leaks
Proxy clients should not see LiteLLM internal provider prefixes (e.g. hosted_vllm/...) in the OpenAI-compatible response model field.
This patch sanitizes the client-facing model name for both:
- Non-streaming responses returned from base_process_llm_request
- Streaming SSE chunks emitted by async_data_generator
Adds regression tests covering vLLM-style hosted_vllm routing for both streaming and non-streaming paths.
* chore(lint): suppress PLR0915 in proxy handler
Ruff started flagging ProxyBaseLLMRequestProcessing.base_process_llm_request() for too many statements after the hotpatch changes.
Add an explicit '# noqa: PLR0915' on the function definition to avoid a large refactor in a hotpatch.
* refactor(proxy): make model restamp explicit
Replace silent try/except/pass and type ignores with explicit model restamping.
- Logs an error when the downstream response model differs from the client-requested model
- Overwrites the OpenAI `model` field to the client-requested value to avoid leaking internal provider-prefixed identifiers
- Applies the same behavior to streaming chunks, logging the mismatch only once per stream
* chore(lint): drop PLR0915 suppression
The model restamping bugfix made `base_process_llm_request()` slightly exceed Ruff's
PLR0915 (too-many-statements) threshold, requiring a `# noqa` suppression.
Collapse consecutive `hidden_params` extractions into tuple unpacking so the
function falls back under the lint limit and remove the suppression.
No functional change intended; this keeps the proxy model-field bugfix intact
while aligning with project linting rules.
* chore(proxy): log model mismatches as warnings
These model-restamping logs are intentionally verbose: a mismatch is a useful signal
that an internal provider/deployment identifier may be leaking into the public
OpenAI response `model` field.
- Downgrade model mismatch logs from error -> warning
- Keep error logs only for cases where the proxy cannot read/override the model
* fix(proxy): preserve client model for streaming aliasing
Pre-call processing can rewrite request_data['model'] via model alias maps.\n\nOur streaming SSE generator was using the rewritten value when restamping chunk.model, which caused the public 'model' field to differ between streaming and non-streaming responses for alias-based requests.\n\nStash the original client model in request_data as _litellm_client_requested_model after the model has been routed, and prefer it when overriding the outgoing chunk model. Add a regression test for the alias-mapping case.
* chore(lint): satisfy PLR0915 in streaming generator
Ruff started flagging async_data_generator() for too many statements after adding model restamping logic.\n\nExtract the client-model selection + chunk restamping into small helpers to keep behavior unchanged while meeting the project's PLR0915 threshold.
The /health/services endpoint rejected datadog_llm_observability as an
unknown service, even though it was registered in the core callback
registry and __init__.py. Added it to both the Literal type hint and
the hardcoded validation list in the health endpoint.
Fixes#16920 for users of the stable release images.
The previous fix (PR #18092) added libsndfile to docker/Dockerfile.alpine,
but stable releases are built from the main Dockerfile (Wolfi-based),
not the Alpine variant.
Fixes#19478
The stream_chunk_builder function was not handling image chunks from
models like gemini-2.5-flash-image. When streaming responses were
reconstructed (e.g., for caching), images in delta.images were lost.
This adds handling for image_chunks similar to how audio, annotations,
and other delta fields are handled.
* fix(proxy_server): pass search_tools to Router during DB-triggered initialization
* fix search tools from db
* add missing statement to handle from db
* fix import issues to pass lint errors
* fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic
Fixes#19891
Vertex AI Anthropic models don't support URL sources for images. LiteLLM
already converted image URLs to base64 for user messages, but not for tool
messages (role='tool'). This caused errors when using ToolOutputImage with
image_url in tool outputs.
Changes:
- Add force_base64 parameter to convert_to_anthropic_tool_result()
- Pass force_base64 to create_anthropic_image_param() for tool message images
- Calculate force_base64 in anthropic_messages_pt() based on llm_provider
- Add unit tests for tool message image handling
* chore: remove extra comment from test file header
* fix(vertex_ai): replace custom model names with actual Vertex AI model names in passthrough URLs (#19948)
When the passthrough URL already contains project and location, the code
was skipping the deployment lookup and forwarding the URL as-is to Vertex AI.
For custom model names like gcp/google/gemini-2.5-flash, Vertex AI returned
404 because it only knows the actual model name (gemini-2.5-flash).
The fix makes the deployment lookup always run, so the custom model name
gets replaced with the actual Vertex AI model name before forwarding.
* add _resolve_vertex_model_from_router
* fix: get_llm_provider
* Potential fix for code scanning alert no. 4020: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
The regex in get_vertex_model_id_from_url() was using [^/:]+
which stopped at the first slash, truncating model names like
'gcp/google/gemini-2.5-flash' to just 'gcp'. This caused
access_groups checks to fail for custom model names.
Changed the pattern to [^:]+ to allow slashes in model names,
only stopping at the colon before the action (e.g., :generateContent).
* Cleanup code for user cli auth, and make sure not to prompt user for team multiple times while polling
* Adding tests
* Cleanup normalize teams some more
When using langfuse_otel callback, empty traces were being sent to Langfuse
for requests that didn't result in actual LLM calls (e.g., auth operations,
health checks, failed requests). These traces contained only internal proxy
operations (auth, postgres, proxy_pre_call) with no useful LLM data.
Root cause: LangfuseOtelLogger extends OpenTelemetry, which sets itself as
the proxy's open_telemetry_logger. This caused create_litellm_proxy_request_started_span
to be called for every request, creating a parent span that was sent to Langfuse
even when no LLM call occurred.
Fix: Override create_litellm_proxy_request_started_span in LangfuseOtelLogger
to return None, preventing the creation of empty parent spans. This is consistent
with the existing overrides for async_service_success_hook and async_service_failure_hook
which already prevent service-level logs from being sent to Langfuse.
Fixes: Empty traces in Langfuse v3 when using langfuse_otel callback
