Fixes for mapped test failures across multiple test suites:
1. **DataDog Integration** (test_datadog_llm_obs_agent):
- Fixed DD_API_KEY/DD_SITE requirement check to respect agent mode
- When LITELLM_DD_AGENT_HOST is set, API key and site are optional
2. **Prometheus Logging** (enterprise callbacks):
- Updated test assertions to include new model_id label
- Added client_ip and user_agent labels where expected
- Fixed label argument ordering to match implementation
3. **Proxy Server Tests** (get_image):
- Made tests async (get_image is an async function)
- Fixed os.path.exists mocking to not return True for cache file
- Removed unnecessary os.getenv mocking
4. **Vector Store Tests**:
- Fixed prisma_client patch path (use proxy_server.prisma_client)
- Added missing team_id and user_id attributes to mock UserAPIKeyAuth
5. **Key Management Tests**:
- Added 5-second buffer for timing comparison in budget_reset test
6. **Vertex AI Passthrough Tests**:
- Updated test to expect URL preservation when project/location present
7. **Presidio Guardrail Tests**:
- Removed incorrect assertion about session closure (sessions are cached)
8. **Azure SDK Tests**:
- Added acancel_batch to skip list (uses cached client)
9. **Cost Calculation Tests**:
- Fixed expected calculation to account for double-counting detection
* Add async_post_call_response_headers_hook to CustomLogger (#20070)
Allow CustomLogger callbacks to inject custom HTTP response headers
into streaming, non-streaming, and failure responses via a new
async_post_call_response_headers_hook method.
* async_post_call_response_headers_hook
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
* fix: bedrock invoke - does not support prompt-caching-scope
* fix: UNSUPPORTED_BEDROCK_INVOKE_BETA_PATTERNS
* init requirements.txt
* init README for claude Agent SDK
* fix: using converse models with UNSUPPORTED_BEDROCK_CONVERSE_BETA_PATTERNS
* fix main.py
* init: proxy_e2e_anthropic_messages_tests
* fix(hosted_vllm): route through base_llm_http_handler to support ssl_verify
The hosted_vllm provider was falling through to the OpenAI catch-all path
which doesn't pass ssl_verify to the HTTP client. This adds an explicit
elif branch that routes hosted_vllm through base_llm_http_handler.completion()
which properly passes ssl_verify to the httpx client.
- Add explicit hosted_vllm branch in main.py completion()
- Add ssl_verify tests for sync and async completion
- Update existing audio_url test to mock httpx instead of OpenAI client
* feat(hosted_vllm): add embedding support with ssl_verify
- Add HostedVLLMEmbeddingConfig for embedding transformations
- Register hosted_vllm embedding config in utils.py
- Add lazy import for embedding transformation module
- Add unit test for ssl_verify parameter handling
* fix(proxy): prevent provider-prefixed model leaks
Proxy clients should not see LiteLLM internal provider prefixes (e.g. hosted_vllm/...) in the OpenAI-compatible response model field.
This patch sanitizes the client-facing model name for both:
- Non-streaming responses returned from base_process_llm_request
- Streaming SSE chunks emitted by async_data_generator
Adds regression tests covering vLLM-style hosted_vllm routing for both streaming and non-streaming paths.
* chore(lint): suppress PLR0915 in proxy handler
Ruff started flagging ProxyBaseLLMRequestProcessing.base_process_llm_request() for too many statements after the hotpatch changes.
Add an explicit '# noqa: PLR0915' on the function definition to avoid a large refactor in a hotpatch.
* refactor(proxy): make model restamp explicit
Replace silent try/except/pass and type ignores with explicit model restamping.
- Logs an error when the downstream response model differs from the client-requested model
- Overwrites the OpenAI `model` field to the client-requested value to avoid leaking internal provider-prefixed identifiers
- Applies the same behavior to streaming chunks, logging the mismatch only once per stream
* chore(lint): drop PLR0915 suppression
The model restamping bugfix made `base_process_llm_request()` slightly exceed Ruff's
PLR0915 (too-many-statements) threshold, requiring a `# noqa` suppression.
Collapse consecutive `hidden_params` extractions into tuple unpacking so the
function falls back under the lint limit and remove the suppression.
No functional change intended; this keeps the proxy model-field bugfix intact
while aligning with project linting rules.
* chore(proxy): log model mismatches as warnings
These model-restamping logs are intentionally verbose: a mismatch is a useful signal
that an internal provider/deployment identifier may be leaking into the public
OpenAI response `model` field.
- Downgrade model mismatch logs from error -> warning
- Keep error logs only for cases where the proxy cannot read/override the model
* fix(proxy): preserve client model for streaming aliasing
Pre-call processing can rewrite request_data['model'] via model alias maps.\n\nOur streaming SSE generator was using the rewritten value when restamping chunk.model, which caused the public 'model' field to differ between streaming and non-streaming responses for alias-based requests.\n\nStash the original client model in request_data as _litellm_client_requested_model after the model has been routed, and prefer it when overriding the outgoing chunk model. Add a regression test for the alias-mapping case.
* chore(lint): satisfy PLR0915 in streaming generator
Ruff started flagging async_data_generator() for too many statements after adding model restamping logic.\n\nExtract the client-model selection + chunk restamping into small helpers to keep behavior unchanged while meeting the project's PLR0915 threshold.
The /health/services endpoint rejected datadog_llm_observability as an
unknown service, even though it was registered in the core callback
registry and __init__.py. Added it to both the Literal type hint and
the hardcoded validation list in the health endpoint.
Fixes#19478
The stream_chunk_builder function was not handling image chunks from
models like gemini-2.5-flash-image. When streaming responses were
reconstructed (e.g., for caching), images in delta.images were lost.
This adds handling for image_chunks similar to how audio, annotations,
and other delta fields are handled.
* fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic
Fixes#19891
Vertex AI Anthropic models don't support URL sources for images. LiteLLM
already converted image URLs to base64 for user messages, but not for tool
messages (role='tool'). This caused errors when using ToolOutputImage with
image_url in tool outputs.
Changes:
- Add force_base64 parameter to convert_to_anthropic_tool_result()
- Pass force_base64 to create_anthropic_image_param() for tool message images
- Calculate force_base64 in anthropic_messages_pt() based on llm_provider
- Add unit tests for tool message image handling
* chore: remove extra comment from test file header
* fix(vertex_ai): replace custom model names with actual Vertex AI model names in passthrough URLs (#19948)
When the passthrough URL already contains project and location, the code
was skipping the deployment lookup and forwarding the URL as-is to Vertex AI.
For custom model names like gcp/google/gemini-2.5-flash, Vertex AI returned
404 because it only knows the actual model name (gemini-2.5-flash).
The fix makes the deployment lookup always run, so the custom model name
gets replaced with the actual Vertex AI model name before forwarding.
* add _resolve_vertex_model_from_router
* fix: get_llm_provider
* Potential fix for code scanning alert no. 4020: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
