- Update test_anthropic_via_responses_api expected_events to include
CONTENT_PART_ADDED between OUTPUT_ITEM_ADDED and OUTPUT_TEXT_DELTA
- Add TestEnsureOutputItemContentPartAdded with 3 mock tests:
message item emits content_part.added, reasoning item does not,
and the event is only emitted once
Verify that spend_logs_metadata is correctly merged into combined_metadata
and flows through to Prometheus custom labels. Tests cover: basic extraction,
precedence when keys overlap, all three metadata sources combined, and None
handling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Thread project_alias alongside project_id through the metadata pipeline so
callbacks receive the human-readable project name. DRY up duplicate metadata
dict construction in proxy_track_cost_callback and pass_through_endpoints by
reusing get_sanitized_user_information_from_key — future metadata fields only
need adding in one place.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two independent bugs prevented post-call OpenAI Moderation guardrail
results from reaching downstream logging callbacks (Langfuse, Datadog).
Bug 1: process_output_response() created a throwaway request_data dict,
so guardrail info written by @log_guardrail_information was discarded.
Fixed by threading the real request_data from the unified guardrail
dispatcher through all 13 BaseTranslation handlers, with litellm_metadata
injection preserved for third-party guardrails (Zscaler, Prompt Security).
Also extended to process_output_streaming_response for consistency.
Bug 2: The @log_guardrail_information decorator collapsed the full
moderation API response (categories, scores, flagged status) to "allow".
Fixed by overriding _process_response/_process_error on
OpenAIModerationGuardrail to stash and log the full response, following
the established Model Armor pattern.
Both bridges (Responses→CC and CC→Responses) independently encoded the
same field mapping knowledge. This extracts 4 shared mappings into a
single module so future changes only need to happen in one place.
Shared mappings:
- status ↔ finish_reason bidirectional dicts and functions
- response_format ↔ text.format paired conversion functions
- provider_specific_fields normalization helper
- usage field name translation (input_tokens ↔ prompt_tokens, etc.)
No behavioral changes — bridge methods now delegate to the shared module.
The `dimensions` parameter was correctly mapped to `outputDimensionality`
in `optional_params` but never placed in the request body. The Vertex AI
predict endpoint expects it under a `parameters` field.
Add `parameters` dict to `VertexMultimodalEmbeddingRequest` TypedDict and
populate it from `optional_params` in `transform_embedding_request`.
Fixes#24392
Gemini API returns a DOCUMENT modality in promptTokensDetails for PDF
inputs, but the token parser only handled TEXT, IMAGE, AUDIO, and VIDEO.
DOCUMENT tokens were silently dropped, causing cost to be undercounted
by up to 99% for PDF-heavy requests.
Map DOCUMENT tokens to text_tokens since Gemini bills documents at the
text token rate. Applied to all four modality parser loops:
promptTokensDetails, cacheTokensDetails, responseTokensDetails, and
candidatesTokensDetails.
Fixes#24375
Replace _is_gemini_3_model() substring check with a
web_search_billing_unit field in model_prices JSON:
- "per_query": each search query billed individually (Gemini 3.x)
- "per_prompt" (default): flat fee per grounded API call (Gemini 2.x)
Add web_search_billing_unit to 23 Gemini 3.x model entries.
Update docs and tests accordingly.
Add tests for the gpt-5.1/5.2/5.4 reasoning.effort interaction:
- gpt-5.1 with no reasoning allows flexible temperature
- gpt-5.1 with effort='high' drops temperature
- gpt-5.4 with effort='none' allows flexible temperature
- Gemini 2.x charges per grounded prompt (flat $0.035), clamped to 1
regardless of internal query count
- Gemini 3.x charges per search query ($0.014 each)
- Extract web_search_requests from groundingMetadata in non-streaming
responses (parity with streaming path)
- Add search_context_cost_per_query to vertex_ai and base Gemini entries
- Move tests to tests/test_litellm/ (CI directory)
The Gemini web search cost calculator hardcoded $0.035 per request,
which is only correct for Gemini 2.x models. Gemini 3.x models
charge $0.014 per request.
Read from search_context_cost_per_query in model_info (same field
used by Anthropic, OpenAI, and Perplexity) with fallback to the
legacy $0.035 for models not yet updated in the JSON.
Also add search_context_cost_per_query to all 25 Gemini models
that support web search in model_prices_and_context_window.json.
Fixes#24369
The Responses API map_openai_params passed all params through without
applying model-specific validation. GPT-5 models (except gpt-5-chat)
only accept temperature=1 unless reasoning.effort="none" on models
that support it (5.1, 5.2, 5.4).
Reuse the existing OpenAIGPT5Config logic from chat completions to
validate temperature in the Responses API path. With drop_params=True,
unsupported temperature values are silently dropped; without it,
UnsupportedParamsError is raised.
Fixes#16090
The Gemini batch embedding transformation was spreading all
optional_params into the request body via **gemini_params. Params
like max_tokens (injected by add_provider_specific_params_to_optional_params)
would reach the Gemini API and cause a 400 BadRequestError.
Extract _filter_embed_params() that maps dimensions/task_type and
keeps only the fields Gemini embeddings actually accept
(outputDimensionality, taskType, title). Applied to both
transform_openai_input_gemini_content and
transform_openai_input_gemini_embed_content.
This also fixes drop_params: true not preventing the error, since
the param was re-injected after the drop_params check.
Fixes#24293
When the Responses API converts function_call and message output items
into chat completion messages, they can become two consecutive assistant
messages. The Bedrock Converse transformer merges these into one, but
the merge preserves input order — so if function_call came first, the
toolUse block ends up before the text block.
Claude models (Sonnet 4, Haiku 3.5+) reject this ordering with:
"tool_use ids were found without tool_result blocks immediately after"
Add _sort_bedrock_assistant_content_blocks() that reorders content
blocks within assistant messages: reasoningContent → text → toolUse.
Applied in both sync and async Bedrock Converse transformation paths.
Fixes#24361
When Gemini sends tool call arguments in the same streaming chunk as a
content block transition, the Anthropic adapter discarded the
processed_chunk containing the input_json_delta. This caused tool_use
blocks to arrive with empty input: {}.
Queue the processed_chunk alongside the block transition events when it
contains input_json_delta data. Applied to both sync and async paths.
Fixes#24134
When Azure sends stream_options.include_usage=True, it emits an initial
chunk with choices=[] (prompt_filter_results) before the first content
chunk. Previously, LiteLLM inflated this empty-choices chunk with a
default StreamingChoices, which consumed the sent_first_chunk flag and
caused strip_role_from_delta to strip role from the real first chunk.
Additionally, the first real chunk with role='assistant' and content=''
was discarded by is_chunk_non_empty as "empty".
This fix:
- Forwards chunks with choices=[] faithfully (no inflated default)
- Only marks sent_first_chunk for chunks with real choices
- Treats chunks with role in delta as non-empty
- Guards choices[0] access in __next__/__anext__ and stream_chunk_builder
Fixes#24221
Allows wrapping multiple inputs in a nested list to produce a single
combined embedding (text + image = 1 vector). Flat lists continue to
produce separate embeddings per input (OpenAI-compatible default).
Examples:
input=["text", "image"] → 2 separate embeddings
input=[["text", "image"]] → 1 combined embedding
input=[["text", "image"], "x"] → 2 embeddings (1 combined + 1 separate)
When multiple inputs were passed to the Gemini embedding endpoint and any
contained multimodal data (images, audio, etc.), LiteLLM incorrectly used
the `embedContent` endpoint which combines all inputs into a single
aggregated embedding. Now uses `batchEmbedContents` with each input as a
separate request, returning N embeddings for N inputs as expected.
Also fixes hardcoded index=0 in batch embedding responses.
