Extract the Soniox SRT/VTT cue grouping and rendering into a shared
litellm_core_utils/audio_utils/subtitle_utils module, have Gemini
transcription request word timestamps whenever response_format is srt or
vtt, and let the http handler rewrite the response text into the
synthesized subtitle document (dropping the internally requested words
array) for any provider config that opts in via
supports_subtitle_synthesis
Gemini Live sends no usageMetadata and no turnComplete for
gemini-3.5-transcribe-live sessions, so realtime spend logged as 0.0.
Attach estimated usage to the input_audio_transcription.completed event
using Google's published billing estimate (25 audio tokens/sec of input,
175 text tokens/min of output) derived from the streamed pcm16 audio
duration, gated to audio_transcription-mode models so conversational
Live models keep billing through usageMetadata. Also capture that usage
in the provider_config backend path so realtime cost calculation sees it.
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
OpenAI's chat completions API rejects tool_reference content parts in
role tool messages, so a mixed text plus reference tool result carried
through the Anthropic adapter turned a previously working request into
a 400 on chat-routed OpenAI and Azure deployments. Strip the reference
parts there, keeping a reference-only result as an empty-text tool
message so the preceding tool_call stays answered, mirroring the
Responses bridge skip.
For non-Anthropic models served over /v1/messages, the outer wrapper recomputes
cost over the adapter-translated Anthropic response dict. That dict dropped every
web search usage signal, so the recompute overwrote the correct cost breakdown
with a token-only one: x-litellm-response-cost-tool-usage read 0.0 and
x-litellm-response-cost-original excluded the search cost, while the total kept it.
The adapter now maps web search request counts (from Usage.server_tool_use or
Gemini's prompt_tokens_details) into usage.server_tool_use.web_search_requests,
matching the Anthropic API shape, and the Gemini web search cost calculator falls
back to server_tool_use when prompt_tokens_details carries no count. The shared
get_web_search_requests helper is now public since five modules consume it.
Resolves LIT-6288
get_fireworks_session_id fell back to litellm_trace_id when no session id was
given. That id is generated per request (uuid4 when absent), so x-session-affinity
carried a different value every time and Fireworks prompt caching never hit;
cached_tokens stayed 0 across identical prompts.
The None path the original change described was effectively unreachable because
of it. Drop the fallback so affinity comes only from an id the caller actually
supplied: litellm_session_id, session_id, or metadata.session_id.
Callers who were relying on a trace id for affinity can pass litellm_session_id
instead, which is stable across the requests they want grouped.
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Gemini API Maps-grounded prompts were billed as web search and Vertex AI Maps-grounded prompts were not billed at all. Classify grounding metadata per candidate into web search vs Maps requests, carry a distinct google_maps_grounding_requests usage counter through non-streaming and streaming paths, and price it via the new google_maps_grounding_cost_per_query cost map key with per-query and per-prompt defaults keyed off web_search_billing_unit. Fixes#35906
Live probes against every gemini_native_audio model on both providers show
setup accepts a valid prebuilt voice and 1007s only unknown voice names, so
the strip predicate rested on a false premise and silently discarded the
client's voice on AI Studio native-audio sessions
