CodeQL flagged two module-level cyclic imports introduced by defining
CachedTokensDetails in litellm.types.llms.openai and importing it from
litellm.types.utils and litellm.cost_calculator. The class now lives in
litellm.types.llms.base, which imports nothing from litellm, and every
user imports it from there.
Also pins that combining realtime usages where only one response.done
carries cached_tokens_details keeps the earlier modality split in both
orders, and commits the regenerated dashboard API types.
A deployment with base_model set was priced from the region's own row once
completion_cost forwarded the response region into cost_per_token, which now
strips the provider prefix and finds bedrock/<region>/<base_model>. Explicit
pricing (base_model or custom pricing) suppresses the region for cost_per_token
the same way _select_model_name_for_cost_calc already does
The estimate looked the reported per-token rates up a second time, with the
provider this endpoint resolved rather than the one completion_cost infers.
The provider decides whether a token tier threshold is inclusive, so an
unrouted xai model sitting exactly on 200k billed at the tier rate and
reported the base rate, half of it.
completion_cost now hands back the rates its own lines were billed at, and
the endpoint reports those.
Claude-Session: https://claude.ai/code/session_01RLKy5DMi3XCBUJ37WzfNi1
Extract one immutable helper for reading the deployment's model_info off the
logging object, stop rebinding the model_info parameter inside ocr_cost, drop
the explanatory comment blocks, and move the OCR custom pricing regression
tests into tests/test_litellm/test_cost_calculator.py
POST /cost/estimate now accepts cache_read_input_tokens,
cache_creation_input_tokens and reasoning_tokens, bills them at the
model's cache and reasoning rates, and reports each share per request,
per day and per month next to the rates it used.
Custom-priced deployments also get cache and reasoning lines in the cost
breakdown now, so the estimate and the spend logs reconcile with their
totals instead of showing zero for those tokens.
Requested by a customer (Pylon #7365).
Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
Google prices Lyria per generated clip, so every Vertex Lyria entry in the
price map now carries a single output_cost_per_image and both the speech
and the passthrough cost paths read that one field. The old
output_cost_per_second and audio_seconds_per_prediction pair assumed a
30 second clip, which does not match the 32.768 second WAV Vertex returns,
and no other model in the map priced audio that way
Drops max_audio_length_hours and max_audio_per_prompt from the price map,
its schema, the generator, and ModelInfo, since nothing reads them, and
drops the audio_mime_type hidden param for the same reason: the response
already carries the resolved content type on its own header
Folds the per-model bundled catalog lookups into one cached parse of the
local cost map, validated with a TypeAdapter over a ReadOnly TypedDict
The realtime usage writer passed the provider's output_token_details through as sent, so spend logs and callbacks kept a text_tokens that still contained reasoning_tokens while every other completion_tokens_details producer stores the partitioned share. The writer now applies the same rule the cost calculator uses, moved to litellm/types/utils.py so both read one definition, and the calculator keeps it for usage objects that arrive nested from elsewhere
OpenAI and Azure realtime usage reports output_tokens == text_tokens + audio_tokens
with reasoning_tokens counted inside text_tokens, so generic_cost_per_token billed
the reasoning share twice. When the output token details sum past completion_tokens,
the nested reasoning overlap is now subtracted from text_tokens before pricing;
shapes where text_tokens already excludes reasoning are unchanged.
Adds the OpenAI gpt-6-astra entry to both price files with standard, flex, priority (fast mode), batch, and above-272K long-context rates, and regression tests covering each tier and the batch rates.
The unconditional region read let a base_model or custom pricing
deployment resolve to the regional cost-map key: a bedrock kimi
base_model shifted to regional rates and vertex claude-opus-5 with a
us-east5 key priced 0.0. Region now applies only when the model name
comes from the provider response (provider_response_model or the
response's own model), matching the base branch. Restores the #38069
regression test and adds region-on-provider-model and base-model-free
cases
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
Vertex Gemini 3.x models route through cost_per_character (the cost_router
token-path gate only matches gemini-2), and its token fallbacks dropped
service_tier, so ON_DEMAND_FLEX responses were still billed at the standard
rate. Pass the tier through the call site and all four fallbacks.
The GA vertex model had no cost map entry, and the realtime cost handler
accepted the router's price-less auto-registered deployment entry for the
session.created model at zero-defaulted rates, so sessions billed 0.0 even
when base_model pointed at the priced preview key. Adds the GA entry at its
published rates and makes the handler fall through zero-defaulted candidates
unless their cost map entry explicitly declares pricing.
