Router fallbacks configured in router_settings were attempted without
re-checking whether the calling key could use the fallback model, so a key
limited to one access group was served by any model listed as a fallback
for something it could call. Auth only validated the requested model and
fallbacks sent in the request body.
Add a fallback_access_check predicate to Router, consulted before every
cross-model-group fallback attempt; rejected targets are skipped and the
primary's own error is raised when none remain. The proxy injects a check
that runs the same key, team and project model access checks the requested
model goes through.
Under otel_v2, a client that propagates W3C trace context in params._meta
(SEP-414) pulled the tools/call span out of the gateway's trace:
resolve_mcp_span_context parented the MCP span to the client's remote
context and demoted the gateway's own transport span to a span link. The
gateway's tracing backend only ever receives the gateway's half of such a
trace, so the span was unreachable from the trace view and the POST
transaction showed a dangling link.
Invert the anchoring: the MCP tool-call and tools/list spans now always
nest under the transport span of the request carrying the message, and the
client's propagated context is recorded as the span link instead, so the
correlation survives while every trace stays renderable. With no transport
at all the span roots its own trace and still carries the link, keeping a
single shape for the event. Both returned contexts are built on an
explicitly empty base so ambient session state can never leak in, and the
span inherits the transport's sampling decision like every other
request-level span.
* fix(key_management): allow /key/update to keep or shrink MCP server grants the key already holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(key_management): reuse key row's included object_permission instead of a second lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): skip guaranteed-miss team lookup for the litellm-dashboard sentinel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: assert builder result instead of swallowing exceptions; drop redundant comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): opt-in budget rollover carrying overage into the next window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): zero under-cap rows before decrementing over-cap rows in cascade resets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: suppress misleading register_model unresolved-cost warnings for entries without custom pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: do not warn about zero cache costs for tiered pricing entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep upstream OAuth Authorization when jwt signer hook injects one on tools/call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): only treat server credential as occupying Authorization when it maps to that header
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Extract the Soniox SRT/VTT cue grouping and rendering into a shared
litellm_core_utils/audio_utils/subtitle_utils module, have Gemini
transcription request word timestamps whenever response_format is srt or
vtt, and let the http handler rewrite the response text into the
synthesized subtitle document (dropping the internally requested words
array) for any provider config that opts in via
supports_subtitle_synthesis
* feat(health): opt-in model-group allowlist for background health checks and health-check routing
* fix(health): merge shared health states per writer scope instead of replacing
* refactor(health): drop restating comment and parameterize test scope annotations
* chore: remove stray generated prisma migration file
* fix(health): merge health states against the Redis snapshot, not the pod-local copy
* fix(health): fall back to the pod-local snapshot when the Redis read returns nothing
Gemini Live sends no usageMetadata and no turnComplete for
gemini-3.5-transcribe-live sessions, so realtime spend logged as 0.0.
Attach estimated usage to the input_audio_transcription.completed event
using Google's published billing estimate (25 audio tokens/sec of input,
175 text tokens/min of output) derived from the streamed pcm16 audio
duration, gated to audio_transcription-mode models so conversational
Live models keep billing through usageMetadata. Also capture that usage
in the provider_config backend path so realtime cost calculation sees it.
* fix(ui_sso): resolve highest privilege Entra app role, not first in claim
A user assigned more than one Entra app role — commonly by belonging to
several assigned groups — arrives at the Microsoft SSO callback with every
role in the id_token `roles` claim. LiteLLM stores a single role per user,
and get_microsoft_callback_response collapsed the list by taking the first
value that resolved to a LitellmUserRoles and breaking.
Entra does not guarantee the ordering of the `roles` claim, so which role
won was effectively arbitrary: a user in one group mapped to internal_user
and another mapped to proxy_admin_viewer could be silently demoted to
internal_user, and proxy_admin could lose to either.
The generic/Okta path already resolves this correctly via
determine_role_from_groups, which walks a documented privilege hierarchy.
Hoist that hierarchy into LITELLM_USER_ROLE_HIERARCHY and reuse it, so
app-role logins and group-mapping logins agree.
Extract the selection into MicrosoftSSOHandler.get_user_role_from_app_roles
so it is directly testable — the existing tests re-implemented the loop
inline, which is why the ordering bug was not caught.
Behaviour is unchanged for single-role claims, unrecognised values, and
empty claims. Roles the hierarchy does not rank (org_admin, team, customer)
are resolved deterministically rather than by claim order.
* refactor(ui_sso): trim role selection prose and use immutable annotations
Addresses review feedback on the app role selection helper.
Drop the explanatory comments and the Args/Returns docstring boilerplate that
restated the control flow, keeping only the part a reader cannot infer from the
code: that Entra does not guarantee claim ordering, and how unranked roles
resolve.
Type the parameter as Sequence[str] rather than list[str] and build the resolved
set as a frozenset, so the helper stops adding an LIT001 mutable-collection
annotation. Make LITELLM_USER_ROLE_HIERARCHY a tuple for the same reason.
No behaviour change: the ordering regression tests still fail against the
previous first-match-wins logic and pass here.
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
Adds pricing (0.15/0.50 per 1M tokens, 0.03 cached read), the 1M context window, and capability flags (tools, parallel tools, tool choice, response schema, reasoning, vision) for Together AI's zai-org/GLM-5.3-Flash, mirrored into the backup cost map, with exact-value regression tests.
OpenAI's chat completions API rejects tool_reference content parts in
role tool messages, so a mixed text plus reference tool result carried
through the Anthropic adapter turned a previously working request into
a 400 on chat-routed OpenAI and Azure deployments. Strip the reference
parts there, keeping a reference-only result as an empty-text tool
message so the preceding tool_call stays answered, mirroring the
Responses bridge skip.
* feat(newrelic): per-team cost and usage metrics via team callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(newrelic): retry transient 429/408 metric posts instead of dropping
* fix(newrelic): drop only records queued when the drain began, not mid-drain arrivals
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
