The shared SelectContent wrapper defaulted alignItemWithTrigger to true,
which puts Base UI's positioner into item-aligned mode and places the
popup so the active item sits on top of the trigger. In that mode the
side and sideOffset the wrapper passes two lines above are ignored, and
the popup reports data-side="none".
The overlap only becomes visible once the items are tall enough to
matter, which is why the autorouter Template picker shows it clearly:
its options are three-line cards, so the popup covers both the select
box and its own label.
No call site in the dashboard asked for item-aligned mode. 21 of them
across 15 files already passed alignItemWithTrigger={false} by hand to
undo the default, and the remaining 127 inherited the bug. Flipping the
default makes side and sideOffset live, so collision handling works and
a select with no room below now flips above the trigger rather than
covering it. The 21 hand-written opt-outs are deleted as redundant.
* fix(key_management): allow /key/update to keep or shrink MCP server grants the key already holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(key_management): reuse key row's included object_permission instead of a second lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): skip guaranteed-miss team lookup for the litellm-dashboard sentinel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: assert builder result instead of swallowing exceptions; drop redundant comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): opt-in budget rollover carrying overage into the next window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): zero under-cap rows before decrementing over-cap rows in cascade resets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: suppress misleading register_model unresolved-cost warnings for entries without custom pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: do not warn about zero cache costs for tiered pricing entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep upstream OAuth Authorization when jwt signer hook injects one on tools/call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): only treat server credential as occupying Authorization when it maps to that header
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(health): opt-in model-group allowlist for background health checks and health-check routing
* fix(health): merge shared health states per writer scope instead of replacing
* refactor(health): drop restating comment and parameterize test scope annotations
* chore: remove stray generated prisma migration file
* fix(health): merge health states against the Redis snapshot, not the pod-local copy
* fix(health): fall back to the pod-local snapshot when the Redis read returns nothing
Gemini Live sends no usageMetadata and no turnComplete for
gemini-3.5-transcribe-live sessions, so realtime spend logged as 0.0.
Attach estimated usage to the input_audio_transcription.completed event
using Google's published billing estimate (25 audio tokens/sec of input,
175 text tokens/min of output) derived from the streamed pcm16 audio
duration, gated to audio_transcription-mode models so conversational
Live models keep billing through usageMetadata. Also capture that usage
in the provider_config backend path so realtime cost calculation sees it.
* fix(ui_sso): resolve highest privilege Entra app role, not first in claim
A user assigned more than one Entra app role — commonly by belonging to
several assigned groups — arrives at the Microsoft SSO callback with every
role in the id_token `roles` claim. LiteLLM stores a single role per user,
and get_microsoft_callback_response collapsed the list by taking the first
value that resolved to a LitellmUserRoles and breaking.
Entra does not guarantee the ordering of the `roles` claim, so which role
won was effectively arbitrary: a user in one group mapped to internal_user
and another mapped to proxy_admin_viewer could be silently demoted to
internal_user, and proxy_admin could lose to either.
The generic/Okta path already resolves this correctly via
determine_role_from_groups, which walks a documented privilege hierarchy.
Hoist that hierarchy into LITELLM_USER_ROLE_HIERARCHY and reuse it, so
app-role logins and group-mapping logins agree.
Extract the selection into MicrosoftSSOHandler.get_user_role_from_app_roles
so it is directly testable — the existing tests re-implemented the loop
inline, which is why the ordering bug was not caught.
Behaviour is unchanged for single-role claims, unrecognised values, and
empty claims. Roles the hierarchy does not rank (org_admin, team, customer)
are resolved deterministically rather than by claim order.
* refactor(ui_sso): trim role selection prose and use immutable annotations
Addresses review feedback on the app role selection helper.
Drop the explanatory comments and the Args/Returns docstring boilerplate that
restated the control flow, keeping only the part a reader cannot infer from the
code: that Entra does not guarantee claim ordering, and how unranked roles
resolve.
Type the parameter as Sequence[str] rather than list[str] and build the resolved
set as a frozenset, so the helper stops adding an LIT001 mutable-collection
annotation. Make LITELLM_USER_ROLE_HIERARCHY a tuple for the same reason.
No behaviour change: the ordering regression tests still fail against the
previous first-match-wins logic and pass here.
Adds a Gemini audio transcription config that maps /v1/audio/transcriptions
onto the Interactions API (speaker attribution and word timestamps land on
the OpenAI verbose_json shape), registers both models with published pricing,
routes text-only Live sessions to TEXT responseModalities so
gemini-3.5-transcribe-live sessions survive, and makes the token-priced
transcription cost path provider-aware instead of hardcoding OpenAI.
* feat(newrelic): per-team cost and usage metrics via team callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(newrelic): retry transient 429/408 metric posts instead of dropping
* fix(newrelic): drop only records queued when the drain began, not mid-drain arrivals
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`test_gemini_chat_returns_content_and_logs_cost` asks gemini-2.5-flash to
"reply with the single word pong" under `max_tokens=32`, and has been seen
returning no content at all:
completion_tokens=29, reasoning_tokens=29, content=None
gemini-2.5-flash defaults to dynamic thinking, and `max_tokens` maps to
`maxOutputTokens`, which on the 2.5 family counts thinking tokens as well as
visible output. So the model is free to spend the entire budget on thoughts and
emit nothing, which is exactly what the usage above shows.
Raising the limit alone does not fix this. Dynamic thinking on 2.5 Flash is
documented up to 24576 tokens, so no budget small enough to be reasonable for a
one-word smoke test is safe. The fix is to take thinking out of the picture:
`reasoning_effort="none"` maps to `thinkingConfig.thinkingBudget=0` for the 2.5
family, so the whole limit is available to visible output. Verified against this
checkout:
get_optional_params(model="gemini-2.5-flash", custom_llm_provider="gemini",
max_tokens=32)
-> {'max_output_tokens': 32} # no thinkingConfig at all
get_optional_params(model="gemini-2.5-flash", custom_llm_provider="gemini",
max_tokens=64, reasoning_effort="none")
-> {'max_output_tokens': 64,
'thinkingConfig': {'thinkingBudget': 0, 'includeThoughts': False}}
This mirrors what the OpenAI tool tests in this same file already do with
gpt-5.6 for the same failure mode. `max_tokens` goes to 64 for headroom; with
thinking disabled that is ample for a one-word answer.
Neither `covers` claim changes: the call still exercises the gemini chat
translation path and still produces a costed SpendLogs row.
`_cacheable_system_block` embedded the per-run marker in all 300 paragraphs, so
the block's token count moved with the marker's own tokenization. Measured over
40 random markers the size ranged 3611-5408 tokens (median 4509): 15% of runs
landed under the 4096-token minimum cacheable prefix of Haiku 4.5, despite the
docstring claiming the prompt was comfortably above it.
When the system block is under the minimum, no cache entry is written at the
system breakpoint. The entry at the second breakpoint still gets written,
because system + first user turn clears the minimum -- which is why the failures
report a large cache_creation with cache_read stuck at 0
(`cache_creation_input_tokens=5610 cache_read_input_tokens=0`, and 5610 is the
whole prefix, not the user turn's share). `_prime_prompt_cache` rotates the user
turn on every attempt, so that second entry never prefix-matches the next
attempt either. Every attempt re-creates the full prefix, cache_read never rises
above 0, and the loop burns its 60s deadline:
prompt cache never became readable in full within 60.0s
That is the single most frequent flake in the e2e suite, 9 of 38 runs, and it
hits all three provider classes identically because they share this helper.
Move the marker out of the repeated paragraph so it appears once, and size the
block at 1500 paragraphs. The prefix is now 8056-8060 tokens across markers --
spread 4 tokens instead of 1797, and 1.97x the minimum in the worst case. The
same marker-per-repetition pattern in `_first_turn_user_text` is fixed the same
way. Both copies of the helpers stay byte-identical.
The test claiming mgmt.key.generate.happy_path signed in and then only read
/key/list, so nothing proved the session key an admin's sign-in mints is
actually accepted on /key/generate. It now does what an admin filling in
Create New Key does: POST /key/generate under the session key, read the new
key back from /key/info, see it in the dashboard's own /key/list, and drive
real traffic through it to confirm its model scope is enforced.
Adds ManagementClient.generate_key with the same caller_key seam update_key
and key_list already use, so the suite can call the route as the master key
or as a virtual key. Also wraps the over-long models import.
Refusing the dashboard session key on /key/generate turns only this test red;
the master-key generate, the key edit, and regenerate stay green.
Google withdrew gemini-live-2.5-flash-preview-native-audio-09-2025 from the
Vertex Live API. Every session dies at setup:
received 1007 (invalid frame payload data)
gemini-live-2.5-flash-preview-native-audio-09-2025 is not supported in the live api.
The client sees session.created (the proxy synthesizes it on connect) and then
nothing, so both vertex_ai realtime tests time out waiting for session.updated.
Confirmed by probing the Vertex Live endpoint directly with the e2e stack's own
credentials:
gemini-live-2.5-flash-preview-native-audio-09-2025 -> 1007, not supported
gemini-live-2.5-flash-native-audio -> setupComplete
so this swaps to the non-preview sibling, which is the same native-audio class
and is what the cost map already carries for vertex_ai.
Not a litellm regression. The suspicion fell on #38395 because it removed the
native-audio speechConfig strip, but the setup payload this suite sends is
byte-identical either side of that change: the strip only fires when a client
sends a voice, and the e2e SessionConfig has no voice field. Google's rejection
names the model, not a field.
The gemini (Google AI Studio) provider keeps the -09-2025 id, which still works
there; only the Vertex endpoint dropped it.
* feat(complexity_router): heuristic-first classifier chaining
Adds classifier_type 'heuristic_first', which scores locally on every request and
only calls the LLM classifier for traffic the scorer could not place at or below
heuristic_first_max_tier. A request short-circuits when the scorer landed at or
below the threshold and produced at least one signal; everything else escalates.
The signal requirement is load-bearing. A prompt where no dimension fires scores
exactly 0.0, which is under simple_medium, so the score-to-tier mapping calls it
SIMPLE by default rather than by evidence, and that is about half of general
traffic. Gating on the tier alone would route it to the cheapest model without
ever consulting the classifier.
Introduces uses_llm_classifier as the single owner of 'does this router call the
classifier model', replacing the classifier_type == 'llm' comparisons in the
config validator, the prompt prebuild, the health dependency graph, the
routing-test authorizer, and six dashboard sites.
* fix(complexity_router): reuse the heuristic verdict on classifier failure, load the threshold on edit
Three review findings, one push.
The heuristic-first fallback re-scored the prompt after a classifier failure,
which the README already documented as a reuse. The outcome computed before
escalation is now handed to the failure path, so the scorer runs once per request.
The edit modal never hydrated heuristic_first_max_tier, while save rebuilds every
managed key from form state, so opening a heuristic-first router and saving it
dropped a field the proxy requires. The dropdown's display fallback hid it. Both
are fixed, and the hydration is extracted into a pure function so a test can pin
the invariant: every managed key present in a stored config survives an untouched
open-and-save. That test also covers every field added later.
Classifier radio labels lost their em dashes, per the repo writing convention.
The two `surface: ui` cells in the coverage registry, mgmt.key.generate.happy_path
and mgmt.key.update.happy_path, had no covering test. The existing key tests all
call /key/generate and /key/update with the master key, which is not how the
dashboard reaches those routes: an admin signs in, the proxy mints a UI session
key scoped to the litellm-dashboard team, and every subsequent create or edit is
written under that session key.
TestDashboardKeyRoutes covers that path. The first test signs in through
/v2/login, decodes the master-key-signed session JWT the way the dashboard does,
and asserts the minted key carries the admin role and the dashboard team, then
that it can actually read the key inventory the Virtual Keys page renders. The
second edits a key under that session key and asserts both halves of the
contract: /key/info reports the new models and limits with the alias untouched,
and the gateway flips enforcement to match.
ManagementClient grows dashboard_login plus caller-aware key_list and update_key,
so a test can say who is driving a management route instead of always implying
the master key. update_key returns its Result rather than raising, which lets a
caller poll a route that is only transiently refusing; a freshly minted session
key is briefly unauthorized while the auth cache picks up its user row.
litellm-e2e-ui 68 failed the test this PR was meant to stabilise: "fallback
never took effect", streak 4 of a required 5, 60s timeout. Requiring a
consecutive streak of 200s after the fallback is set was wrong. It asserts that
the fallback path succeeds five times running, which is a reliability claim the
test never intended to make, and the path is inherently retry-ish because the
broken primary is attempted first on every call. One intermittent non-200
resets the streak, so a mostly-working fallback never converges.
The two directions are not symmetric:
before the write proving NO replica serves it -> needs every replica
after the write proving the fallback serves it -> one success is the claim
So the control keeps a multi-sample window and the success assertion goes back
to polling for a first sighting, on the wider 60s budget rather than the
original 30s that expired on litellm-e2e-ui 63.
Also drops the two local rebinds Greptile flagged against the repo's
no-reassignment convention: the streak counter is gone with the helper it lived
in, and the cache-round loop is now a lazy generator consumed by next().
