Invalid JSON typed into the Default LiteLLM Params textarea was silently
swallowed on blur (console.error only, no on-screen indication), so an admin
could think their edit was saved when it was actually discarded. Now flags
the field with an error state and shows a warning notification, and clears
both once the field is edited again.
default_litellm_params.cache_control_injection_points was only editable as
raw JSON on the Router Settings page, requiring an admin to hand-write
[{"location": "message", "role": "system"}] to enable prompt cache routing -
clunky next to the structured Switch + row editor already used for the same
field on the Add Model page.
Extracted the row editor (location/role/index inputs, add/remove) out of
add_model/cache_control_settings.tsx into a form-agnostic shared component,
CacheControlInjectionPointsEditor, driven by plain value/onChange props
instead of antd Form bindings. cache_control_settings.tsx now delegates to it
(and picks up a real fix along the way: its role/index onChange handlers read
form.getFieldValue("cache_control_points"), a field that was never
registered under that name, so those edits silently never synced into
litellm_extra_params - now reads the correct "cache_control_injection_points"
field via Form.useWatch).
New DefaultLitellmParamsSection renders that same editor for Router Settings
plus a JSON textarea for the remaining default_litellm_params keys
(timeout, max_retries, metadata, ...). Fully controlled through React state
(matching the optional_pre_call_checks pattern) rather than the page's
generic DOM-read save path, and excluded from ReliabilityRetriesSection's
raw-JSON rendering so it isn't shown twice.
Could not visually verify in a live browser this session (a chrome-extension
tooling conflict blocked screenshots/typing); verified via 174 passing
frontend tests covering both components and their Add Model / Router
Settings integrations, plus a clean tsc typecheck.
update_settings(default_litellm_params=...) merged the incoming dict into the
existing one ({**old, **new}), which can only add or overwrite keys, never
remove one. Since the Admin UI reads and re-submits the entire
default_litellm_params object as a single JSON blob, a merge meant clearing a
field (e.g. removing cache_control_injection_points) by editing it out of the
UI's textarea and saving had no effect: the old key survived the merge and
stayed visible from /get/config/callbacks and active on the live router.
Replace wholesale instead, matching how every other dict-shaped router
setting (e.g. model_group_alias) is already handled via the generic setattr
path - callers are expected to submit the complete desired object, which the
UI already does by round-tripping the full current value.
Also drop explanatory comments/docstrings added earlier in this branch that
weren't requested, per this repo's no-unrequested-comments convention.
It's a literal in the OptionalPreCallChecks type union, but
add_optional_pre_call_checks has no handler for it - selecting it from the
Admin UI's new multi-select would save successfully and show as enabled while
the router does nothing with it. Drop it from optional_pre_call_checks'
exposed options until it's actually implemented.
Router.__init__ auto-enables router_budget_limiting whenever a deployment has
max_budget/budget_duration set or provider_budget_config is configured
(RouterBudgetLimiting.should_init_router_budget_limiter), independent of what
optional_pre_call_checks explicitly lists. _remove_optional_pre_call_checks
didn't account for that: a save (via the Admin UI's new multi-select, or a
config-sync payload) that simply omitted "router_budget_limiting" from the
list would unregister the RouterBudgetLimiting callback and null out
router_budget_logger, silently letting deployments keep serving requests
after their configured budget is exhausted.
_remove_optional_pre_call_checks now checks should_init_router_budget_limiter
before actually removing the callback for this one check, and returns the
checks it kept active despite being in removed_checks so
_apply_optional_pre_call_checks_setting can fold them back into the tracked
optional_pre_call_checks list - keeping the UI's displayed state honest about
what's still enforced.
Extend the removal regression tests to exercise enforce_model_rate_limits
(alongside prompt_caching/router_budget_limiting), all three
DeploymentAffinityCheck flags including the loop's skip-non-matching-callback
path (a mixed optional_callbacks list with prompt_caching present), and the
separate EncryptedContentAffinityCheck flag. Also drop the dead
optional_callbacks-is-None guard in _remove_optional_pre_call_checks: it can
never be None by the time removed_checks is non-empty, since anything in
self.optional_pre_call_checks only got there via add_optional_pre_call_checks,
which always initializes the list first.
update_settings(optional_pre_call_checks=...) only ever unioned incoming
checks into self.optional_pre_call_checks - it never removed anything absent
from the incoming list. Unchecking a check in the Admin UI's new multi-select
and clicking Save silently did nothing live: the DB got the smaller list, but
the router kept the old value and its registered callback (e.g.
PromptCachingDeploymentCheck, RouterBudgetLimiting) active until a restart,
diverging from what the UI showed as saved.
_remove_optional_pre_call_checks mirrors add_optional_pre_call_checks for the
removal direction: clears the relevant flag on the shared DeploymentAffinityCheck
/ EncryptedContentAffinityCheck instance for the affinity-based checks, and
unregisters the dedicated callback (via the existing
logging_callback_manager.remove_callbacks_by_type) for prompt_caching,
enforce_model_rate_limits, and router_budget_limiting. optional_pre_call_checks
is now set to exactly the incoming list rather than a strictly-growing union.
router_code_coverage.py detects test coverage via a static AST scan for
literal `.method_name(` calls in test files, not real coverage
instrumentation - it flagged _merge_default_litellm_params_setting and
_apply_optional_pre_call_checks_setting as untested even though they're
exercised through update_settings(default_litellm_params=...) /
(optional_pre_call_checks=...) in test_router.py, matching the existing
_merge_tools_from_deployment / _invalidate_access_groups_cache precedent for
private helpers only called indirectly.
- schema.d.ts was stale after adding default_litellm_params/optional_pre_call_checks
to UpdateRouterConfig; applied the exact diff CI's schema-vs-spec check expects.
- update_settings's two new elif branches pushed its cyclomatic complexity from 14
to 16, crossing ruff-strict.toml's max-complexity=15 budget. Replaced both branches
with a single `var in _CUSTOM_UPDATE_SETTINGS_HANDLERS` dispatch (one branch instead
of two) so adding a custom-handled setting doesn't grow this function's branch count
per field; also switched the two new helper signatures to `X | None` per UP045.
- prettier --write on the two test files flagged by frontend-lint.
_add_router_settings_from_db_config merges config.yaml router_settings with
the DB router_settings row and calls update_settings(**combined) directly,
without going through UpdateRouterConfig's exclude_none filtering. An
explicit `default_litellm_params: null` or `optional_pre_call_checks: null`
in either source therefore reached the new elif branches verbatim:
`{**dict, **None}` and iterating `None` both raise TypeError, crashing
proxy startup / config sync.
default_litellm_params is merged into every completion call's kwargs, so an
operator can put a shared api_key or an Authorization header under
extra_headers there. Router.get_settings() now returns it (needed so the
Admin UI's Router Settings page can display/edit it), but /get/config/callbacks
forwarded router_settings verbatim regardless of caller role - unlike the
callback and alerting env vars on the same response, which already redact for
non-full-admin callers (e.g. PROXY_ADMIN_VIEW_ONLY). That let a read-only
admin read another admin's upstream provider credentials.
Add the same role gate used for callback/alerting env vars, scoped to
default_litellm_params via the existing SensitiveDataMasker so any
key/secret/token/auth-shaped field is masked for non-full-admin callers while
full admins keep seeing the real value.
The Router Settings page rendered optional_pre_call_checks as free-text JSON,
requiring admins to know and correctly type the exact valid check names. Add
a dedicated multi-select populated from the field's known options (already
returned by /router/fields), matching how routing_strategy already gets its
own selector instead of a raw text field.
The value now flows through React state (like routing_strategy/enable_tag_filtering)
instead of the page's DOM-querySelector-based save mechanism, since an antd
Select doesn't produce a plain named <input> for that mechanism to read.
default_litellm_params keeps the raw-JSON editor since it has no fixed set of
keys to offer as options.
Router.update_settings() silently dropped default_litellm_params and
optional_pre_call_checks (not in the allow-list, and optional_pre_call_checks
was never even stored as a readable attribute), so the Admin UI's Router
Settings page could not display or persist either setting - e.g. enabling
cache_control_injection_points or prompt_caching pre-call routing required
editing config.yaml directly.
Router now persists optional_pre_call_checks and returns both fields from
get_settings(). update_settings() merges default_litellm_params instead of
replacing it (a full replace would drop the timeout/max_retries/metadata
defaults Router.__init__ sets), and diffs optional_pre_call_checks against
what's already applied before calling add_optional_pre_call_checks(), since
that method has no dedup guard for prompt_caching/enforce_model_rate_limits
and would otherwise register a duplicate callback on every re-save.
The bridge refresh path decided whether an upstream token-endpoint rejection was invalid_grant by substring-matching the raw response body, so a rejection whose actual error is something else but whose error_description merely contains the string invalid_grant would false-match, map to invalid_grant, and trigger a needless authorization_code re-run
Parse the RFC 6749 section 5.2 error object and compare the error field. A non-JSON body, or an error that is not invalid_grant, now propagates as the upstream error rather than being reinterpreted. The regression test drives an invalid_client rejection whose description contains the string invalid_grant and asserts it is not mapped, mutation-checked against the substring match
The prior fix sent the sealed scope on a refresh, but the re-minted refresh envelope re-seals scope from the upstream response, and RFC 6749 section 5.1 lets an upstream omit scope when it is unchanged. So after one refresh whose response omitted scope, the new envelope sealed scope=None and every subsequent refresh dropped it, letting a stricter upstream narrow the renewed token
When the upstream omits scope on a bridge refresh, seal the scope we requested (which RFC 6749 section 5.1 defines as the granted scope when omitted) into the renewed access and refresh envelopes, so the scope survives the whole refresh chain. The regression test refreshes against an upstream that omits scope, asserts the new refresh envelope still carries it, and refreshes again off that envelope to prove the chain does not lose it, mutation-checked
* refactor(ui): standardize debounce waits behind shared DEBOUNCE_WAIT_MS constant
* build(ui): bump @tanstack/react-pacer from 0.2.0 to 0.22.1
* refactor(ui): migrate straightforward value debounces to react-pacer
* refactor(ui): migrate callback debounce sites to react-pacer with regression tests
* chore(ui): restore trailing newline in eslint-suppressions.json
* test(ui): mock all pacer debounce hooks in VirtualKeysTable test
* fix(ui): update merged debounce tests for OldTeams to Teams rename
* fix(ui): render the sidebar scrollbar with shadcn ScrollArea
The sidebar navigation scrolled through a native overflow-y-auto container, so the browser drew its default scrollbar. It now scrolls through the shadcn ScrollArea primitive so the thumb matches the rest of the dashboard
Switching to ScrollArea surfaced a latent styling gap. The Base UI scroll-area, tabs, and separator primitives rely on data-horizontal and data-vertical Tailwind variants that resolve to [data-orientation="horizontal"] and [data-orientation="vertical"], and those variants ship in shadcn's shared stylesheet. The project never imported it, so the classes matched nothing and the scrollbar collapsed to zero width. This adds shadcn as a devDependency and imports shadcn/tailwind.css, which also repairs the vertical tabs and separator styling. See shadcn-ui/ui#9196 for the upstream tracking issue
* refactor(ui): inline the Base UI data-* variants, drop the shadcn dep
The earlier fix imported shadcn/tailwind.css through the shadcn devDependency, which pulled 219 packages and tied the CSS build to shadcn's package exports (an open Turbopack-breaking bug, shadcn-ui/ui#10931). shadcn's model is that we own the components, so the custom variants those components depend on belong in our own stylesheet rather than a runtime dependency. This inlines the nine data-* custom variants and the no-scrollbar utility that the Base UI primitives reference into globals.css, and removes the shadcn package.
The refresh envelope seals the upstream scope as the scope to re-request (RefreshCredential), but _prepare_bridge_refresh dropped it, unwrapping only the refresh token, and the exchange added scope to the upstream request only from the client's HTTP form. A DCR/MCP client typically omits scope on refresh, so the sealed scope was never sent and a stricter upstream could narrow or drop the renewed token's scope
Thread the sealed scope through _BridgeRefreshReady.upstream_scope and fall back to it when the client sends none; a client-supplied scope still wins, which RFC 6749 section 6 bounds to the original grant. The regression test drives a refresh where the client omits scope and asserts the upstream POST carries the sealed scope, mutation-checked against both the drop and the fallback
* fix(anthropic-messages): send bare-string tool_choice to Responses API, propagate router-alias litellm_params
The Anthropic /v1/messages -> Responses API adapter always wrapped
tool_choice in an object ({"type": "auto"}, {"type": "required"}), but
the Responses API's tool_choice schema for these cases is a bare
string ("auto"/"required"/"none"). Sending the object shape to an
OpenAI-compatible backend (e.g. vLLM) fails Pydantic validation with a
400. The "none" case also fell through to "auto" instead of mapping to
"none".
Separately, litellm_params configured directly on a router-alias
deployment (auto_router/complexity_router, adaptive_router,
quality_router, or semantic auto_router) - e.g.
cache_control_injection_points, drop_params - were silently dropped
for every request through that alias. async_pre_routing_hook swaps
`model` from the alias name to the selected tier/route's model before
the deployment lookup runs, so the outbound call only ever merged in
the tier deployment's own litellm_params, never the alias's. Register
non-routing-config litellm_params from the alias deployment and apply
them to the request whenever a pre-routing hook substitutes the model.
* fix: satisfy ruff-strict-budget UP006 and router coverage checker
Use builtin dict[...] generics instead of typing.Dict for the two new
annotations introduced in the previous commit, since they pushed
UP006 over the codebase ceiling in ruff-strict-budget.json. Add a
direct unit test for _register_pre_routing_alias_overrides so the
text-based router_code_coverage.py checker sees it exercised by name.
* fix(router): replace alias-param denylist with a tight allowlist
_PRE_ROUTING_ALIAS_RESERVED_PARAMS excluded router-init-only keys from
the alias's litellm_params before forwarding the rest as request
kwargs, but GenericLiteLLMParams also holds deployment-management
fields (tpm, rpm, weight, tags, max_budget, budget_duration,
use_in_pass_through, litellm_credential_name, ...) on the same object.
Any of those left off the denylist would get silently forwarded as if
they were request kwargs.
Replace the denylist with a tight allowlist of exactly the two
request-shaping params this feature exists for - drop_params and
cache_control_injection_points - so unrelated management fields never
reach the outbound call regardless of what else GenericLiteLLMParams
grows to hold.
* fix(router): re-register adaptive-alias overrides on set_model_list reload
set_model_list() unconditionally clears pre_routing_alias_overrides on
every call (e.g. /config/reload), but _finalize_adaptive_router_if_configured()
skips rebuilding an AdaptiveRouter whose model_name already exists in
self.adaptive_routers - so _register_pre_routing_alias_overrides() never
ran again for an auto_router/adaptive_router alias after a reload,
silently dropping its drop_params/cache_control_injection_points.
Build the Deployment unconditionally and re-register its overrides even
on the skip-existing-router path; only the (expensive) AdaptiveRouter
construction itself stays skipped.
* style: ruff format after merging litellm_internal_staging
* fix(router): drop the alias-param allowlist, exclude only model
Per review discussion: instead of a router.py-local allowlist of exactly
which litellm_params an alias (auto_router/complexity_router,
adaptive_router, quality_router, semantic auto_router) can forward to
the request it routes, _register_pre_routing_alias_overrides now
forwards everything except `model` (the alias marker itself, e.g.
auto_router/complexity_router, never a real provider model).
Router-init-only fields (complexity_router_config,
complexity_router_default_model, auto_router_config,
auto_router_config_path, auto_router_default_model,
auto_router_embedding_model, adaptive_router_config,
adaptive_router_default_model, quality_router_config,
quality_router_default_model) now flow into request_kwargs unfiltered
too. That's safe because litellm.completion()/acompletion() already
strips anything in litellm.types.utils.all_litellm_params before
building the provider request - added these 10 keys there, alongside
the deployment-management fields (tpm, rpm, weight, ...) already listed.
Verified live: without that addition, complexity_router_config lands in
extra_body and ships raw to the provider; with it, it's stripped.
This moves the "which fields aren't real LLM params" list from a
router.py-local allowlist to the single existing global list every
completion() call already depends on, instead of maintaining two.
* refactor(router): look up alias litellm_params on demand instead of caching them
_register_pre_routing_alias_overrides cached each alias's litellm_params
into self.pre_routing_alias_overrides at deployment-init time, which
required keeping that cache in sync with set_model_list() reloads - the
exact bug the previous adaptive-router-reload fix was patching around
(AdaptiveRouter survives a reload, but the cache didn't always get
refreshed to match).
Delete the cache and the registration method entirely. async_pre_routing_hook
now looks up the alias's own litellm_params directly from self.model_list
via self.model_name_to_deployment_indices at request time, the same
model_list that's already correctly rebuilt on every set_model_list()
call. No second piece of state to invalidate, so the reload staleness
bug class isn't possible anymore, and it's less code than before.
Mutation testing surfaced branches in cost_tracking_settings and common_utils that the suite executed but never asserted on. Pin those behaviors with targeted tests: the returned (model, provider) from _resolve_model_for_cost_lookup for deployments carrying a custom_llm_provider and for deployments missing the litellm_params / model_info keys, plus the exact error-response bodies, the caller-identity lookup arguments, and the member and guard branches in common_utils.
Four fixes to the refresh_token grant for dcr_bridge oauth_delegate, surfaced by an adversarial pass over the exchange path
Route the user-subject re-validation's outage check through the chain-aware classifier, so a transient DB outage (which get_user_object wraps in a bare ValueError) reports as unavailable (a retryable 503) rather than collapsing to no_active_key and an invalid_grant, matching how admission now handles the same wrapper
When the upstream reports its own refresh token as already elapsed (refresh_expires_in non-positive), do not seal it into a full-TTL refresh envelope; return no refresh so the exchange degrades to an access-only response, mirroring how the access grant refuses an already-elapsed access token instead of capping it
When the upstream rejects the sealed refresh token with 400 invalid_grant (revoked or expired at the IdP), return an RFC 6749 invalid_grant response so the OAuth client re-runs authorization_code, rather than surfacing the opaque upstream error it cannot act on
Gate key-subject renewal on the owner's SCIM state, mirroring admission's _reject_if_admitted_owner_scim_deactivated, so an offboarded user cannot keep refreshing a still-active key; the check fails open on a missing owner or a DB blip so a key that outlives its owner record does not get wrongly revoked
Each fix has a mutation-checked regression test
Three review findings on the refresh path, addressed at the root:
_BridgeRefreshReady.upstream_refresh_token was a plain str, the one credential in the envelope/bridge
layer that escaped the SecretStr discipline every other one follows (RefreshCredential.refresh_token,
UpstreamTokenGrant.access_token, EnvelopeKeys.signing_key). A repr or a traceback capturing a local
_BridgeRefreshReady would have logged the raw upstream refresh token. It is now a SecretStr, carried as
the SecretStr open_bridge_refresh_envelope already returns and unwrapped only at the point the exchange
builds the upstream request body.
_prepare_bridge_refresh took a request it never read; on the refresh path identity comes entirely from
the sealed envelope, not the HTTP request, so the parameter was dead and misleadingly implied it read
from the request the way the authorization_code prepare does. Removed, and the caller updated.
_reload_active_user_by_id misclassified a missing user as unresolvable (500). This is the same root
cause as the admission user-reload fix: get_user_object raises a bare Exception for a deleted user
rather than a ProxyException, so its except-Exception arm must fail closed to no_active_key (which the
refresh path maps to invalid_grant) for anything that is not a database-service-unavailable outage,
rather than treating a missing user as an opaque gateway fault. Regression tests cover the missing-user
and DB-outage classifications directly.
The live proof showed a refresh envelope presented at the MCP tool-call edge was rejected, but through
the generic oauth2 arm ("expected a virtual key starting with sk-") rather than the bridge arm, because
the admission routing gate is_bridge_envelope_shaped matched only the access prefix. The rejection was
already fail-closed and never forwarded anything upstream, but the path was imprecise and the unit test
modelled a route the real router did not take.
Match either envelope kind in is_bridge_envelope_shaped so the bridge arm engages for a refresh envelope
too, and have resolve_bridge_envelope return BridgeEnvelopeInvalid for it: a refresh envelope is a valid
gateway credential but only ever presented back to the token endpoint, never usable to authenticate a
tool call. Admission now fails it closed with the bridge arm's own 401 ("Invalid or expired
credential"), live-verified, with the upstream never touched. is_bridge_envelope_shaped has a single
caller (the admission routing gate), so the change is contained.
A dcr_bridge oauth_delegate access envelope is capped at one hour, and until now the mode had no refresh
at all: when the envelope expired the client had to re-run the interactive authorization_code flow. This
adds a second client-held credential, the refresh envelope, so the client renews on a back channel and
only re-authenticates when the refresh envelope expires or the upstream refresh token dies.
The refresh envelope is a distinct llm_refresh_ credential that seals only the upstream refresh token
(never the access token) bound to the same litellm identity and MCP server as the access envelope, under
the same master-key-derived keys, with nothing stored server-side. Both envelopes now carry a signed
kind claim ("access" or "refresh") that open() requires to match, so a refresh envelope can never open as
an access credential even if its wire prefix is swapped (the prefix is not signed; the claim is). A
refresh envelope presented at the MCP tool-call edge is not an access envelope, so admission fails it
closed the same way it already fails any non-access bearer.
At the token endpoint the authorization_code mint now returns a refresh envelope alongside the access
envelope whenever the upstream returned a refresh token, and the refresh_token grant is supported for
bridge servers: the client presents its refresh envelope, the endpoint opens it, re-validates the sealed
litellm key so a revoked key cannot keep refreshing, unwraps the real upstream refresh token, exchanges
it with the upstream IdP, and returns a fresh access envelope. Because the endpoint re-seals a refresh
envelope only when the upstream returns a new refresh token, the design mirrors the upstream's own
rotation policy rather than reinventing it: with a rotating upstream the client rotates and reuse is
detected upstream; with a non-rotating upstream the original refresh envelope stands until its bounded
14-day TTL. Both preconditions and the unwrap run before the exchange, so a rejected refresh never
consumes or rotates an upstream token.
The pure envelope and credential layers stay side-effect free: mint/open share one signing, size, and
kind gate across both envelope kinds, and every failure is a value. Tests cover the refresh round-trip,
the kind-claim and server-id bindings, the revoked-key gate, upstream rotation carried through, the
unwrap sending the real upstream token upstream, and edge rejection of a refresh envelope; the three
security bindings are mutation-checked. Limitation documented in the PR: gateway-enforced refresh
rotation with reuse detection would require server-side state, which this zero-custody mode omits by
design, so the refresh envelope inherits the upstream's rotation posture plus gateway identity binding
and a bounded TTL.
Several dashboard pages wrap their content in a div styled w-full mx-4, so the
element's width is 100% of the scrollable main while mx-4 adds 16px of margin on
each side. That makes the margin-box 100% + 32px wide, which overflows main by
exactly 32px. Because main uses overflow-y-auto its overflow-x computes to auto,
so the overflow surfaces as a horizontal scrollbar along the bottom of the whole
content area under the pagination
The wrapped block is already full width without w-full, so removing that one
token keeps the layout and drops the overflow to 0. This is the same fix already
applied to the Virtual Keys page in #33112, extended to the remaining pages that
share the wrapper: Models + Endpoints, Tag Management, Organizations, Vector
Stores, AI Hub, and Logging & Alerts
* fix(openai/responses): clamp max_output_tokens below API minimum
Claude Code sends a max_tokens=1 warmup probe when running /model, which
the Anthropic Messages -> Responses adapter forwards as max_output_tokens=1.
OpenAI's Responses API rejects values below 16, so the probe failed with a
400. Clamp anything below the minimum up to 16 in map_openai_params so all
Responses API entrypoints (direct, chat->responses, anthropic->responses)
are covered.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* refactor(openai/responses): extract _enforce_min_max_output_tokens helper
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(ui): address Virtual Keys redesign review nits
Restore sorting by budget on the merged Spend / Budget column. The column now
uses a new DataTableMultiSortHeader whose chevron opens a menu offering Spend
and Budget in both directions plus Reset, so the progress-bar cell stays merged
while the sort field becomes an explicit choice. Sorting is server-side, so the
chosen field id (spend or max_budget, both accepted by /key/list) flows straight
through as sort_by
Fill the DataTable to its container width when column resizing is on. The table
width was pinned to the sum of column widths, so hiding columns left an empty
gutter on the right. It now keeps that width as a minimum and stretches to 100%
on underflow while still scrolling on overflow, which also covers the same gap
in TeamVirtualKeysTable since both share the component
Drop the dark background box behind the page-header icon so the Virtual Keys
header reads like the Teams header, and pull the 4-line inline filter lambda in
SearchSelect out into a named matchesQuery helper
Extends the DataTable and VirtualKeysTable tests to cover the new multi-field
sort menu (field id maps to sort_by, active indicator, reset) and the
fill-to-container width
* fix(ui): emphasize the active field in the Spend / Budget sort header
The merged Spend / Budget header always read "Spend / Budget" regardless of
which field drove the sort, so after picking Budget descending there was no way
to tell what was sorted without reopening the menu. The header now builds its
label from the sort fields and emphasizes whichever one is active (bold,
full-strength text) while muting the other, so the sorted column reads at a
glance alongside the direction chevron. Drops the now-redundant title prop since
the label is derived from the fields
* fix(ui): remove w-full so the keys page content stops overflowing by 32px
The virtual keys content wrapper used "w-full mx-4", which sets the width to
100% of the parent and then adds 16px of horizontal margin on each side, so its
margin-box came to 100% + 32px and overflowed the scrollable main region by
exactly 32px. That surfaced as a horizontal scrollbar along the bottom of the
whole content area, under the pagination. A block div is already full-width, so
dropping w-full lets mx-4 inset it correctly with no overflow
* fix(ui): darken the clickable Key cell on hover so it reads as clickable
The Key cell was the click target that opens the key detail, but hovering only
faded the chevron in with no change to the cell itself, so there was no cue that
the area was clickable. Give the cell a subtle muted background and a pointer
cursor on hover. The button spans the full cell (a negative inline margin plus a
matching width offset so the hover fill reaches both cell edges while the title
stays aligned with the other columns)
* fix response not being redacted for custom callbacks with streaming enabled
* reduce code duplication
* add unit test
* fix: resolve lint violations in adopted redaction fix
* fix: scope streaming response redaction to the opted-out custom logger
---------
Co-authored-by: Moritz Müller <moritz.mueller2@tu-dresden.de>
get_user_object catches every DB failure in a broad except and re-raises a bare ValueError (litellm/proxy/auth/auth_checks.py), so a real outage and a missing user look identical and the original error survives only as __context__. The dcr_bridge admission path keyed its 503-vs-401 decision on the exception type, so a transient outage during a user-subject reload surfaced as a 401 rather than a retryable 503, and the regression test injected a raw ConnectionError, a shape get_user_object never produces, so it passed on a fiction
Add PrismaDBExceptionHandler.is_database_service_unavailable_error_in_chain, which walks __cause__/__context__ (bounded and cycle-safe) the PEP 3134 way, and route _raise_503_if_db_unavailable through it. Move the user's object_permission resolution inside the single classified try so an outage there is a 503 too, never an opaque 500. Pin get_user_object's wrapping with a contract test that drives the real function, and drive the reload tests with that same faithful shape so a chain-blind regression fails them
* feat(ui): rebuild the Virtual Keys table on the shared DataTable
Replaces the hand-rolled Tremor table and bespoke toolbar/pagination on the admin
Virtual Keys page with the shared DataTable: server-side sort, paginate, and
filter, a sticky scrolling body, a search plus column-visibility plus filters
toolbar, a right-side filter drawer, and a rows-per-page footer. A page header
with the existing key icon carries the Create New Key action.
Adds reusable, shadcn-default building blocks for the tables migrating onto the
DataTable next: shared IdentityCell, ModelsCell, and SpendBudgetCell in
shared/table_cells, plus a shared PageHeader. The models cell reveals overflow in
a hover tooltip and the spend/budget cell uses the Meter primitive.
All data and domain logic is preserved, including the useKeys query, team and org
alias resolution, the user popover, and the KeyInfoView detail swap. The rich
async Team/Org/Alias filters move into the drawer, and the toolbar search maps to
the key-alias substring search. Status now also reflects key expiry alongside
blocked and SCIM-blocked.
The VirtualKeysTable tests are updated to the new markup and extended with focused
coverage for each new shared cell
* fix(ui): address Virtual Keys redesign review feedback
Fold the status badge into the clickable Key cell and drop the separate Status
column so a key's alias, secret, and status read as one unit. The Key cell is
now the single click target that opens the key detail; the whole-row click is
removed
Migrate the filter drawer off AntD to shadcn. A new Combobox composed from
Popover and Input backs the Team, Organization, and Key Alias filters, keeping
search and the alias infinite-scroll
Show $0.00 for zero spend instead of a hyphen, and extend the shared DataTable
with badge, chips, and meter skeleton shapes so the loading state matches the
loaded cells (status pill, model chips, spend meter) rather than uniform bars
Fix key sorting: the Key column sent its column id "key" as sort_by, which
/key/list rejects with 400. It now sorts by the backend field key_alias
* fix(ui): use the shadcn base combobox and refine the keys filters and skeletons
Replace the hand-rolled filter combobox with the supported shadcn Base UI combobox
(ui/combobox, added via the CLI and reused through a small SearchSelect wrapper).
Its vended input-group and textarea deps are written for React 19 (plain functions
with ref-as-prop); this app is on React 18, where those subcomponents drop the refs
Base UI passes for focus and anchoring, so InputGroupInput, InputGroupButton, and
ComboboxTrigger are adapted to forwardRef. Those ui/ files now diverge from the
registry, and a future shadcn add would overwrite the adaptation until the app moves
to React 19. Adds class-variance-authority, which input-group needs
Give loading skeletons a per-column renderSkeleton escape hatch on the shared
DataTable and mirror the Key cell exactly (alias line, secret, status pill), so
skeleton rows match the real rows instead of being shorter and simpler
Resolve the automated review: the toolbar search and the drawer Key Alias filter
both mapped to the key-alias query, so the search silently overrode the drawer value
while its chip stayed visible. Consolidate to a single alias search in the toolbar
(placeholder now "Search by key alias…") and drop the redundant drawer field. Re-add
coverage for the Created By column's alias-over-email display
Refine the Team and Organization filters: they match on name and id, so the labels
read "Team" and "Organization" rather than "... ID", each option shows the name with
the id on a muted second line instead of "name (id)", and the active-filter chip
shows the friendly name
* chore(ui): drop duplicate class-variance-authority, use the repo cva package in input-group
_reload_admitted_user returned a bare UserAPIKeyAuth(user_id=...), so the shared
get_allowed_mcp_servers found no key/team/object-permission grants and an interactive SSO client could
admit successfully yet see zero tools on a normal (allow_all_keys=False) server. The key path returns
the full key record whose object permission drives that computation; the user path dropped it.
Resolve the user's own MCP object permission and put it on the returned auth, so the same
get_allowed_mcp_servers the key path uses grants the user their litellm-granted servers and access
groups. This reuses get_object_permission (the id-to-grants resolver keys and teams already use) and
does not duplicate any permission logic; get_user_object does not load object_permission, so it is
resolved from the user's object_permission_id the same way the key and team paths do.
Only the user's own object permission is bound. A UserAPIKeyAuth carries a single team_id while a user
may belong to many teams, so team-inherited MCP grants for a user are a follow-up: they need a
many-teams union get_allowed_mcp_servers does not do off one auth object, and faking one here would be
the kind of half-measure that spawns more bugs. Tests cover the user's object permission riding onto the
admitted auth, and the existing admit/SCIM/missing-user/503 cases still hold.
_reload_admitted_user mirrored only part of _reload_admitted_key's error contract: it caught
ProxyException and HTTPException but had no arm for anything else, so a transient DB outage surfaced as
an opaque 500 instead of the retryable 503 the key path guarantees, and a missing user surfaced as a 500
too. The missing-user case is the subtle one: get_user_object raises a bare Exception for a deleted user
(not a ProxyException like get_key_object does for a missing key), so the ProxyException/HTTPException
clause never caught it and the user_object-is-None branch it was supposed to hit is unreachable on the
production path.
Add the same except-Exception arm the key path uses, with the one deliberate difference the differing
get_user_object contract requires: a database-service-unavailable error still raises the retryable 503,
while a missing user or any other non-outage resolution failure fails closed as a 401 rather than
propagating as a 500. The regression tests now drive the real behavior (get_user_object raising) rather
than a None return that never happens in production, and cover both the 503 outage and the 401
missing-user paths.
Completes the oauth_delegate bridge for real DCR clients (Claude Code, Claude
Desktop), which send no litellm key and cannot use the scripted two-header path.
On the short-circuit bridge arm the gateway now captures the SSO-authenticated
litellm user from the browser session at /authorize and seals it into the OAuth
state; at /callback it seals that user plus the upstream code into a gateway
authorization code the client echoes back; at /token it recovers the user,
exchanges the real upstream code, and mints a user-subject envelope. The user
identity captured in the browser thus rides to the back-channel token call with
nothing stored server-side, and admission opens the envelope under that user. The
scripted key_hash path is unchanged (raw upstream code, key from the request);
without a session the browser is sent through login first.
The scripted two-header client mints under a virtual key it presents at the token
endpoint (key_hash), but the interactive DCR client authenticates via SSO at the
bridged authorize, which yields a user, not a key. Make EnvelopeIdentity a
discriminated subject (subject_type key_hash | user_id) with key_hash_identity /
user_identity constructors, and dispatch admission on it: a key_hash reloads the
key, a user_id reloads the user and admits them as themselves (user-level budget
and SCIM enforced via the same centralized gate; no team bound, since a user
belongs to many teams or none). The interactive producer that mints a user_id
envelope lands in the follow-up commit.
Pass-through endpoints configured with auth=false reach the cost-tracking callback with no key/user/team/end-user, so _should_track_cost_callback returned False and the spend-log write was skipped, leaving the request out of request/usage logs. Track pass-through call types even when unauthenticated so the SpendLog row is still written.
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
