* feat(router): random-pick multi-model complexity tiers
Tier pools already make sense without adaptive; stop pinning lists to
index 0 and shuffle within the classified tier instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): format complexity router config
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): use PEP 585 types for tier pools
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Selecting the LLM classifier without picking a model only surfaced a
toast on submit; the classifier model select now gets the same red
outline and helper text as the tier and embedding selects once a submit
attempt has failed.
Clicking Add Auto Router with the name empty returned early with only a
toast, so blank tier selects never got their inline error state. The
empty-name branch now sets showValidationErrors and triggers antd
validation on the name field, so every unfilled mandatory field is
flagged at once. Adds a regression test for the tab component.
The Add Auto Router complexity tab let chat models fill the embedding-model
slot (and vice versa) since neither dropdown filtered on ModelGroup.mode, and
submit only required at least one of the four tiers instead of all four. Adds
getMissingTiersError alongside the existing getSemanticConfigError, and
highlights unfilled tier/embedding selects inline once a submit attempt fails.
* refactor(ui): convert endpoint usage charts to shadcn/recharts
Adds a LineChart wrapper to the shared charts kit, mirroring the
BarChart/AreaChart composition with connectNulls and curveType props,
and converts EndpointUsageBarChart and EndpointUsageLineChart from
tremor to the shared wrappers. Both endpoint chart tests now assert on
real recharts SVG output instead of tremor mocks.
* refactor(ui): drop unused endpointData prop from EndpointUsageLineChart
* fix(ui): point endpoint chart test type imports at the UsagePage types alias after colocation move
* refactor(ui): colocate the usage view, keeping the shared usage components
Split for the usage (UsagePage) segment. Most of the folder is the usage page's
own view, but four pieces are reused elsewhere and stay in @/components/UsagePage:
TopKeyView (old-usage), KeyModelUsageView and value_formatters (activity_metrics),
and the shared types (activity_metrics, chartUtils). The other 21 files move into
usage/_components, preserving the folder structure.
The external consumers import only the retained files, so they are untouched. The
moved files' imports of the retained files become @/components/UsagePage paths,
other escaping relative imports are absolutized, and lint suppressions are re-keyed
for moved files only. No behavior change.
* refactor(ui): colocate the mcp-servers view, keeping the shared mcp_tools surface
* feat(ui): adopt openapi-react-query and convert useCustomers to $api
Add openapi-react-query and expose $api = createQueryClient(fetchClient)
alongside fetchClient. Rewrite useCustomers as
$api.useQuery("get", "/customer/list", {}, { enabled, select }), which
derives the query key from method + path (dropping the hand-written
createQueryKeys entry and the manual key) and forwards the request signal
for cancellation. The response type still flows from schema.d.ts as
CustomerResponse[]. Tests assert the path, the admin/token enabled gate,
and the empty-body select fallback.
* test(ui): read the last render's options in useCustomers helper
The lastCallOptions helper was named for the last call but read
mock.calls[0]. Harmless while each test renders once, but it would
silently assert against first-render options if a test ever re-renders.
Read the final call instead.
XecGuard's async_logging_hook wrote a bare dict to
standard_logging_object["guardrail_information"] while the typed
contract is Optional[List[StandardLoggingGuardrailInformation]].
Readers that iterated the field walked dict keys, raised on
info.get, or silently dropped the entry from guardrail usage
tracking and spend-log writes
Construct the typed entry and append it to the existing list or
create a new one, matching the shared helper pattern. Record the
configured guardrail name instead of a hardcoded "xecguard" and
pass the GuardrailEventHooks enum for guardrail_mode
Split for the usage (UsagePage) segment. Most of the folder is the usage page's
own view, but four pieces are reused elsewhere and stay in @/components/UsagePage:
TopKeyView (old-usage), KeyModelUsageView and value_formatters (activity_metrics),
and the shared types (activity_metrics, chartUtils). The other 21 files move into
usage/_components, preserving the folder structure.
The external consumers import only the retained files, so they are untouched. The
moved files' imports of the retained files become @/components/UsagePage paths,
other escaping relative imports are absolutized, and lint suppressions are re-keyed
for moved files only. No behavior change.
* feat(proxy): add expires filter to GET /key/list
Add an opt-in expires query param to GET /key/list so callers can fetch
only expired or only active keys without paginating every page and
filtering client-side. 'expired' matches keys whose expires is in the
past (NULL expires excluded); 'active' matches keys that never expire or
expire in the future. Omitting the param preserves existing behavior for
every caller. An unrecognized value returns HTTP 400 rather than silently
returning all keys.
The filter is pushed to the database via the existing Prisma where
builder so callers avoid pulling the full key table into application
memory.
Resolves LIT-3387
* refactor(proxy): declare VALID_EXPIRES_FILTER_VALUES before its first use
max_tokens=1 makes reasoning models (o1/o3/...) return a 400 "max_tokens
reached" because reasoning tokens count against the cap, so a reachable
reasoning tier showed a false failure in Test Connection. Live-verified: o3
400s with the cap and succeeds without it.
Extract the request shape into a pure buildModelGroupTestRequest and cover it
with a test asserting the chat body carries no max_tokens (or
max_completion_tokens), so this regression is caught in unit tests instead of
only against a live reasoning model.
* feat(guardrails): add pre_mcp_call support to Content Filter
* test(guardrails): cover canonical MCP key gate under pre_mcp_call mode
* fix(guardrails): scan MCP arguments per value and gate mixed-mode scans by call type
* fix(guardrails): cap MCP argument scan depth and register the walker with the recursion detector
* test(guardrails): update LIT-4226 UI settings tests for content filter pre_mcp_call support
* fix(guardrails): use builtin generics in MCP scan annotations to satisfy strict-rule budget
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor(ui): convert entity usage and usage page charts to shadcn/recharts
Swap the tremor BarChart/DonutChart render sites in EntityUsage,
SpendByProvider, TopKeyView, TopModelView, KeyModelUsageView and
UsagePageView to the shared shadcn/recharts wrappers. Convert the two
sole-chart Daily Spend cards and the KeyModelUsageView card to the
shadcn Card primitives.
Close the donut parity gap with strictly additive optional DonutChart
props: showLabel/label render a center total (tremor showed
valueFormatter(sum) by default) and startAngle/endAngle forward to the
Pie so both provider donuts keep tremor's clockwise-from-12 layout.
Defaults preserve the previous wrapper behavior.
DailyData and two site-local row types move from interface to type
alias so they satisfy the wrappers' Record<string, unknown> constraint;
interfaces lack implicit index signatures.
Tests now assert on real recharts output: bar/sector counts, cyan
fills, axis labels, donut center totals, and the TopKeyView bar-click
drill-down into the key info modal. The dead tremor chart mocks in
UsagePageView.test.tsx are removed and lint metrics/suppressions are
regenerated for the dropped tremor imports.
* fix(ui): compute donut center label only when shown and assert Model Usage renders as a card title
Live testing showed the first cut was broken: /health/test_connection merges
{...configParams, ...requestParams}, so passing the public model_group name as
the request model overrode the resolved provider model and every tier failed
with "LLM Provider NOT provided". The frontend only has the public group name,
not the underlying litellm_params, so it cannot build the request that endpoint
needs.
Switch to testing each model group the way production actually routes it: send a
minimal request to /v1/chat/completions (or /v1/embeddings for the embedding
model) by public group name through the shared apiClient. The router resolves
the group, credentials, and provider itself, so a green row means the tier is
genuinely reachable. Verified live: voyage embedding returns 200, a tier with a
bad key returns the real provider auth error.
Also address Greptile feedback: rows now update progressively as each probe
settles instead of all at once, and TIER_ORDER is derived through a
`satisfies Record<keyof ComplexityTiers, null>` guard so adding a tier without
listing it is a compile error.
* fix(ui): show sidebar copy confirmation only on a successful write
The sidebar account menu's copy button switched to the checkmark
synchronously, before the clipboard write settled, so it confirmed a
copy that never happened when navigator.clipboard was undefined on
non-secure origins or when writeText rejected. The handler now guards
navigator.clipboard, awaits the write, and flips to the checkmark only
on success
Also updates the header accent emoji in the same menu
* refactor(ui): extract a shared CopyButton for the sidebar account menu
The copy-icon-to-checkmark pattern was hand-rolled in several places,
including the sidebar account menu whose private copy button held the
false-confirmation bug. Extract a single canonical CopyButton into
components/shared, built on the Button primitive with a guarded and
awaited clipboard write so the checkmark appears only on a real
success, and have SidebarAccountMenu consume it
The success and failure-mode coverage now lives in the shared
component's own test; the sidebar test keeps one case asserting the
email row is wired to it
The consolidated auto-router tab dropped the Test Connection button because
the shared prepareModelAddRequest helper returns an empty array for an auto
router (it has no model_mappings), so the caller crashed destructuring
result[0].litellmParamsObj. That is the crash in #31590 and the open PR
#31794. #31794 only silenced the crash by pointing the test at
auto_router/complexity_router, which is not a provider model, so the
/health/test_connection health check (a real litellm.ahealth_check
completion) would still error.
Bring the button back and make it meaningful: an auto router dispatches to
saved model groups, so Test Connection now probes those directly. It builds
a deduped target list from the configured tiers (tiers sharing a model group
collapse to one probe) plus the embedding model when semantic keyword
matching is on, then runs a live /health/test_connection against each and
shows per-target pass/fail. This never touches prepareModelAddRequest, so the
original destructure crash cannot recur.
Scope is the recommended complexity router only; the to-be-deprecated
semantic router is untouched. No backend changes.
Supersedes #31794. Resolves#31590.
* refactor(ui): convert user agent and per-user usage charts to shadcn/recharts
Swap the tremor BarChart import for the shared shadcn/recharts wrapper in
user_agent_activity.tsx (DAU/WAU/MAU charts) and per_user_usage.tsx (usage
distribution histogram). All chart props are unchanged; the wrapper exposes
the same tremor prop surface with matching defaults.
Extend user_agent_activity.test.tsx and add per_user_usage.test.tsx with
parity assertions on the real recharts SVG output: bar series per category,
stacked x positions, resolved fill colors, axis bucket labels, legend text,
and value formatter output on axis ticks. Remove the dead ResizeObserver
polyfill in user_agent_activity.test.tsx now that the scoped global mock in
tests/setupTests.ts renders charts, which also lowers the no-explicit-any
metric by one.
* test(ui): harden bar x-position parsing against recharts path format
Some tiered Dashscope models price reasoning output above standard output
(output_cost_per_reasoning_token > output_cost_per_token). The reservation charged
all output at the standard rate, so a reasoning-heavy request reserved too little
and concurrent calls could exceed the budget before reconciliation. The reasoning
share is unknown before the request runs, so reserve every output token at the
higher of the two configured rates, for both tiered and flat pricing.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): filter Add-Guardrail mode dropdown per provider
The GET /guardrails/ui/add_guardrail_settings endpoint returned every
GuardrailEventHooks value in one flat supported_modes list, so the Admin
UI rendered pre_mcp_call as a selectable Mode for every guardrail. Saving
Content Filter or Tool Permission with pre_mcp_call then failed with a
400 because those guardrails' server-side supported_event_hooks list
excludes it.
Expose each guardrail's supported hooks as a get_supported_event_hooks
classmethod on CustomGuardrail (mirrors the existing get_config_model
pattern) and have the endpoint iterate guardrail_class_registry to build
a supported_modes_by_provider map. The UI Mode dropdown filters by that
map when the selected provider is known and falls back to the global
list otherwise. __init__ now sources its own supported_event_hooks list
from the classmethod so the two sides can't drift.
Also register BedrockGuardrail, ToolPermissionGuardrail, lakera,
lakera_v2, and presidio in guardrail_class_registry so they participate
in the map (they were previously only in guardrail_initializer_registry
and had no class-registry entry).
Behavior change: guardrails that previously had no supported_event_hooks
declared (aim, javelin, azure/text_moderation, cato_networks,
crowdstrike_aidr, headroom, hiddenlayer, lasso, noma, onyx,
prompt_security, qualifire, repelloai, zscaler_ai_guard, aporia_ai,
lakera_ai, lakera_ai_v2, mcp_jwt_signer, model_armor, presidio) now
validate the configured mode at instantiation. Existing configs where
the mode was silently a no-op will fail at proxy startup with a clear
validation error rather than running as a broken guardrail.
Resolves LIT-4226
* fix(guardrails): add LITELLM_STRICT_GUARDRAIL_MODES escape hatch, preserve current mode in edit form
Address Greptile P1 (startup break) and P2 (edit form UX):
LITELLM_STRICT_GUARDRAIL_MODES defaults to true (raise on unsupported
event_hook, unchanged behavior for the guardrails validated pre-PR).
Setting it to false logs a warning and continues, giving deployments an
opt-out while they fix configs that now surface as errors instead of
silently no-op'ing. Regression test covers both modes.
Edit form now surfaces the currently-saved mode even when it is not in
the filtered per-provider list, so a legacy row (e.g. content_filter
saved with pre_mcp_call before this fix) no longer disappears from the
dropdown; the option renders with a 'not supported by <provider>' note
so the user knows to pick another.
* fix(guardrails): correct audited hook lists, prune stale modes on provider switch, clean form lint
Audited every get_supported_event_hooks classmethod against the hooks
each guardrail's own tests exercise and its handler methods. Five were
too narrow and their tests caught it in CI: rubrik gains pre_call,
presidio gains during_call and pre_mcp_call, prompt_security, onyx and
qualifire gain during_call. The remaining classes match either their
original __init__ declarations or their exercised modes exactly.
Cursor review fixes: the Add form now drops selected modes the new
provider does not support when the user switches providers, so a
pre_mcp_call selection cannot ride along into a provider that rejects
it at save; the edit form handles list-shaped stored modes instead of
treating mode as always a string.
Extracted shared toModeArray and getSupportedModesForProvider helpers
into guardrail_info_helpers so both forms use one implementation, typed
the remaining any usages in both forms, removed nested ternaries, and
committed the ratcheted-down eslint metrics and pruned suppressions
Adds the regression test for the warning-drop branch in the Converse
adaptive-thinking translation, mirroring the chat completions path's
test_raw_adaptive_thinking_dropped_when_max_tokens_too_small.
The rebase onto staging changed _is_adaptive_thinking_model to require
custom_llm_provider (no default), so the one-arg call in the raw adaptive
thinking branch raised TypeError at runtime for any /chat/completions
caller sending thinking={type: adaptive}. Use self._resolved_provider,
matching the reasoning_effort branch just below. Caught by Greptile.
The envelope arm bypasses user_api_key_auth, so it never ran
pre_db_read_auth_checks (request-size and body-safety limits, the IP allowlist,
and the general_settings route allowlist) that the normal MCP admission path
runs before any key lookup. A caller blocked by IP or a disallowed proxy route
could be admitted through an envelope where the same principal on the normal
path is rejected. Run those gates before the envelope crypto, mirroring the
pipeline's pre-DB ordering; a blocked IP or route surfaces its own 403.
Follow-up to #32867 (native /v1/messages) and the /chat/completions
commit earlier on this branch, extending the same adaptive-thinking
translation to the Bedrock Converse path.
Clients like Claude Code send thinking={type: "adaptive"} on every
request. When routed via Bedrock Converse to pre-4.6 models
(claude-haiku-4-5, claude-sonnet-4-5), this was forwarded as-is and
rejected by the model. Mirrors the translation already applied on the
/chat/completions and /v1/messages paths: map to legacy
thinking={type: enabled, budget_tokens}, capped below max_tokens.
Also fixes the missing custom_llm_provider arg in the chat completions
path's call to AnthropicConfig._map_reasoning_effort.
* feat(ui): add redesigned sidebar account menu
Introduce SidebarAccountMenu, a sidebar-only account/logout menu built on
shadcn Popover/Switch/Badge/Separator/Button, and wire it into leftnav in
place of the shared UserDropdown. The panel has a LiteLLM header with the
bouncing moon and a clickable version tag, Tier/Role/Email/User ID rows
with copy actions, the five display toggles, and Logout.
UserDropdown is left untouched so the control-plane / chat navbar keeps
its existing menu. The version tag links to the same release notes page
as the navbar tag, and the bouncing icon reuses the existing header
animation gated by the Hide Bouncing Icon toggle.
* test(ui): point account-menu e2e specs at the migrated sidebar menu
The sidebar account menu moved from an antd Dropdown to a Base UI popover
(SidebarAccountMenu), so the login, logout, proxy-logout-url, and internal
user identity specs were still waiting on antd-era locators
(.ant-dropdown, the popupRender wrapper class, the user-dropdown-panel test
id, and a menuitem-role Logout). Point them at the new panel test id
(sidebar-account-menu-panel) and the button-role Logout instead. The logout
behavior is unchanged since both menus call the same useLogout handler.
The complexity router's info log didn't say what drove a routing decision.
Literal and semantic keyword matches logged an identical "keyword rule fired"
line (no way to tell which mechanism fired), and the scorer's line carried no
consistent marker tying it to the same question.
Emit one greppable line per decision naming the cause: literal_keyword_match,
semantic_keyword_match, or complexity_scorer. The hook already knows which ran
(the config's semantic_keyword_matching flag distinguishes lexical from
semantic; the override-vs-scorer branch distinguishes keyword match from
scorer), so this is label-only: no behavior change, no new types, no added
latency.
Adds regression tests asserting each decision path logs its cause; they fail if
a label is swapped or the cause= marker is dropped.
Alibaba Model Studio (Dashscope) tiered pricing is all-or-nothing: the tier is
selected by a request's total input tokens and every token, input and output, is
billed at that one tier's rate. The reservation path used graduated slicing and,
worse, picked the output tier from the output-token count, so a long-context
request with a large output allowance reserved far less than the provider charges
and could slip past a depleted budget. Select the tier from input tokens and apply
its rates to all input and output tokens.
Reservation also read tiered pricing from only the first deployment in a model
group. A caller could hit an alias whose cheaper deployment was listed first and
exceed the budget once routed to a costlier sibling. Estimate against every
eligible deployment's pricing and reserve the maximum.
Co-authored-by: Cursor <cursoragent@cursor.com>
Clients that pass thinking={"type": "adaptive"} directly (not via the
reasoning_effort alias) on the /chat/completions interface had it forwarded
unmodified to pre-4.6 Anthropic models, which reject the shape. Mirrors the
translation already applied on the native /v1/messages passthrough (#32867):
translate to legacy thinking={type: enabled, budget_tokens}, capped below
max_tokens, dropping thinking when max_tokens can't fit even the minimum
budget. Hoists the shared budget-capping helper onto AnthropicConfig so both
paths use one implementation.
Add an inline comment explaining that the effort-capable non-adaptive branch in
_translate_adaptive_effort_for_non_adaptive_model exists for models like Claude Opus
4.5 that accept output_config.effort but reject adaptive thinking, and why effort-only
requests pass through while adaptive requests with an unsupported effort level fall
through to the legacy translation.
YAML-parsed tier costs can arrive as strings (e.g. "4e-07"), which broke
arithmetic in the graduated tiered-pricing calculation. Coerce per-token
costs to float in both the in-range and remaining-tokens paths.
Move the tiered-cost helper out of the Dashscope module into a
provider-neutral home so the proxy budget reservation no longer depends on
a provider-specific module.
Co-authored-by: Cursor <cursoragent@cursor.com>
