* refactor(ui): map hardcoded Tailwind palette classes onto semantic tokens
The dashboard painted itself with literal palette utilities (text-gray-500,
bg-blue-50, border-red-200) that resolve to one fixed color regardless of
theme, so the shadcn token layer and its .dark block could never take effect.
A codemod (scripts/codemod-color-tokens.mjs) rewrites 3,232 of those across
254 files onto the existing token scale: neutrals become foreground /
muted-foreground / muted / border / card, and red, green, amber and blue
collapse into destructive, success, warning and info, with the pale -50 to
-300 tints expressed as opacity modifiers on the same token. Hover and focus
variants map to accent so they lift rather than recess. 210 now-redundant
dark: variants are dropped since the tokens carry both modes.
The .dark palette is retuned to a neutral gray ramp with the sidebar recessed
below the content canvas, replacing the blue-tinted shadcn default where the
sidebar read as a full-height card floating on a near-black page.
Nothing sets the .dark class yet, so light mode is unchanged and dark mode
stays inert until a theme toggle lands.
* chore(ui): drop the one-shot color-token codemod script
* fix: net prompt-caching savings against the cache-write premium
Prompt-caching savings priced only the cache-read discount and ignored what
the provider charges to create the cache entry. Anthropic bills cache writes
at 1.25x the input rate, so a request that writes a large cache and reads
little from it is a net loss that the dashboard reported as a gain -- or, on
a pure cold write, as a flat zero.
The counterfactual the number answers is "what would this have cost with
caching off", where every token is billed at the input rate. Since
prompt_tokens partitions disjointly into text + reads + writes, that gives
savings = reads * (input - read_rate) - writes * (write_rate - input)
The write term is the premium over the input rate, not the full write cost:
the tokens would have been paid for at the input rate anyway, so only the
markup is attributable to caching.
The premium stays signed rather than clamped. Three models in the pricing map
price writes below input, and clamping would silently drop that saving.
A model with no cache_creation_input_token_cost falls open to the input cost,
yielding a zero premium -- this is why the change is a no-op for the implicit
caching providers (OpenAI, Gemini), which publish no write price, and bites
exactly on Anthropic and Bedrock.
Verified live through the proxy on a mock Anthropic rig across four cases
(cold pure-write, warm pure-read, write-heavy, read-heavy). Reported total
matched the derived net to the cent, including the negatives; the read-only
case is unchanged.
Pre-existing rows are not backfilled, so a range spanning the deploy mixes
gross and net.
* fix: read a zero cache-write price as unpublished, not free
deepseek-chat carries a literal 0.0 cache_creation_input_token_cost. The
fall-open only caught None, so the zero was taken at face value and the
premium became 0 - input_cost -- reporting a fabricated saving of
writes * input_cost on traffic that cached nothing.
No provider gives cache writes away, so a falsy price means the same thing
an absent one does.
* test: pin that the read leg keeps a literal zero price
The two zero prices mean opposite things and the asymmetry was unpinned.
A free cache write is unpublished pricing; a free cache read is real, and
15 models charge for input while serving reads for nothing. Copying the
write leg's falsy fall-open onto the read leg would zero out their savings.
* refactor: resolve caching rates through the established pricing helpers
Addresses Greptile's P1 and P2, and replaces hand-rolled pricing lookup with
the patterns this file and the cost calculator already own:
- Deployment pricing first: rates now resolve through _effective_model_info
(Router.get_deployment_model_info), the same helper the autorouter driver
uses, falling back to _model_info public rates. A deployment with negotiated
cache rates previously priced at the public map -- a 3x error on the repro.
- Individual prices read via _get_cost_per_unit, the cost calculator's
accessor, which also coerces string prices from config.yaml and resolves
service-tier suffixes; the previous raw .get() handled neither.
- Pricing tests no longer monkeypatch litellm.get_model_info; each case now
pins a real pricing-map entry with a fixture-drift assertion, and the
deployment-rate case follows the existing Router-fixture test pattern.
Behaviour on public rates is unchanged: 101 tests pass, including the exact
same live-verified formula.
* fix(cost-optimization): computeCacheLeakage divides net savings by all cached tokens, not reads alone
prompt_caching_savings_spend is net of the cache-write premium since PR #36452.
computeCacheLeakage was still dividing by cache_read_tokens alone, which:
1. Overstates the per-token rate on traffic that writes and reads cache equally:
a 1:1 read:write key shows rate = 0.002, not 0.001, if net savings is /bin/zsh.002
2. Flips the sign on write-heavy traffic: when writes cost more than reads save
(common on Anthropic and Bedrock), the aggregate net can go negative, but
dividing by reads alone would show a positive 'potential savings' for keys
that don't cache yet — recommending they start caching when it's currently
losing money overall
Fix: divide realizedCachingSavings by (cacheReadTokens + cacheCreationTokens),
matching the semantic that a key starting to cache pays those write premiums too.
When the rate is non-positive, price nothing (potentialSavings stays null, renders
as '—'), reusing the existing no-data fallback path. The card can't meaningfully
estimate savings from a losing rate.
Rename discountPerToken → netSavingsPerCachedToken to surface the semantics and
prevent this drift in future.
Update Usage tab and Cache Leakage card tooltips to describe net-of-premium cost.
Add tests for 1:1 read:write traffic and write-heavy negative-net traffic.
* feat(complexity_router): let operators rename the four complexity tiers
Adds an optional tier_labels map to complexity_router_config so a deployment can
put its own vocabulary on the four tiers, e.g. Cheap / Standard / Premium / Deep,
instead of reading SIMPLE / MEDIUM / COMPLEX / REASONING in its dashboard, its
spend logs, and the rubric the LLM classifier reasons with.
Labels are display-only. Every config key stays canonical, so tiers,
keyword_tier_rules[].tier, and tier_boundaries are written exactly as they are
without labels, and partial maps are fine with unlisted tiers keeping their
default name. A validator rejects blank labels, two tiers sharing a label, and a
label that is another tier's canonical name, since any of those would make a log
row or a rubric line ambiguous. That validator runs on the /model/new and
/model/update write path already, so an ambiguous config gets a 400 rather than
being stored for the router to refuse later.
Under the default heuristic scorer the names are cosmetic: the scorer maps a
weighted score to a rung and never reads a tier name, verified by running the
eval corpus with and without a rename and getting identical tier and identical
score on all 29 cases. Under classifier_type: llm the labels are the names in the
rubric and the values the classifier must return, so the response format's enum
is now built from the configured labels and a reply is resolved back to its tier
against labels first, then canonical names, case-insensitively. An unresolvable
reply degrades to the heuristic on the existing fallback path. A test pins the
generated schema for an unrenamed deployment as equal to the shipped
TierClassification schema, so the wire shape can't drift.
Spend logs keep routing_decision.tier canonical so rows from before and after a
rename stay comparable, and gain routing_decision.tier_label on the tiers that
were renamed.
* refactor(complexity_router): drop added comments and the Counter construction
Review feedback: the repository guide bans new comments, so the explanatory
comments and the appended docstring paragraphs this branch added come back out.
One-line docstrings stay in complexity_router.py, matching that file's own
convention.
The duplicate-label check no longer builds a Counter, which the mutable-collection
budget counts, and the error text drops its list() reprs for joined strings. The
labels are stripped in tier_label() now rather than by rewriting the field in the
validator, so the stored config keeps exactly what the operator wrote.
schema.d.ts is regenerated: ComplexityRouterConfig is exposed in the OpenAPI spec,
so tier_labels surfaces there.
* fix(ui): carry tier_labels through the auto-router preset prefill
buildPresetPrefill maps every payload key onto form state, but the tier_labels
key added by this branch had no line, so a preset shipping labels would apply
its tiers and silently drop its names.
* fix(ui): bundle provider logos as static imports and unify fallback in Logo component
providerLogoMap values are now content-hashed bundle URLs emitted by
static imports instead of /ui/assets/logos/ path strings, so any
deployment that serves the app JS also serves the logos: dev server,
proxy /ui mount, server_root_path sub-paths, and the split-chart nginx
image where the old route 404d in production. A missing file is now a
build error instead of a silent runtime 404.
resolveLogoSrc passes /_next/ URLs through untouched so bundled values
never get double-prefixed with the server root path. The new Logo
molecule owns resolution and the letter-avatar fallback and warns with
the failing URL on load error; ProviderLogo delegates to it. The three
bare img sites in the agents wizard render through Logo, fixing their
broken-image bug.
Dashscope now uses qwen.png, RunwayML the on-disk runway.png, and the
GradientAI entry is removed (no plausible asset exists). soniox.svg and
ai21.svg drop a single mismatched intrinsic dimension attribute that
Turbopack's import-time image parser rejects. Dead logoSrc lookup in
AddModelForm deleted. Vitest resolves image imports to Next's
StaticImageData shape via a config plugin so tests exercise the same
/_next/ URLs as production.
* fix(ui): retry logo load when src changes after an error
Track which src errored instead of a boolean so a Logo instance whose
source changes in place (agents modal title) attempts the new URL
rather than staying on the letter-avatar until remount.
* refactor(ui): migrate inline provider logo lookups to the shared Logo component
Patterns B, C, and E from the logo consolidation: every inline
providerLogoMap lookup feeding a bare img with a hand-rolled DOM
fallback now renders through Logo (credential modal, vector store
create/info views, cost tracking margin and discount forms and tables).
getProviderDisplayInfo, handleImageError, and ProviderDisplayInfo are
deleted; getProviderLogoAndName is a strict superset of the exact-match
helper. The vector store logo map no longer duplicates provider logo
paths: shared entries reference providerLogoMap and the three
vector-store-only logos become static imports. The map itself stays
because milvus and s3_vectors have no Providers enum equivalent.
Sites that rendered nothing for an unmapped provider now render the
letter avatar. Representative tests per pattern assert the rendered img
src against providerLogoMap so a wrong provider-to-enum mapping fails,
plus letter-avatar fallbacks for unmapped providers.
* fix(ui): resolve vector store slugs through the vector store logo map
The vector store info provider badge fed backend slugs like pg_vector,
milvus, and s3_vectors to getProviderLogoAndName, which only knows LLM
providers, so those stores showed a letter avatar and a raw slug. The
pre-existing inline lookup had the same wrong-domain bug via
provider_map. Reinstate getVectorStoreProviderLogoAndName resolving
through vectorStoreProviderMap first with a fallback to the LLM
resolver, so vector-store-only providers get their own logo and display
name for the first time.
* feat(router): add LLM-based classifier option to complexity router
Adds classifier_type: "heuristic" | "llm" to complexity_router_config.
When set to "llm", the router calls a configured model (e.g. a small
model like haiku) via structured output to pick the complexity tier,
falling back to the existing regex/keyword scorer on any error, empty
response, or unparseable output.
* feat(ui): add classifier_type option to complexity router UI, fix edit flow
Adds an "Advanced: Classification Method" section to ComplexityRouterConfig
with a heuristic/LLM toggle, revealing a classifier model picker and timeout
when LLM is selected.
Also fixes the auto router edit modal, which never rendered the complexity
router UI at all (it only handled the semantic router), and the "Edit Auto
Router" button visibility check, which was gated on auto_router_config and
never matched complexity router deployments.
* fix(router): attribute classifier calls to caller, raise default timeout
Forwards the original request's litellm_metadata into the classifier's
acompletion call. Without it, the proxy's cost-tracking gate sees no
user_api_key/team_id/user_id and silently drops spend logging and budget
accounting for every classifier call, letting an authenticated user rack
up unaccounted provider spend via repeated requests.
Also raises the default classifier timeout from 400ms to 3000ms (400ms
undershoots real LLM latency and would silently degrade to the heuristic
scorer on most requests) and corrects the module/class docstrings, which
still claimed zero external API calls after the llm classifier path was
added.
* fix(ci): resolve ruff strict-budget and frontend-lint failures
- Use PEP 585 generics (dict/tuple/list) in the new aclassify/_classify_with_llm
signatures instead of typing.Dict/Tuple/List, and suppress BLE001 on the
intentionally broad except in aclassify's fallback path with a reason.
- Fix prettier formatting in ComplexityRouterConfig.tsx.
- Regenerate eslint-metrics.json (was stale after the classifier UI changes).
* fix(ci): regenerate stale eslint-metrics.json
* fix(router): strip parent budget reservation from classifier metadata
The classifier's internal acompletion call previously forwarded the
parent request's full litellm_metadata, including its budget
reservation (user_api_key_budget_reservation / user_api_key_auth).
That reservation belongs to the routed completion the classifier is
deciding on, not to the classifier call itself, so it's now stripped
while key/team attribution fields are still forwarded for spend
logging.
Renames each segment's local components/ folder to _components/ (private to the
route, matching Next's _ route-exclusion). Both folders are imported only by
their own page.tsx via the folder index (verified zero external importers
across src, tests, and e2e_tests), so each is a straight rename plus repointing
that one index import; a folder rename keeps every file at the same depth, so
all internal and relative imports are unaffected.
Grandfathered lint suppressions under the two folders (31 entries: cost-tracking
15, prompts 16) are re-keyed to the new paths with counts unchanged. No behavior
change.
* feat(mcp): support oauth2_token_exchange auth type via REST API and dashboard
OAuth 2.0 Token Exchange (RFC 8693, a.k.a. OBO) for MCP servers could previously only be
configured through config.yaml; the create/update REST API and the dashboard had no way to
express it. This wires token_exchange_endpoint, audience, and subject_token_type end to end.
These three are persisted as dedicated columns on LiteLLM_MCPServerTable, mirroring how token_url
and oauth2_flow are stored, so the edit form prefills them and they are unaffected by the
credentials-blob clearing on auth_type change. build_mcp_server_from_table reads the columns first
and falls back to the credentials blob so servers persisted before the columns existed still load.
client_id and client_secret continue to ride the existing encrypted credentials path.
On the dashboard, "OAuth Token Exchange (OBO)" is a distinct auth-type option with its own field
section. The McpOAuthMode classifier gains a token_exchange arm keyed off auth_type; the previous
catch-all oauth2 mode was renamed from "obo" to "authorization_code" so the two on-behalf-of
mechanisms are no longer conflated. The token-exchange IdP endpoint and audience are scrubbed from
non-admin and virtual-key responses, matching how token_url is treated.
* fix(ui): gate the MCP list-401 re-auth on authorization_code, not token_exchange
The renamed 401 gate keyed off isTokenExchange, but that condition exists for authorization_code:
when a stored per-user credential is present yet the tools/list 401s (the backend's refresh could
not mint a token), the user must re-authorize via the browser flow. token_exchange has no
gateway-side authorize step, so the Authorize gate never applied to it. The prior isObo flag was
undefined (a compile error) and, per this file's convention and its tests, meant authorization_code;
renaming it to isTokenExchange changed the behavior and broke the mcp_tools auth-gate test for an
authorization_code server whose token expired with no usable refresh. Gate on isAuthorizationCode
instead and drop the now-unused isTokenExchange
* fix(mcp): clear flow-scoped endpoint config when a server's auth_type changes
Switching an existing oauth2 server to oauth2_token_exchange left the old flow's
token_url on the row. The OBO resolver treats token_exchange_endpoint or token_url
as the configured exchange endpoint, so the stale value both suppressed the RFC
9728/8414 discovery this PR adds and sent the exchange grant (client credentials
plus the user's subject token) to the previous flow's token endpoint
update_mcp_server now mirrors its existing stale-credentials rule for the
flow-scoped columns (authorization_url, token_url, registration_url, oauth2_flow,
token_exchange_endpoint, audience, subject_token_type): when auth_type changes,
each one is cleared unless the same request explicitly provides it, so a
deliberate override in the switch request still wins. Updates that keep the
auth_type never touch these columns, which keeps legacy OBO rows that use
token_url as their exchange endpoint working
The edit form sends explicit nulls for the previous flow's fields on an auth type
switch; antd preserves unmounted field values by default, so without this the old
token_url would be re-sent verbatim and read as an explicit override. Transitions
are detected against the persisted auth_type, so saves that keep the auth type
send nothing extra
Reported by Cursor Bugbot on the PR
* fix(mcp): lift legacy blob token-exchange settings into their columns on every write
The three token-exchange settings live in dedicated columns but also exist on
MCPCredentials as the pre-column REST shape. Writes now lift incoming blob
values into the columns (an explicit top-level value wins, including an
explicit null) and strip them from the stored blob; the same-auth credentials
merge migrates legacy rows the same way. The read-time column-or-blob fallback
then only ever serves rows current code has never written, so clearing a column
to re-enable RFC 9728/8414 discovery can no longer be silently undone by a
stale blob copy.
Also asserts the auth-switch clearing fires on the external fields_set path
(PUT /v1/mcp/server).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(mcp): single source for the RFC 8693 default subject_token_type
The default was applied at four egress build sites plus two model defaults,
each with its own copy of the literal. All sites now share
DEFAULT_SUBJECT_TOKEN_TYPE from litellm.types.mcp. A DB-level DEFAULT is
deliberately not used: Prisma writes explicit values on insert, so a column
default would rarely apply, and NULL-means-RFC-default keeps existing rows
correct.
Also documents two review decisions in place: the audience column keeps the
RFC 8693 parameter name (RFC 8707 resource indicators are already a separate
concept named resource in the v2 egress types), and the migration's
out-of-order timestamp is safe under prisma migrate deploy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: fix import sort order in outbound_credentials/types.py (I001 strict budget)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): purge legacy blob copies when a token-exchange column is written without credentials
The migrate-on-write in the credentials merge lifts blob values into null
columns, which is correct for legacy rows but could repopulate a column an
admin had cleared in an earlier no-credentials update (that path never touched
the blob, so the stale copy survived to be lifted later). An explicit
token-exchange column write (set or clear) now migrates the row even when the
update carries no credentials: untouched null columns are lifted, every blob
copy is stripped, and unrelated blob keys stay as-is. A cleared column can then
never be resurrected, because no write path leaves a blob copy behind.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(mcp): state the blob-to-column lift contract on the legacy credential keys
The three token-exchange keys on MCPCredentials are the pre-column REST shape
(the only REST shape from 2026-05 until this PR). Document on both the blob
type and the request models that the dedicated columns are authoritative and
that writes lift blob values into them and strip the stored copy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): scrub subject_token_type in the non-admin and virtual-key sanitizers
The other two token-exchange fields were cleared while subject_token_type was
left visible. It is a public RFC 8693 URN with no disclosure value, but the
sanitizers' rule is that these views receive no token-exchange config at all —
cleared for uniformity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(ui): cut projects and access-groups over to path routes
Same recipe as playground (#30185): MIGRATED_PAGES entries route the
sidebar and redirect the legacy ?page= URLs, the switch arms are
deleted, and the e2e fixture grows two entries. Both components were
already zero-prop and self-fetching via React Query hooks, so the
route wrappers are trivial.
* refactor(ui): move Projects and AccessGroups components into their route folders
Both folders were imported only by the legacy switch, so they colocate
wholesale under (dashboard)/{projects,access-groups}/components. Their
React Query hooks stay in the shared (dashboard)/hooks layer. eslint
suppressions are re-keyed to the new paths.
* test(ui): enable enable_projects_ui in e2e global setup
The projects migration smoke clicks the Projects sidebar link, which
only renders when the enterprise-gated enable_projects_ui setting is
on; the seeded e2e database starts with it off, so the locator timed
out in both e2e_ui_testing jobs. CI already launches the proxy with
LITELLM_LICENSE for premium UI coverage, so flip the setting in
globalSetup via the same /update/ui_settings call the admin UI toggle
makes, failing loudly if the PATCH is rejected.
* test(ui): use Playwright request context instead of raw fetch in global setup
The frontend lint bans raw fetch() outside src/lib/http/; the e2e
convention for proxy API calls is Playwright's APIRequestContext, as
in routerSettings.spec.ts.
Replace the non-paginated Key Alias filter with a new PaginatedKeyAliasSelect component that mirrors the existing PaginatedModelSelect pattern. This aligns the UI with the paginated /key/aliases endpoint from PR #22137.
Changes:
- Added useInfiniteKeyAliases hook for paginated key alias fetching
- Created PaginatedKeyAliasSelect component with infinite scroll (80% threshold)
- Updated keyAliasesCall in networking to accept page/size/search params
- Replaced Key Alias filter in Request Logs and Virtual Keys tables to use customComponent
- Removed fetchAllKeyAliases helper and related upfront fetching logic
- Added 22 tests for new component and hook; all existing tests pass (54 tests)
Fixes the issue where the UI was fetching all key aliases at once, causing performance issues with large key sets.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
