Commit graph

4703 commits

Author SHA1 Message Date
tin-berri
71dfab7177
feat(router): record why the auto-router picked a tier and show it in the logs (#35016)
Auto-routed requests were indistinguishable from ordinary ones once logged:
the spend log recorded the requested model group and the resolved deployment,
but nothing about which tier was chosen or what chose it. That information
existed only inside verbose_router_logger f-strings, so answering "why did my
prompt land on the cheap model" required log access and a running proxy.

The complexity, quality, and adaptive pre-routing strategies now return a typed
StandardLoggingRoutingDecision on their PreRoutingHookResponse, and
Router.async_pre_routing_hook records it once for every attempt. Those three
previously side-channelled their own state through three different metadata
keys; the decision now travels on the hook contract itself, so the bucket is
resolved in one place, through get_or_create_metadata_bucket, which already
owns the question of which dict holds proxy-internal metadata and replaces a
non-dict value instead of skipping the write. Recording happens on every
attempt rather than only on a successful route: a fallback from an auto-router
group to a plain group re-enters the hook with the same request kwargs, and a
decision left behind there would attribute the first router's tier to the
deployment that actually served the retry. The log details drawer renders the
result as a Routing card between Request Details and Metrics; the card is
absent on rows that carry no decision, so ordinary and pre-upgrade rows are
unchanged.

Three defects surfaced while making the recorded cause truthful, each of which
would have persisted a wrong answer. The complexity router hardcoded
cause=complexity_scorer even when the LLM classifier decided, and its silent
fallback to the heuristic on classifier failure meant a row could claim an LLM
verdict the LLM never gave; the cause now reports the path that actually ran.
The keyword that triggered a tier rule was discarded before logging, as was
the escalation keyword. The 2-reasoning-marker override returned REASONING with
a score far below the REASONING boundary and no marker saying so, which reads
as a scoring bug to anyone comparing the two; it now emits a reasoning-override
signal, and the card labels those rows as an override instead of claiming the
score met a boundary. The LLM path no longer reports a synthetic score of 1.0,
and heuristic decisions carry a snapshot of the tier boundaries that mapped the
score, so a historical row stays interpretable after the boundaries change.

Signals name a matched term only when the caller's own message contains it.
Scoring still reads the system prompt, but a term matched solely there is
reported as a count, since signals reach a spend row the caller can read and
naming one would disclose a term from a prompt it cannot see.

routing_decision is stripped from caller-supplied metadata at ingress, so a
client cannot forge its own provenance.
2026-07-30 11:55:10 -07:00
tin-berri
708d010115
Merge pull request #35009 from BerriAI/litellm_routing_nav_autorouter
feat(ui): give auto-routers their own tab on Models + Endpoints
2026-07-30 11:54:18 -07:00
Tin Chi Lo
516953b073 feat(ui): let team admins create auto-routers; authorize models by team, not created_by
The Auto-Routers tab was proxy-admin only, while Add Model on the same page already
admits team admins. The asymmetry was not a policy decision; the auto-router create form
simply never mounted a team selector, so a team admin's submit was unscoped and POST
/model/new rejects an unscoped create from any non-proxy-admin. Mounting the shared
TeamDropdown closes it, and the tab now takes the same audience as its sibling.

Fixing that surfaced a second, larger problem. The dashboard decided who may edit or
delete a deployment with `(userRole === "Admin" || created_by === userID) && db_model`,
but `created_by` is written at creation and never read by any backend auth check. The API
authorizes on team-admin membership of model_info.team_id, so the dashboard was wrong in
both directions: it hid controls from team admins the API accepts, and offered them to
former team admins the API rejects. Verified against a live proxy; a model created by the
proxy admin was PATCHed and DELETEd 200 by a team admin who did not create it, while the
same key got 403 on another team's row and on an unscoped row.

Both questions now have one owner in utils/modelPermissions.ts, deliberately shaped as a
mirror of ModelManagementAuthChecks. Creation returns a tagged union rather than a pair of
booleans, so "may not create" and "may create unscoped" cannot be confused, and the five
places that had each invented their own spelling (the models page, the auto-routers tab
and panel, the auto-router form, and both branches of AddModelForm) call it instead.

Row affordances are now per row rather than per tab, because opening the tab to team
admins puts routers they cannot act on in the same list.

Note for reviewers: collapsing AddModelForm onto the shared owner changes behaviour for
org_admin and Admin Viewer who also admin a team. They previously got the optional team
selector, because all_admin_roles counts them as admins, and could submit an unscoped
create that the API always 403s; they now get the required selector.

Also corrects stale copy left by the auto-router move. The exclude_auto_routers API
description named a dashboard page, which went stale inside a single PR; it now describes
the concept so it cannot drift with the UI again.

The eslint-suppressions prune includes one entry for caching/_components/cache_dashboard.tsx,
which this branch does not touch. Its baseline was already stale; the gate measures the whole
tree, so it could not be left behind.
2026-07-30 00:18:45 -07:00
Tin Chi Lo
fec7f5f246 feat(ui): give auto-routers their own tab on Models + Endpoints
Auto-routers had no home and no list. The create form was mounted in two unrelated places,
inside Models + Endpoints > Add Model and again under Cost Optimization, and neither showed
which auto routers already existed; seeing or editing one meant finding its row in the models
table and drilling in. They now get a dedicated Auto-Routers tab beside All Models, listing
every auto_router/* deployment with create, edit and delete in one place, and both former
entry points are removed.

Creating opens in a shadcn dialog rather than swapping the whole panel out, so the list stays
on screen behind it; the dialog caps its height and scrolls, since the complexity form is long.
The form's own heading goes with it, the dialog header owning that now.

An auto router is a routing construct rather than a deployment, so it also comes off the All
Models table. That table pages server-side off total_count, so a client-side filter would page
over a total including rows it never renders; /v2/model/info therefore gains
exclude_auto_routers (default false, so every existing caller is unaffected) and the filter
runs before the count. /v1/models is untouched, so clients still see auto-routers as models.

Clicking a router opens the same `?model=` drill-in the All Models table uses, so it lands in
ModelInfoView with the full Model Settings, Edit Settings, Edit Auto Router and Delete. An
earlier revision had a bespoke detail page here, which was a partial reimplementation of that
view and showed the router's type twice, once as a Type pill and again as a "Routing strategy"
field saying the same thing. Both are gone.

The auto-router list is keyed under the same `models/list` namespace as the models table
rather than a private one. It reads the same /v2/model/info data, and six call sites across
the app already invalidate ["models","list"] after a write; a separate key meant an edit made
through ModelInfoView left the tab stale until a full reload, and every future writer would
have had to remember a second key.

An auto router has no upstream credential, so its detail header drops Update API Key and
Re-use Credentials, and the destructive action names what it removes rather than saying model.
Test Connection was gated on the editor-aware predicate, which let adaptive and quality routers
through to a check that builds its targets from complexity config they do not have; it now
gates on the deployment predicate.

The edit modal also applies the semantic-matching guard the create form has. It renders those
controls now, and the backend raises on semantic_keyword_matching without an embedding model or
keyword rules, so skipping the shared validator turned an inline message into a raw 400.

Whether a row is writable has two independent axes and the dashboard needs both. STRATEGY:
there are four auto_router/* kinds and only complexity and semantic have a form here, so
adaptive and quality must not be handed an editor that would write auto_router_config onto a
deployment storing its settings elsewhere. ORIGIN: a config.yaml row reports db_model false and
the API refuses it whatever its strategy (PATCH /model/{id}/update 404s, POST /model/delete
400s). Capability is derived per capability rather than as one editable flag, because the
constraints differ: editing needs an editor, deleting removes a row by id and never reads its
config, so a DB-created adaptive router stays deletable. Both axes live in
add_model/auto_router_strategies.ts as a declarative table, one record per strategy, so a fifth
strategy is a table row rather than another branch. That also retired four copies of "is this a
complexity router", one of which was written twice in a row in model_info_view.

Creation narrows to the complexity router, which the UI calls Auto-Router v2; the semantic
option was already badged "to be deprecated" in the picker, so the picker goes away along with
the semantic submit path and its validation helper. Existing semantic routers stay editable.

The edit modal mounted ComplexityRouterConfig without the keyword, escalation and
semantic-matching handlers, so those sections never rendered and could only be set at create
time. It now hydrates them from the stored config, and the five keys become managed only when a
caller supplies that state, so a caller rendering no such control still carries them through. A
component-level round-trip test covers it: a payload-builder test cannot see a hydration bug.

A complexity tier is str | list[str] on the backend, and the UI carried three readers of that
rule, one of which dropped a pinned string. They collapse into one owner,
add_model/complexity_router_tiers.ts.
2026-07-29 19:45:02 -07:00
Tin Chi Lo
7041f5768f fix(mcp): never write discovery results to the row, heal rows a release already stamped, and retry failed discovery with backoff
An interactive oauth2 MCP server created with explicit endpoint URLs and no issuer served
400 "authorization url is not configured" from /authorize about a minute after creation,
with the admin's endpoints intact in the row the whole time (#34985). Discovery wrote its
trust-on-first-use issuer into the same column an admin writes, so the next registry build
read the gateway's own output back as an admin pin, anchored the server to RFC 8414
section 3.3, and discarded the stored endpoint columns; one transient metadata fetch
failure then had nothing to serve, and the reload fast path pinned the broken entry until
an unrelated config write

The core of the fix is a deletion. The gateway no longer writes discovery results anywhere:
the OAuth columns and credentials.scopes carry admin intent alone, and everything discovery
learns lives on the in-memory registry entry, as the existing carry-forward already
assumes. With no gateway write there is no value whose provenance a later build can
misread, so the accidental anchoring cannot be expressed

Deleting the write cannot fix a row a released version already stamped, which still reads
as pinned, so a one-time startup heal clears those stamps. The signal is necessarily a
heuristic: updated_by records only the most recent writer and no audit trail says which
field it touched. A row is therefore healed only on the full signature of the defect, which
is discovery as the last writer plus an issuer plus at least one configured endpoint column
that anchoring is actively discarding; rows with an issuer but no configured endpoints are
left alone, since for them both paths resolve from the same upstream document. Every heal
logs the cleared value so an admin who pinned deliberately can re-pin, and the heal records
its own actor, which makes it idempotent

The reload fast path exempts servers missing an endpoint their flow needs, so failed
discovery retries on the normal reload cadence rather than waiting for a config write. Flow
requirements are read through effective_oauth2_flow, the column-first shape-fallback judge
every flow decision uses, so a legacy null-flow M2M row is classified exactly as the
request path classifies it instead of re-discovering forever; a dcr_bridge server with no
configured client needs its registration endpoint for the relay arm, and an entra_obo
server needs a scope, both of which discovery can supply. Retries back off per server,
doubling from one reload cadence to a fifteen-minute cap, so a permanently unresolvable
server cannot re-run the RFC 9728 to 8414 chain and re-log its warning every cycle forever

Deployments with store_model_in_db unset or false loaded MCP servers exactly once at
startup, leaving that retry with no driver, so they now refresh the registry on the same
reload interval. That job deliberately calls a reload-only entry point rather than the
startup composite, keeping the one-time oauth2_flow backfill and issuer heal out of a
recurring path

Losing the persisted trust-on-first-use issuer also means the issuer column no longer
changes underneath the OAuth token identity, so user tokens are purged only when an admin
actually edits the server

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-29 17:51:22 -07:00
ryan-crabbe-berri
0a6b372126
feat(ui): link organization teams to their team detail pages (#35120)
Some checks failed
CodSpeed Benchmarks / benchmarks (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
* feat(ui): link organization teams to their team detail pages

On the organization info page the teams shown for an org were plain
badges, so walking to a team meant copying its id and finding it by
hand on the teams page

Team badges now link to /teams?team=<team_id>, which opens that team's
detail page directly since #35112. Adds a shared BadgeLink (a badge
rendered as a real anchor with modifier-aware client-side navigation,
so cmd-click opens a new tab) and a teamDetailHref builder for reuse by
future entity links

* fix(ui): format BadgeLink, split its modifier-click chain, and size it up

prettier wanted the Badge props wrapped, and local/no-long-condition-chain
flagged the four-way modifier-click guard; the guard is now two named
conditions. Linked badges also render slightly larger (text-sm, roomier
padding) than plain badges so clickable entries stand out

* feat(ui): size org model badges to match the linked team badges

BadgeLink's href is now optional; without one it renders the same
enlarged plain badge (no pointer, no hover), so the org page's model
badges share the component and the size while staying non-clickable
2026-07-29 17:20:59 -07:00
ryan-crabbe-berri
ba7d8ae17f
feat(ui): deep link organization detail page via ?org= query param (#35117)
* feat(ui): deep link organization detail page via ?org= query param

The organizations page kept the selected organization in React state, so
an org detail page had no URL: it could not be shared, bookmarked, or
opened from another page, and the browser back button dropped you out of
the page instead of closing the detail view

Adds useOrgDetailRouting reading ?org= (same pattern as the api-keys,
models, logs, and teams deep links) and derives the open organization in
OrganizationsPanel from the URL

* fix(ui): reset org edit mode on plain row selection and type test mocks

Greptile P1: with the selected org now URL-derived, browser Back leaves
the detail view without running onClose, so a stale editOrg=true made
the next plain row click open on the Settings tab. Reset the flag on
row selection, matching the teams page

Greptile P2: type the panel test's captured table and detail-view props
from the real components instead of any
2026-07-29 11:56:37 -07:00
ryan-crabbe-berri
bb769702b1
feat(ui): deep link team detail page via ?team= query param (#35112)
The teams page kept the selected team in React state, so a team detail
page had no URL: it could not be shared, bookmarked, or opened from
another page, and the browser back button dropped you out of the page
instead of closing the detail view

Adds useTeamDetailRouting reading ?team= (same pattern as the api-keys,
models, and logs deep links) and derives the open team in Teams.tsx from
the URL. TeamInfo now also derives team-admin rights from the fetched
team data, so team admins arriving via a deep link are not stuck with a
read-only view
2026-07-29 11:35:06 -07:00
yuneng-jiang
74244ddd45
Merge pull request #35041 from BerriAI/litellm_/ui-perf-regression-d7888c
fix(ui): point the navbar and sidebar logos at the dashboard home route
2026-07-29 10:59:53 -07:00
Mateo Wang
2348ccc977
Merge pull request #35107 from BerriAI/litellm_usage_public_model_names
fix(ui): show public model names in usage breakdowns
2026-07-29 10:42:13 -07:00
ryan-crabbe-berri
fdea50daa2
feat(ui): shareable log links via log_id query param on the logs page (#34879)
* feat(ui): shareable log links via log_id query param on the logs page

Clicking a log row now writes ?log_id=<request_id> to the URL, closing the
drawer removes it, and loading the logs page with ?log_id= opens the drawer
for that log. When the log is not in the loaded page, it is fetched by
request_id (the backend already drops the date window for id lookups), so
links keep working for logs of any age. Drawer open state derives from the
URL, mirroring the models page ?model= pattern.

* fix(ui): close the log drawer on browser back after opening via session id

Session opens now write ?session_id= to the URL instead of holding local
state, so back removes both params and the drawer closes (Greptile P1).
Session views become shareable links as a side effect. In-drawer log
switching now replaces the history entry instead of pushing, so back
always closes the drawer in one step rather than replaying every viewed
log.

* fix(proxy): scope /spend/logs/session/ui to the requesting user's visible logs

Non-admin callers now only receive session rows they could already see on
/spend/logs/ui: their own logs plus logs of teams where they hold the
spend-logs permission. Previously any authenticated user could read any
session's log metadata by id, which shareable ?session_id= links made
trivial to trigger. Admin views are unchanged. Also, clicking a log row
now clears a lingering ?session_id= from the URL so the drawer shows the
clicked log instead of a stale session (Greptile P1).
2026-07-29 17:01:09 +00:00
ryan-crabbe-berri
9b7a6b9b90
feat(ui): split failed requests into their own series on the cache dashboard (#34862)
* feat(ui): chart failed requests as their own series on the cache dashboard

Spend logs for failed requests are stored with an empty call_type, so the
Cache Hits vs API Requests chart lumped them into an Unknown bar that read
as normal LLM API traffic. The activity query now also returns a per-group
failed_rows count (status = 'failure') and the dashboard charts it as a
third stacked series, so failures are visibly separate from successful
requests and cache hits. The chart data transform moves into a pure
summarizeCacheActivity helper with unit tests; header stats keep their
existing semantics (cache hit ratio still counts failures in the
denominator).

* refactor(ui): move cache dashboard aggregation server-side with a typed response

The /global/activity/cache_hits endpoint previously returned raw per
(key, call_type, model) spend-log aggregates typed as LiteLLM_SpendLogs
(wrong), and the dashboard reduced them in the browser: grouping by
call_type, relabeling empty call_type as Unknown, and computing the stat
card totals. All of that now happens server-side. The SQL groups per
call_type and splits cache hits vs successful vs failed requests, a new
cache_activity module validates rows into Pydantic models and computes
totals plus the key-alias/model filter options, and the endpoint declares
a real response_model so schema.d.ts types it correctly. The dashboard
consumes it through a typed $api react-query hook (filters ride the
query key and are applied in SQL instead of the browser), the hand-rolled
summarizeCacheActivity transform and the adminGlobalCacheActivity fetch
helper are deleted, and the refresh button now actually refetches.

The endpoint is UI-internal (hidden from the public swagger), so the
response reshape is not a public API break.
2026-07-29 09:48:17 -07:00
ryan-crabbe-berri
40878a1ed5
fix(proxy): allow /key/update to identify the key by key_alias (#34851)
* fix(proxy): allow /key/update to identify the key by key_alias

* fix(ui): drop machine-dependent union-order churn from generated schema.d.ts
2026-07-29 09:48:08 -07:00
ryan-crabbe-berri
fe1670fc06
fix(ui): size object permissions card grid by container width (#35019)
The card variant used viewport breakpoints (md:grid-cols-2 lg:grid-cols-3)
but every card usage sits in a one-third-width grid cell, so on desktop the
narrow card still rendered three internal columns of roughly 100px each and
the text spilled out of its boxes. Switch to Tailwind container queries so
the internal column count follows the card's own width
2026-07-29 09:47:54 -07:00
mateo-berri
802ed1c74f fix(ui): show public model names in usage breakdowns 2026-07-29 09:46:49 -07:00
yuneng-jiang
f4a68a75ff
feat(ui): mark Cost Optimization as beta in the left nav (#34984) 2026-07-28 12:04:45 -07:00
Tin Chi Lo
06a58efb2e feat(mcp): manual authorization-code delivery for headless MCP clients
The aggregate gateway DCR flow ends in a 303 to the client's loopback
redirect_uri. When the MCP client runs on a browserless machine (EC2,
SSH box, container) the user authorizes from a browser on another
machine, so the 303 dereferences the wrong loopback and the code never
reaches the client.

The connect banner now offers manual delivery for loopback clients: the
finish form posts delivery=manual and /authorize/complete renders the
callback URL on a no-store page instead of redirecting. The user pastes
it into the client (Claude Code v2.1.191+ accepts a pasted callback URL)
or fetches it from the client machine's terminal. Manual codes keep the
same sealing, PKCE binding, and single-use guard, with a 5 minute
expiry instead of 2 to survive the copy-paste hop; the used-code marker
TTL derives from the code's own remaining lifetime so the single-use
property holds for the full 5 minutes. The default redirect path is
unchanged.

Resolves LIT-4863
2026-07-28 11:23:37 -07:00
tin-berri
d91fd084f7
Fix cache leakage card layout to keep date picker on right (#34885)
* Fix cache leakage card layout to keep date picker on right and prevent content overlap

Removes flex-wrap and mt-3 to ensure date picker stays pinned to the right side of the card header regardless of zoom level, preventing it from covering card content below

* Remove overflow-hidden from Card to allow dropdowns and overlays to display fully

Fixes date picker dropdown being clipped when opened in cards like the Cache Leakage Card. By removing overflow-hidden from the Card container, popovers, dropdowns, and other overflow content can now display properly without being clipped by the card boundaries.

* Make cache leakage card descriptions consistent with line clamping

Adds line-clamp-2 to ensure both 'by model' and 'by virtual key' cards maintain consistent height. Removes conditional anthropic-specific text that caused height variations between dimensions.
2026-07-28 10:12:00 -07:00
tin-berri
9bb75d67af
Merge pull request #34675 from BerriAI/litellm_tool_spend_rollup
fix(proxy): roll up tool spend daily instead of scanning SpendLogs
2026-07-27 15:31:19 -07:00
ryan-crabbe-berri
0171170fc7
fix(ui): validate default team values in Default User Settings (#34815)
* fix(ui): validate default team values in Default User Settings

The Default User Settings form accepted any free-text team id, and the
proxy persisted it without checking the team exists. New users were then
silently never added to the default team because the consume-time 404
from team_member_add was swallowed at debug level.

Backend: PATCH /update/internal_user_settings now rejects unknown and
duplicate team ids with a 400 naming them, before any persistence or
team budget side effects. Team-add failures in _add_user_to_team now log
at ERROR with user and team ids.

UI: DefaultUserSettings rewritten as a shadcn + react-hook-form + zod
form following the org-settings pattern. The team id free-text input is
replaced with a searchable server-backed team picker, so only existing
teams can be selected; zod blocks empty and duplicate rows. The shared
deriveErrorMessage helper now unwraps the HTTPException detail.error
shape so backend validation errors surface readably in toasts.

* fix(ui): restore read-only view with Edit Settings toggle on default user settings

Parity with the pre-migration form: the tab renders a read-only summary
of the saved defaults, Edit Settings opens the RHF form, Cancel discards
pending edits and returns to the summary, and a successful save returns
to the summary showing the new values. Model sentinel labels in the
summary are derived from ModelSelect's now-exported special values
instead of duplicating the strings.

* refactor(ui): rename MODEL_SELECT_SPECIAL_VALUES_ARRAY to MODEL_SENTINEL_OPTIONS

* fix(ui): move Edit Settings into the card header action slot
2026-07-27 15:05:51 -07:00
yuneng-jiang
2b7e01bb7e
Merge pull request #34691 from BerriAI/litellm_/management-endpoint-standards-b1cd57
refactor(management): move the logs end-user filter onto /management/v1
2026-07-27 11:30:51 -07:00
Yuneng Jiang
c9d067fccc
chore(deps): bump gitpython to 3.1.55 and brace-expansion to 5.0.8
gitpython arrives transitively through mlflow-skinny; re-resolved with uv so the
lock moves that one package only. brace-expansion is a dev-only transitive dep
already pinned in the dashboard 'overrides' block, so the pin is bumped
alongside the lockfile to keep the change durable across reinstalls.

5.0.8 narrows its engines range from '18 || 20 || >=22' to '20 || >=22'; the
dashboard already requires node >=20.9.0 and every CI job pins node 20, so
nothing loses support.
2026-07-27 10:06:50 -07:00
yuneng-jiang
2f2e1e7519
Merge pull request #34689 from BerriAI/litellm_/model-table-dropdown-truncate-2b2a3f
fix(ui): truncate long team names in the models table team dropdown
2026-07-27 09:54:34 -07:00
yuneng-jiang
19348db0a6
Merge pull request #34679 from BerriAI/litellm_/modal-size-restoration-c06977
fix(ui): restore the Add MCP Server dialog size and header spacing
2026-07-27 09:45:52 -07:00
yuneng-jiang
9354849cc8
Merge pull request #34684 from BerriAI/litellm_/model-table-divider-center-b75b6d
fix(ui): center vertical toolbar dividers
2026-07-27 09:45:29 -07:00
Tin Chi Lo
1240c1a76d fix(proxy): close the adversarial-review findings on the tool spend rollup
Three fixes from an adversarial review of this branch, each at the owning
seam rather than the report site.

The flush retried DB_CONNECTION_ERROR_TYPES, which includes ReadTimeout.
A ReadTimeout is the committed-but-unacked case: the review reproduced the
engine abandoning the transaction open on the pooled connection, the retry
stacking its statements into it, and one commit applying both increment
sets while the flush reports success. The retry now covers only
ConnectError, the one failure that proves the statements never reached the
database; post-send failures drop the batch with an error log. The
docstring no longer claims an idempotency the pattern does not have. The
same hazard exists in the untouched daily spend writer and is left for its
own change.

get_tool_calls_from_response read choices[0] only, so a tool invoked in a
later choice of an n>1 response earned spend but never reached the rollup,
the index, or the registry. Choice scope is now an explicit parameter:
accounting passes include_all_choices=True because every choice costs
money; guardrails keep the primary-choice default because they rebuild the
primary assistant message. First multi-choice fixtures in the suite pin
both scopes.

maxBarSize=64 had been added to the shared BarChart unconditionally,
resizing every existing consumer. It is now a prop; only the tool spend
charts opt in. The legend flex-wrap changes stay global because clipping
overflow was a defect, not a preference.
2026-07-26 01:55:47 -07:00
Yuneng Jiang
78e76fff4d
Merge branch 'litellm_internal_staging' into litellm_/management-endpoint-standards-b1cd57 2026-07-25 23:57:32 -07:00
Yuneng Jiang
cb78491482
refactor(management): move the logs end-user filter onto /management/v1
`/customer/aliases` shipped two days ago and has not been in a release, so its
wire contract is still free to change. This lands it on the control-plane
contract before that stops being true, since after a release the path, the param
names and the envelope would all need a permanent legacy adapter

The endpoint becomes `GET /management/v1/spend_logs/end_users`. It is a facet,
the distinct values one column takes over a filtered query on a resource, not an
entity collection; naming it after `customers` implied it listed the end-user
table when it actually reads spend logs, which is a different row set. Serving it
under the parent resource means its filters are the parent's filters, so the
dropdown offers exactly the values the logs table can show without two endpoints
having to keep agreeing on that

Contract changes: `size` becomes `page_size`, `search` becomes `q`, the window
moves from flat `start_date` / `end_date` to `filter[startTime][gte]` / `[lte]`,
and the body becomes `{data, meta, links}`. Unknown query params are now a 400
rather than being silently dropped, because an ignored filter over-returns data.
Errors are RFC 9457 problem documents on this prefix only; every other route
keeps the shape its callers already parse

`links` is what makes the rest deferrable. The dashboard hook follows the
server's `links.next` instead of computing `page + 1`, so moving this to cursor
pagination later changes the links and nothing the client does. That matters
because the inner scan is a sliding window, so offset paging can currently skip
or repeat an end user across pages; the fix is a follow-up, and the hypermedia
means it will not be a breaking one

Cursor mode, `sort`, `include`, ETag / `If-None-Match` and the generic `ListSpec`
framework are all deliberately out of scope here. They are additive or internal,
so none of them needs to beat the release
2026-07-25 23:57:25 -07:00
tin
708a3a19df fix(ui): use a single muted blue ramp for the tool charts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-26 06:53:26 +00:00
Yuneng Jiang
7b655086e5
Merge branch 'litellm_internal_staging' into litellm_/model-table-dropdown-truncate-2b2a3f 2026-07-25 22:56:41 -07:00
Yuneng Jiang
55ff0e10eb
fix(ui): truncate long team names in the models table team dropdown
The Team dropdown popup is pinned to the trigger width via
w-(--anchor-width) and clips its overflow, while Base UI's ItemText
wrapper is flex-1 shrink-0 with min-width: auto, so it sizes itself to
the full nowrap label and simply overflows the popup. Teams without a
team_alias render their 36-char id, so those options were sliced
mid-character with no ellipsis.

Clears min-width: auto off the text wrapper and truncates the label at
the call site. The underlying gap is in the shared Select primitive,
which any long-labelled select in the dashboard will hit; that is left
for a separate change.
2026-07-25 22:56:37 -07:00
tin
5d77c39bbb fix(ui): color spend-by-tool charts with an ordered ramp
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-26 05:23:57 +00:00
tin
9dbf7c363b fix(ui): keep the spend-by-tool legend from overlapping the charts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-26 04:58:37 +00:00
Yuneng Jiang
1912ea200c
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/mcp-tabs-styling-dd340c 2026-07-25 21:55:17 -07:00
Yuneng Jiang
984051f74d
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/model-table-divider-center-b75b6d 2026-07-25 21:53:39 -07:00
Tin Chi Lo
c8b0530c30 fix(proxy): roll up tool spend daily instead of scanning SpendLogs
GET /v1/tool/spend served the Cost Optimization card with two raw queries
over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load;
the totals query's driving scan was all of SpendLogs in the window. Both
per-request tables reach 1M+ rows at customer scale, so the card cost
O(traffic) per view and had to be capped at 30 days.

The index writer also mined proxy_server_request.tools, i.e. tools DECLARED
in the request body, attributing each request's full spend to tools that
never ran; and all non-MCP mining ran against payload fields that are '{}'
unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently
depended on a privacy setting.

Now the spend writer builds a ToolUsageTransaction at request time from
invoked tools only, resolved by the shared get_tool_calls_from_response
normalizer so every response surface (chat completions, Responses API,
Anthropic Messages) is covered; the tool registry's response arm delegates
to the same owner. Transactions queue beside the spend-log queue and the
flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup
(date, tool_name PK) in one transaction, retrying connection errors with
backoff (a failed batch commits nothing, so the retry cannot double-count)
and dropping the batch with an error log on anything else.

The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS
tools by spend via group_by and daily covers only those tools, so the
response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or
tool-name cardinality; the 30-day clamp is gone. total_spend is dropped
from the response; it was never rendered and its deduplicated semantics
are not computable from a rollup. Spend-log retention deliberately does
not touch the rollup, so tool spend history outlives per-request rows.
2026-07-25 21:52:58 -07:00
tin-berri
f7078e2e08
Merge pull request #34265 from BerriAI/litellm_lit4339_upstream_resource
feat(mcp): send RFC 8707 resource indicators on upstream OAuth legs
2026-07-25 18:53:45 -07:00
Yuneng Jiang
086cbb2d85
fix(ui): center vertical toolbar dividers
The shadcn separator primitive ships `data-vertical:self-stretch` so a bare
vertical divider fills its row, but every call site overrides the height with
`h-5`. A definite cross size makes `align-self: stretch` behave as
`flex-start`, so the dividers rendered flush with the top of their flex line
instead of centered: 0px above and 18px below in the dashboard header, 0px
above and 12px below in the models table toolbar

Routes the three vertical dividers through a ToolbarSeparator that pairs the
fixed height with a same-variant `data-vertical:self-center`. Matching the
variant is what matters; tailwind-merge then drops the conflicting class
outright, whereas a plain `self-center` ties on specificity (the variant is
defined with `:where()`) and loses on utility order. The CLI-managed primitive
is left untouched
2026-07-25 18:41:06 -07:00
Yuneng Jiang
a3f81eddcd
fix(ui): stop the custom-server action colliding with the dialog close button
DialogContent's close button is absolutely positioned 16px from the right
edge at 32px wide, so it overlays the rightmost 24px of the p-6 content
box. The justify-between header pins "+ Custom Server" to that same edge
and, being out of flow, the close button reserves nothing. Give the action
a right margin that clears it; keeping the margin on the button rather
than the row leaves the header rule full-bleed
2026-07-25 18:38:06 -07:00
Yuneng Jiang
2ce5900770
style(ui): match MCP Servers tabs to the dashboard's line tab pattern
The MCP Servers page was the only page-level tab bar using the segmented
(pill) TabsList stretched with w-full, which rendered a full-width grey
bar with a lone pill on the left. Every other page-level tab bar
(budgets, vector stores, access groups, organizations, routing groups,
API reference) uses the underlined line variant, so use that here too.
2026-07-25 17:43:53 -07:00
Yuneng Jiang
ecc491756a
fix(ui): restore the wide Add MCP Server dialog
The shadcn migration carried the antd modal's 1000px width over as an
unprefixed max-w-[1000px], which tailwind-merge keeps alongside the
DialogContent base class sm:max-w-md; the responsive variant wins from
640px up, so the dialog rendered at 448px. Prefix the override so the
merge drops the base clamp
2026-07-25 17:42:47 -07:00
Tin Chi Lo
1fa40bd168 feat(cost-optimization): anchor the savings line at a $0 range start
The "Savings over time" chart plotted a single floating dot for short
ranges: the daily rollup keys spend by YYYY-MM-DD, so a one-day range is
one point by construction. Rather than stand up an hourly SpendLogs data
source, read that same daily rollup and make the cumulative line legible.

- Cumulative | Per day toggle. Cumulative accumulates within the range;
  Per day shows the raw stacked bars.
- Cumulative prepends a synthetic $0 point at the range start
  (withStartAnchor) so the line rises from zero to the running total
  instead of floating. An empty series is left untouched so the chart's
  own "No data" state shows.
- Order the daily series oldest-first (the rollup arrives newest-first)
  so the axis reads left to right and the total accumulates forward.
- Header legend, dots on small series, and a "No data" guard on BarChart.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 15:35:51 -07:00
tin-berri
7c1d9fa9ab
Merge pull request #34598 from BerriAI/litellm_cost_savings_tooltip
fix(cost-optimization): swap methodology Collapse for a shadcn HoverCard
2026-07-25 14:24:49 -07:00
devin-ai-integration[bot]
16550edd00
ci: drop docker-based SERVER_ROOT_PATH e2e in favor of a unit test (#34642)
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-25 18:49:56 +00:00
Tin Chi Lo
3c287576b2 fix(cost-optimization): replace savings methodology Collapse with per-card info popovers
Swap the antd Collapse "How savings are calculated" panel for click-triggered
shadcn Popovers on each SummaryCard, so the explanation sits next to the
metric it describes instead of in one combined block.
2026-07-25 10:49:03 -07:00
Yuneng Jiang
3467871007
chore(deps): bump gitpython and postcss to advisory-clear versions
Clears five OSV findings the scanner flags on every PR: four gitpython
advisories fixed in 3.1.54, and one postcss advisory fixed in 8.5.18.

gitpython 3.1.55 and brace-expansion 5.0.8 are left for a follow-up; both
were published less than three days ago and are still inside the
dependency cooldown window.
2026-07-25 10:29:06 -07:00
ryan-crabbe-berri
57894b5b5e
revert(ui): return Models + Endpoints tabs to in-memory, keep the ?model drill-in (#34629)
Per-tab path routing for Models made each tab a separate route parsed out
of the pathname, which is fragile under a static export mounted at a
runtime-variable server-root prefix. Revert the tabs to in-memory state:
a single /models-and-endpoints route renders an antd Tabs whose active
tab is React state, and each tab body moves from its own page.tsx into a
non-routed panel component under panels/. Role-gating (which tabs show),
the refresh control and the header are unchanged.

The ?model= / ?team= query drill-in stays: it is query-param based (read
via useSearchParams, written via history.pushState), so it is unaffected
by the server-root prefix and remains shareable. The shared tab-routing
helpers (createTabRoutes / useTabRouting) are untouched; the other four
pages still use them.

Removes the per-tab route dirs, layout.tsx and tabRoutes.ts (+ their
path-routing tests) and replaces the layout's coverage with a page test
for in-memory tab switching, the drill-in overlays and role-gating.
2026-07-25 09:53:13 -07:00
ryan-crabbe-berri
21de59b32e
feat(ui): deep-link virtual key detail view via ?key= query param (#34591)
* feat(ui): deep-link virtual key detail view via ?key= query param

Clicking a key on the Virtual Keys page now sets ?key=<token> with
history.pushState, mirroring the models page's ?model= routing, so the
detail view survives reloads and can be shared as a URL. The key is
resolved from the loaded page when present and fetched via /key/info
otherwise. Extracts the shared navigateWithParams helper out of the
models detailNavigation hook

* test(ui): use a realistic hashed token in the virtual keys fixture
2026-07-25 09:44:52 -07:00
yuneng-jiang
a66bac3adf
Merge pull request #34579 from BerriAI/litellm_/litellm-logs-ui-lag-0ca4b8
fix(logs): scope and bound the End User filter on the logs page
2026-07-25 09:07:23 -07:00
tin-berri
b9b27c2beb
Merge pull request #34582 from BerriAI/litellm_toolspend_30d_bound
fix(proxy): cap /v1/tool/spend window at 30 days and bound every SpendLogs read
2026-07-24 20:05:42 -07:00