Three fixes from an adversarial review of this branch, each at the owning
seam rather than the report site.
The flush retried DB_CONNECTION_ERROR_TYPES, which includes ReadTimeout.
A ReadTimeout is the committed-but-unacked case: the review reproduced the
engine abandoning the transaction open on the pooled connection, the retry
stacking its statements into it, and one commit applying both increment
sets while the flush reports success. The retry now covers only
ConnectError, the one failure that proves the statements never reached the
database; post-send failures drop the batch with an error log. The
docstring no longer claims an idempotency the pattern does not have. The
same hazard exists in the untouched daily spend writer and is left for its
own change.
get_tool_calls_from_response read choices[0] only, so a tool invoked in a
later choice of an n>1 response earned spend but never reached the rollup,
the index, or the registry. Choice scope is now an explicit parameter:
accounting passes include_all_choices=True because every choice costs
money; guardrails keep the primary-choice default because they rebuild the
primary assistant message. First multi-choice fixtures in the suite pin
both scopes.
maxBarSize=64 had been added to the shared BarChart unconditionally,
resizing every existing consumer. It is now a prop; only the tool spend
charts opt in. The legend flex-wrap changes stay global because clipping
overflow was a defect, not a preference.
`/customer/aliases` shipped two days ago and has not been in a release, so its
wire contract is still free to change. This lands it on the control-plane
contract before that stops being true, since after a release the path, the param
names and the envelope would all need a permanent legacy adapter
The endpoint becomes `GET /management/v1/spend_logs/end_users`. It is a facet,
the distinct values one column takes over a filtered query on a resource, not an
entity collection; naming it after `customers` implied it listed the end-user
table when it actually reads spend logs, which is a different row set. Serving it
under the parent resource means its filters are the parent's filters, so the
dropdown offers exactly the values the logs table can show without two endpoints
having to keep agreeing on that
Contract changes: `size` becomes `page_size`, `search` becomes `q`, the window
moves from flat `start_date` / `end_date` to `filter[startTime][gte]` / `[lte]`,
and the body becomes `{data, meta, links}`. Unknown query params are now a 400
rather than being silently dropped, because an ignored filter over-returns data.
Errors are RFC 9457 problem documents on this prefix only; every other route
keeps the shape its callers already parse
`links` is what makes the rest deferrable. The dashboard hook follows the
server's `links.next` instead of computing `page + 1`, so moving this to cursor
pagination later changes the links and nothing the client does. That matters
because the inner scan is a sliding window, so offset paging can currently skip
or repeat an end user across pages; the fix is a follow-up, and the hypermedia
means it will not be a breaking one
Cursor mode, `sort`, `include`, ETag / `If-None-Match` and the generic `ListSpec`
framework are all deliberately out of scope here. They are additive or internal,
so none of them needs to beat the release
The Team dropdown popup is pinned to the trigger width via
w-(--anchor-width) and clips its overflow, while Base UI's ItemText
wrapper is flex-1 shrink-0 with min-width: auto, so it sizes itself to
the full nowrap label and simply overflows the popup. Teams without a
team_alias render their 36-char id, so those options were sliced
mid-character with no ellipsis.
Clears min-width: auto off the text wrapper and truncates the label at
the call site. The underlying gap is in the shared Select primitive,
which any long-labelled select in the dashboard will hit; that is left
for a separate change.
GET /v1/tool/spend served the Cost Optimization card with two raw queries
over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load;
the totals query's driving scan was all of SpendLogs in the window. Both
per-request tables reach 1M+ rows at customer scale, so the card cost
O(traffic) per view and had to be capped at 30 days.
The index writer also mined proxy_server_request.tools, i.e. tools DECLARED
in the request body, attributing each request's full spend to tools that
never ran; and all non-MCP mining ran against payload fields that are '{}'
unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently
depended on a privacy setting.
Now the spend writer builds a ToolUsageTransaction at request time from
invoked tools only, resolved by the shared get_tool_calls_from_response
normalizer so every response surface (chat completions, Responses API,
Anthropic Messages) is covered; the tool registry's response arm delegates
to the same owner. Transactions queue beside the spend-log queue and the
flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup
(date, tool_name PK) in one transaction, retrying connection errors with
backoff (a failed batch commits nothing, so the retry cannot double-count)
and dropping the batch with an error log on anything else.
The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS
tools by spend via group_by and daily covers only those tools, so the
response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or
tool-name cardinality; the 30-day clamp is gone. total_spend is dropped
from the response; it was never rendered and its deduplicated semantics
are not computable from a rollup. Spend-log retention deliberately does
not touch the rollup, so tool spend history outlives per-request rows.
The shadcn separator primitive ships `data-vertical:self-stretch` so a bare
vertical divider fills its row, but every call site overrides the height with
`h-5`. A definite cross size makes `align-self: stretch` behave as
`flex-start`, so the dividers rendered flush with the top of their flex line
instead of centered: 0px above and 18px below in the dashboard header, 0px
above and 12px below in the models table toolbar
Routes the three vertical dividers through a ToolbarSeparator that pairs the
fixed height with a same-variant `data-vertical:self-center`. Matching the
variant is what matters; tailwind-merge then drops the conflicting class
outright, whereas a plain `self-center` ties on specificity (the variant is
defined with `:where()`) and loses on utility order. The CLI-managed primitive
is left untouched
DialogContent's close button is absolutely positioned 16px from the right
edge at 32px wide, so it overlays the rightmost 24px of the p-6 content
box. The justify-between header pins "+ Custom Server" to that same edge
and, being out of flow, the close button reserves nothing. Give the action
a right margin that clears it; keeping the margin on the button rather
than the row leaves the header rule full-bleed
The MCP Servers page was the only page-level tab bar using the segmented
(pill) TabsList stretched with w-full, which rendered a full-width grey
bar with a lone pill on the left. Every other page-level tab bar
(budgets, vector stores, access groups, organizations, routing groups,
API reference) uses the underlined line variant, so use that here too.
The shadcn migration carried the antd modal's 1000px width over as an
unprefixed max-w-[1000px], which tailwind-merge keeps alongside the
DialogContent base class sm:max-w-md; the responsive variant wins from
640px up, so the dialog rendered at 448px. Prefix the override so the
merge drops the base clamp
The "Savings over time" chart plotted a single floating dot for short
ranges: the daily rollup keys spend by YYYY-MM-DD, so a one-day range is
one point by construction. Rather than stand up an hourly SpendLogs data
source, read that same daily rollup and make the cumulative line legible.
- Cumulative | Per day toggle. Cumulative accumulates within the range;
Per day shows the raw stacked bars.
- Cumulative prepends a synthetic $0 point at the range start
(withStartAnchor) so the line rises from zero to the running total
instead of floating. An empty series is left untouched so the chart's
own "No data" state shows.
- Order the daily series oldest-first (the rollup arrives newest-first)
so the axis reads left to right and the total accumulates forward.
- Header legend, dots on small series, and a "No data" guard on BarChart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Swap the antd Collapse "How savings are calculated" panel for click-triggered
shadcn Popovers on each SummaryCard, so the explanation sits next to the
metric it describes instead of in one combined block.
Clears five OSV findings the scanner flags on every PR: four gitpython
advisories fixed in 3.1.54, and one postcss advisory fixed in 8.5.18.
gitpython 3.1.55 and brace-expansion 5.0.8 are left for a follow-up; both
were published less than three days ago and are still inside the
dependency cooldown window.
Per-tab path routing for Models made each tab a separate route parsed out
of the pathname, which is fragile under a static export mounted at a
runtime-variable server-root prefix. Revert the tabs to in-memory state:
a single /models-and-endpoints route renders an antd Tabs whose active
tab is React state, and each tab body moves from its own page.tsx into a
non-routed panel component under panels/. Role-gating (which tabs show),
the refresh control and the header are unchanged.
The ?model= / ?team= query drill-in stays: it is query-param based (read
via useSearchParams, written via history.pushState), so it is unaffected
by the server-root prefix and remains shareable. The shared tab-routing
helpers (createTabRoutes / useTabRouting) are untouched; the other four
pages still use them.
Removes the per-tab route dirs, layout.tsx and tabRoutes.ts (+ their
path-routing tests) and replaces the layout's coverage with a page test
for in-memory tab switching, the drill-in overlays and role-gating.
* feat(ui): deep-link virtual key detail view via ?key= query param
Clicking a key on the Virtual Keys page now sets ?key=<token> with
history.pushState, mirroring the models page's ?model= routing, so the
detail view survives reloads and can be shared as a URL. The key is
resolved from the loaded page when present and fetched via /key/info
otherwise. Extracts the shared navigateWithParams helper out of the
models detailNavigation hook
* test(ui): use a realistic hashed token in the virtual keys fixture
The cap was an env-tunable knob in constants.py. Nothing needs to tune it:
it exists so DISTINCT cannot run over an unbounded row set, and picking a
value is a correctness decision, not deployment configuration. An env var
also makes the bound unverifiable, since the same code can behave very
differently between two proxies.
It is now a plain constant next to its only caller, mirroring how
SPEND_LOGS_PAGINATION_COUNT_CAP sits beside ui_view_spend_logs, and it takes
that constant's value: both reads of LiteLLM_SpendLogs now stop at the same
depth. constants.py goes back to matching staging exactly.
The existing test only asserted the parameter equalled the constant, which
is tautological; raising the constant to a billion kept it green while
removing the bound. A second test pins the value against the logs page's
cap, so an arbitrary change to either one fails.
The clamp floor is end_date minus 30 days, serving up to 31 calendar
dates inclusive: deliberately the same width as the endpoint's default
window, so the dashboard's own default range never triggers the clamp
note. The docstring, card note, and test name now state that invariant
instead of the misleading 'most recent 30 days'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two issues Greptile raised on the filter window and the capped scan.
A preset date range ends at "now", which the logs query re-reads on every
fetch, so live tail keeps moving the table's end bound. The filter window
was memoized on the date controls alone, so it pinned whichever "now" it
was first built with: an end user that started sending traffic afterwards
showed up in the table but stayed missing from the dropdown until
something remounted it.
formatLogsWindow now takes the preset end bound as an argument, and
getLogsWindowEndBound derives it from the logs query's last fetch, rounded
up to the next minute. Rounding up rather than down means the filter window
never trails the table; bucketing means the query key holds steady between
ticks instead of churning once per render. The panel reads it from
logsQuery.dataUpdatedAt so it advances exactly when the table refreshes,
falling back to the stored end time before the first fetch. Deriving it
from Date.now() during render is what the purity rule forbids.
The capped inner scan ordered by startTime alone, so rows sharing a
timestamp could be cut differently between two requests and successive
OFFSET pages would disagree about the set they were paging through.
request_id now breaks the tie, which the (startTime, request_id) index
already covers.
Drift from rows genuinely arriving inside the window between page fetches
is left alone. Removing it means keyset pagination over the distinct set,
which cannot keep the inner row cap, and that cap is what stops this
query from degrading into a full scan of LiteLLM_SpendLogs.
GET /v1/tool/spend aggregated LiteLLM_SpendLogToolIndex joined to
LiteLLM_SpendLogs with a start_time-only predicate the composite
(tool_name, start_time) index cannot serve, and the dedup total query
left the outer SpendLogs scan unwindowed, so every dashboard load
walked both per-request tables end to end.
- clamp the window to the most recent 30 days ending at end_date; the
response start_date reflects the effective window and the dashboard
notes the clamp
- index SpendLogToolIndex on start_time (all schema copies + migration)
- window the SpendLogs side of both queries (1s margin: the two writers
can disagree by ~1ms on the same request)
- expire SpendLogToolIndex rows on the spend-log retention cutoff via a
parametrized batch-delete engine shared with the SpendLogs cleanup
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The End User filter listed every row of LiteLLM_EndUserTable, which is both
unscoped and the wrong source. Team admins and internal users can open the
Logs page, and their log view is already restricted to their own requests
plus the teams they administer, but the filter dropdown offered them every
end user on the proxy.
Team attribution only exists on spend logs, so /customer/aliases now reads
LiteLLM_SpendLogs and applies the same scoping /spend/logs/ui does: a proxy
admin sees the whole window, everyone else sees ("user" = caller OR team_id
IN permitted_teams), reusing _get_permitted_team_ids_for_spend_logs so the
two paths cannot drift. A caller with neither matches FALSE rather than
falling through to unscoped, and a failed team lookup degrades to
own-rows-only.
Querying spend logs safely is the other half. start_date/end_date are now
required, so the query always has the indexed startTime bound, and the
inner scan is capped at MAX_SPENDLOG_ROWS_TO_SCAN_FOR_FILTERS rows ordered
by startTime DESC. DISTINCT therefore runs over a bounded row set instead
of the whole table the way /global/all_end_users does.
Also adds /customer/aliases to spend_tracking_routes. Without it RouteChecks
rejects INTERNAL_USER and INTERNAL_USER_VIEW_ONLY before the handler runs,
which would have made the scoping above dead code; a test pins the route to
the same access tier as /spend/logs/ui.
The dropdown now shows the end users present in the window the table is
showing, so the filter list matches what it filters. formatLogsWindow is
shared with the logs query so the two windows cannot diverge.
* feat(ui): migrate the Create Organization form to shadcn and react-hook-form
* fix(ui): guard double submit, test escape-close reset, drop dead organizationCreateCall
* fix(ui): block org create dialog dismissal while a create is in flight
* fix(ui): render budget duration labels instead of raw values in the shadcn Select
TableHeaderSortDropdown had no importers left; the shared DataTable's
DataTableSortHeader covers the same ascending/descending/reset menu on Base UI.
knip did not flag it because its own test file counted as a usage.
* fix(ui): bind key duration input to one Form.Item so pre-filled expiry submits
The Create and Edit key forms kept the displayed expiry in a Tremor TextInput's
local state while the value actually submitted lived in a separate hidden antd
Form.Item. After the first create, form.resetFields() cleared the hidden field
but not the local state, so a second create showed a stale "1d" that was never
sent unless the user deleted and retyped it
Wrap the visible input in a real Form.Item (name="duration") inside
KeyLifecycleSettings and drop both hidden mirror fields plus the local
durationValue state, so what is displayed is always what is submitted. The
Regenerate key flow already used this pattern
* test(ui): restore custom rotation interval coverage in real-form harness
The KeyLifecycleSettings test rewrite dropped the custom interval branch:
selecting Custom interval, typing a value, propagation to the parent, and
hiding the input when switching back to a predefined interval. Cover it in
the real antd Form harness, asserting the parent-held rotationInterval state
instead of a mocked callback
Rebuilds the Router Settings > Routing Groups table on the shared DataTable
and cell library, the last antd entity grid in the dashboard.
The table splits into a thin RoutingGroupsTable container plus
RoutingGroupsTableColumns, with the usage snippets moving to their own
RoutingGroupUsagePanel on ui/tabs and the shared CodeBlock instead of antd
Tabs and Paragraph copyable. Models render through the shared ModelsCell so
long lists collapse behind "+N more" rather than wrapping the row, and the
two inline icon buttons become a single overflow menu with Edit and Delete.
antd gave the snippet panel its own chevron column; under the shared pattern
a row has two click targets, the name cell and the overflow menu, so clicking
the group name now opens the panel. Column set, order, actions, and the
backend row order are otherwise unchanged.
Tremor's TabPanels hands each child an index via React.Children.map, while
the selected index comes from HeadlessUI counting only real Tab elements.
An empty fragment, false, or null still consumes a panel index but
contributes no tab, so the team-only Agent Activity conditional made the
two lists drift for every non-team entity type: Key Activity resolved to
the empty slot and rendered nothing at all, and Endpoint Activity rendered
the key metrics
Drive both lists from a single tab array so adding or removing a
conditional tab touches one place and the indices cannot diverge
The dashboard declared the server-owned object_permission shape by hand
in five places, each with a different subset of fields and none matching
the OpenAPI schema. That is what hid LIT-4766: KeyResponse.object_permission
never declared mcp_toolsets, so a form that wrote the field without reading
it compiled cleanly and silently wiped the grant
Replace four of those copies with one alias over the generated
LiteLLM_ObjectPermissionTable. The agent shape stays separate because the
agent endpoint really does return a narrower type, so it points at its own
generated AgentObjectPermission
Opening Logs > Filters fetched the entire customer table through
/customer/list, which is an unbounded find_many that eagerly loads the
budget and object-permission relations for every row. On a proxy with
61k customers that is a 20 MB, 7.6 s response; the dropdown then built an
option per row and rendered all of them, since the combobox does not
virtualize. The result was a multi-second freeze every time the drawer
opened.
Adds GET /customer/aliases, a projection of user_id alone with page/size/
search, mirroring /key/aliases. The End User field now uses
PaginatedSearchSelect behind an infinite query, the same shape the Key
Alias and Model filters already use, so it fetches 50 rows at a time and
pushes the typed query to the server.
The response reports has_more instead of a total count. A total needs
COUNT(*) over the whole match set on every keystroke, which is the cost
this endpoint exists to avoid; ordering by the user_id primary key and
fetching one row past the page lets Postgres stop early and still tells
the client whether to request more.
LIKE metacharacters in the search term are escaped, because end-user ids
routinely contain underscores and an unescaped one silently widens the
match.
Drops the now-unused accessToken prop threaded from RequestLogsPanel
through RequestLogsTable into the filters.
The gateway acts as an MCP client toward upstream MCP servers, and the MCP
authorization spec requires an MCP client to send the RFC 8707 resource
parameter on both the authorization request and every token request. The
gateway sent it on none of its upstream OAuth legs, so an authorization server
that requires resource indicators rejected the exchange with invalid_target
with no way to configure around it.
Authorization servers disagree irreconcilably and nothing advertises which
camp they are in, so this is a per-server opt-in rather than a default: most
providers ignore the parameter, some hard-reject it and carry audience in
scopes instead, and strict or MCP-native ones refuse to mint a correctly
scoped token without it. The new upstream_resource setting is unset by
default, which keeps today's requests byte-identical.
Both outbound OAuth stacks resolve the value from the server exactly once and
carry it structurally rather than attaching it per call site. In v1 every
plain-OAuth2 token leg builds its body through one helper that resolves the
resource in the same call as the mandatory client authentication; in v2 the
adapter, the single place an MCPServer becomes an outbound config, resolves it
onto the client_credentials config that the HTTP/SSE M2M path uses, and it
joins the config's mint identity so retargeting a live server refreshes the
token rather than serving the previous audience's. A leg cannot authenticate
without also naming the resource its sibling legs named, which is what an
attach-per-call-site approach kept getting wrong.
The setting is non-secret admin config sharing a blob with real secrets, and
the backend classifies which key is which rather than nulling the blob
wholesale or gating on its truthiness: redaction returns admin config to an
admin, session inheritance ignores it when deciding whether a real credential
was supplied and carries it onto the derived server, and the edit form renders
the same shared OAuth component as create so the field exists on both, an
emptied field submitting an explicit null that the credential merge drops.
litellm already supports Google, Microsoft and generic OIDC SSO through
fastapi-sso, which has no SAML support; AuthMethod.SAML existed only as an
unused enum value. This adds real SAML 2.0 single sign-on for the admin UI.
A new SAMLAuthHandler validates signed assertions with the OneLogin
python3-saml toolkit and maps them onto a CustomOpenID, then reuses the
shared post-login path every other provider goes through, so provisioning,
role/team mapping and the UI session JWT are unchanged. Both SP-initiated
and IdP-initiated HTTP-POST flows are supported. SP-initiated logins are
bound to the browser that started them via an HttpOnly state cookie plus a
cached AuthnRequest id, and the ACS rejects any response whose InResponseTo
doesn't match; unsolicited (IdP-initiated) responses cannot be browser-bound
so they are rejected unless SAML_ALLOW_UNSOLICITED=true. Replays are rejected
by a consumed-assertion guard whose lifetime tracks each assertion's
NotOnOrAfter, and both the replay guard and the login-state binding go
through the proxy's shared in-memory + Redis cache for multi-instance
deployments. The ACS honors DISABLE_ADMIN_UI and re-applies the
free-SSO-user Enterprise gate after the assertion is validated, so an
unvalidated POST can no longer drive the billable-user count query.
SAML is configurable from the admin UI SSO settings (IdP metadata URL or
inline XML, SP entity ID, and an allow-unsolicited toggle), which persists
the SAML_* environment variables the handler reads, exactly like the Google,
Microsoft and generic OIDC providers.
python3-saml is kept as an optional saml extra; its xmlsec and lxml wheels
bundle the native libraries so no system packages are required, and the
import is guarded so the proxy still starts without the package with the
SAML routes returning a clear 501.
Resolves LIT-4016
