OpenAI publishes a long-context column on the Flex tier, at half the standard
long-context rate. We had no field for it, so a >272k flex request fell through
to the standard long-context price and billed 2x: Terra $4/$18 instead of
$2/$9, Luna $0.40/$1.80 instead of $0.20/$0.90, Sol $10/$45 instead of $5/$22.50.
Adding the values to the cost map alone does nothing, because get_model_info
builds ModelInfoBase from an explicit kwargs list and silently drops any key
not named there. Declare the four *_above_272k_tokens_flex fields and wire them
through, then add the values for sol, terra, luna, and the gpt-5.6 alias.
That same gap was already swallowing cache_creation_input_token_cost_flex,
_priority, and _above_272k_tokens, which were present in the cost map but never
reached the calculator; they are wired through here too.
Fast mode (ex-Priority) publishes no long-context column, so nothing is added
there rather than deriving a rate by analogy.
Guardrails could only see the MCP tool call request (pre_mcp_call /
during_mcp_call); the tool result went back to the client unscanned, so a tool
that returns sensitive data bypassed every configured guardrail.
Adds a `post_mcp_call` event hook that runs after the tool executes and routes
the result through the unified apply_guardrail seam, so a text guardrail (e.g.
presidio) can mask sensitive values in the tool output or reject the result
without any MCP-specific code of its own.
- MCPGuardrailTranslationHandler.process_output_response now extracts the tool
result's text content into GenericGuardrailAPIInputs["texts"], calls
apply_guardrail with input_type="response", and writes the returned text back
into the content list in place (the logging payload already references that
object, so a copy would leave the unmasked text in the spend log)
- ProxyLogging.post_mcp_call_hook dispatches guardrails that implement
apply_guardrail, gated on should_run_guardrail(post_mcp_call); guardrails
implementing async_post_mcp_tool_call_hook keep their existing dispatch and
are not run twice
- both MCP tool-call paths (mcp_server and the Responses API handler) now honor
the rewritten result, and the REST path no longer swallows a guardrail
rejection as a logging failure
- shared, duck-typed MCP content helpers live in mcp_server/utils.py next to
extract_mcp_tool_result_error_message
- documents that async_post_mcp_tool_call_hook's return value is discarded by
every call site, so that hook only takes effect by mutating in place
Reverts the shared IdCell change from the previous commit and scopes the
fix to the budgets table instead
IdCell truncates with `block max-w-[15ch]`, a character-count clamp with
no relationship to the column's width. On budgets the Budget ID column
renders 509px wide at a 1400px container while the ID stays pinned at
108px, so UUIDs ellipsize with ~400px of empty space beside them
Changing that clamp in IdCell itself is wrong today because nothing else
bounds the column. DataTable emits `width: <size>px` on each cell but
leaves the table in `table-auto`, where `width` is only a hint and
`max-width` on a cell is ignored outright (measured: a 120px request
yields a 938px column). Only `table-fixed` binds `size`, and DataTable
enables it solely under `enableColumnResizing`, which 4 of 40 tables use.
So an unbounded IdCell lets content drive the column: Request Logs would
render a 64-char key hash in full, taking its key_hash column from 124px
to 494px and pushing the table from 1918px to 2326px, introducing
horizontal scroll at 1920 where there was none
Scope it to the call site instead. `cn` is tailwind-merge backed, so a
`max-w-*` passed via className dissolves the base clamp while leaving
`truncate` in place; budget IDs render in full and still ellipsize at the
cell edge if one ever outgrows the column. No other table moves
This is a workaround. The real fix is to make column `size` authoritative
by separating a fixed-layout option from `enableColumnResizing`, then
dropping the per-cell clamps; 307 of 321 column defs already declare a
size, so the mechanical gap is small, but ~20 tables would gain
horizontal scroll at 1440 and that needs its own review
Reverts #32005. Team-scoped keys are governed by the team and team-member
budgets only; the key owner personal max_budget no longer applies to them,
restoring the hierarchy that existed before that PR.
The skip_user_budget_on_team_key opt-out existed solely to turn the new
behavior back off, so it is removed along with the behavior: the
ConfigGeneralSettings field, the /config/list allowed_args entry that
surfaced it as an Admin UI toggle, and the argument threaded through
reserve_budget_for_request and _get_budget_counters.
Regression tests cover both enforcement points in the restored direction:
test_common_checks_personal_user_budget_skipped_for_team_key for the
read-time check and test_should_not_reserve_user_budget_counter_for_team_key
for the optimistic reservation path.
IdCell truncated with `block max-w-[15ch]`, a character-count clamp that
ignores how much room the column actually has. On the budgets table the
Budget ID column renders 509px wide at a 1400px container while the ID
itself was pinned to 108px, so every UUID showed an ellipsis with roughly
400px of empty space beside it. The same held at 900px and 520px
containers; the clamp never moved because it was never a function of the
available width
Switch to `inline-block max-w-full truncate`, the standard CSS idiom for
shrink-to-fit text that ellipsizes at its container. IDs now render in
full whenever the column has room and clip at the cell edge when it does
not. `inline-block` keeps the pill variant sized to its content rather
than stretching the blue background across the column, which a plain
`block` would do once the character clamp is gone
Measured in Chrome across 1400/900/520px containers and both variants:
row height is unchanged, short IDs shrink from a padded 108px to 51px
(plain) and 67px (pill), and the 36-char UUID renders fully at 260px
The MCP gateway resolved a caller's allowed servers and per-server tool
allowlists from the key, the team, the end user and the agent, but never from
the internal user row, so an admin had no way to bound what a person may call
across every key they hold. Anything the key allowed went through
The internal user now carries the same object_permission an admin already
attaches to a key or a team, and the resolver applies it as a ceiling: the
caller ends up with the intersection of what the key allows and what the user
allows, so adding a user entitlement can only narrow, never widen. A level
that names no server and no tool places no ceiling, which keeps every existing
deployment on its current behavior
/user/new and /user/update accept object_permission and reuse the same
create-or-update helper the team endpoints use, so the row is written once and
the three cached views of it (the user row, the object-permission link and the
permission itself) are invalidated on write. Clearing it with an empty object
now really unlinks the permission instead of being swallowed as an empty value
A row that cannot be read at all places no ceiling, but a row that names a
permission the database cannot return denies the call rather than falling
through to the wider set, so a partial outage cannot hand out access the admin
withheld
The users page grows the MCP servers, access groups, toolsets and per-server
tool pickers the key and team pages already have. A save keeps a tool
allowlist whenever an access group or toolset the admin retained could still
supply that server, since an allowlist is what narrows a grant and an absent
one reads as no restriction; it drops the allowlist once nothing indirect
survives to supply the server, so removing a grant really removes it
Auto-routed requests were indistinguishable from ordinary ones once logged:
the spend log recorded the requested model group and the resolved deployment,
but nothing about which tier was chosen or what chose it. That information
existed only inside verbose_router_logger f-strings, so answering "why did my
prompt land on the cheap model" required log access and a running proxy.
The complexity, quality, and adaptive pre-routing strategies now return a typed
StandardLoggingRoutingDecision on their PreRoutingHookResponse, and
Router.async_pre_routing_hook records it once for every attempt. Those three
previously side-channelled their own state through three different metadata
keys; the decision now travels on the hook contract itself, so the bucket is
resolved in one place, through get_or_create_metadata_bucket, which already
owns the question of which dict holds proxy-internal metadata and replaces a
non-dict value instead of skipping the write. Recording happens on every
attempt rather than only on a successful route: a fallback from an auto-router
group to a plain group re-enters the hook with the same request kwargs, and a
decision left behind there would attribute the first router's tier to the
deployment that actually served the retry. The log details drawer renders the
result as a Routing card between Request Details and Metrics; the card is
absent on rows that carry no decision, so ordinary and pre-upgrade rows are
unchanged.
Three defects surfaced while making the recorded cause truthful, each of which
would have persisted a wrong answer. The complexity router hardcoded
cause=complexity_scorer even when the LLM classifier decided, and its silent
fallback to the heuristic on classifier failure meant a row could claim an LLM
verdict the LLM never gave; the cause now reports the path that actually ran.
The keyword that triggered a tier rule was discarded before logging, as was
the escalation keyword. The 2-reasoning-marker override returned REASONING with
a score far below the REASONING boundary and no marker saying so, which reads
as a scoring bug to anyone comparing the two; it now emits a reasoning-override
signal, and the card labels those rows as an override instead of claiming the
score met a boundary. The LLM path no longer reports a synthetic score of 1.0,
and heuristic decisions carry a snapshot of the tier boundaries that mapped the
score, so a historical row stays interpretable after the boundaries change.
Signals name a matched term only when the caller's own message contains it.
Scoring still reads the system prompt, but a term matched solely there is
reported as a count, since signals reach a spend row the caller can read and
naming one would disclose a term from a prompt it cannot see.
routing_decision is stripped from caller-supplied metadata at ingress, so a
client cannot forge its own provenance.
The Auto-Routers tab was proxy-admin only, while Add Model on the same page already
admits team admins. The asymmetry was not a policy decision; the auto-router create form
simply never mounted a team selector, so a team admin's submit was unscoped and POST
/model/new rejects an unscoped create from any non-proxy-admin. Mounting the shared
TeamDropdown closes it, and the tab now takes the same audience as its sibling.
Fixing that surfaced a second, larger problem. The dashboard decided who may edit or
delete a deployment with `(userRole === "Admin" || created_by === userID) && db_model`,
but `created_by` is written at creation and never read by any backend auth check. The API
authorizes on team-admin membership of model_info.team_id, so the dashboard was wrong in
both directions: it hid controls from team admins the API accepts, and offered them to
former team admins the API rejects. Verified against a live proxy; a model created by the
proxy admin was PATCHed and DELETEd 200 by a team admin who did not create it, while the
same key got 403 on another team's row and on an unscoped row.
Both questions now have one owner in utils/modelPermissions.ts, deliberately shaped as a
mirror of ModelManagementAuthChecks. Creation returns a tagged union rather than a pair of
booleans, so "may not create" and "may create unscoped" cannot be confused, and the five
places that had each invented their own spelling (the models page, the auto-routers tab
and panel, the auto-router form, and both branches of AddModelForm) call it instead.
Row affordances are now per row rather than per tab, because opening the tab to team
admins puts routers they cannot act on in the same list.
Note for reviewers: collapsing AddModelForm onto the shared owner changes behaviour for
org_admin and Admin Viewer who also admin a team. They previously got the optional team
selector, because all_admin_roles counts them as admins, and could submit an unscoped
create that the API always 403s; they now get the required selector.
Also corrects stale copy left by the auto-router move. The exclude_auto_routers API
description named a dashboard page, which went stale inside a single PR; it now describes
the concept so it cannot drift with the UI again.
The eslint-suppressions prune includes one entry for caching/_components/cache_dashboard.tsx,
which this branch does not touch. Its baseline was already stale; the gate measures the whole
tree, so it could not be left behind.
Auto-routers had no home and no list. The create form was mounted in two unrelated places,
inside Models + Endpoints > Add Model and again under Cost Optimization, and neither showed
which auto routers already existed; seeing or editing one meant finding its row in the models
table and drilling in. They now get a dedicated Auto-Routers tab beside All Models, listing
every auto_router/* deployment with create, edit and delete in one place, and both former
entry points are removed.
Creating opens in a shadcn dialog rather than swapping the whole panel out, so the list stays
on screen behind it; the dialog caps its height and scrolls, since the complexity form is long.
The form's own heading goes with it, the dialog header owning that now.
An auto router is a routing construct rather than a deployment, so it also comes off the All
Models table. That table pages server-side off total_count, so a client-side filter would page
over a total including rows it never renders; /v2/model/info therefore gains
exclude_auto_routers (default false, so every existing caller is unaffected) and the filter
runs before the count. /v1/models is untouched, so clients still see auto-routers as models.
Clicking a router opens the same `?model=` drill-in the All Models table uses, so it lands in
ModelInfoView with the full Model Settings, Edit Settings, Edit Auto Router and Delete. An
earlier revision had a bespoke detail page here, which was a partial reimplementation of that
view and showed the router's type twice, once as a Type pill and again as a "Routing strategy"
field saying the same thing. Both are gone.
The auto-router list is keyed under the same `models/list` namespace as the models table
rather than a private one. It reads the same /v2/model/info data, and six call sites across
the app already invalidate ["models","list"] after a write; a separate key meant an edit made
through ModelInfoView left the tab stale until a full reload, and every future writer would
have had to remember a second key.
An auto router has no upstream credential, so its detail header drops Update API Key and
Re-use Credentials, and the destructive action names what it removes rather than saying model.
Test Connection was gated on the editor-aware predicate, which let adaptive and quality routers
through to a check that builds its targets from complexity config they do not have; it now
gates on the deployment predicate.
The edit modal also applies the semantic-matching guard the create form has. It renders those
controls now, and the backend raises on semantic_keyword_matching without an embedding model or
keyword rules, so skipping the shared validator turned an inline message into a raw 400.
Whether a row is writable has two independent axes and the dashboard needs both. STRATEGY:
there are four auto_router/* kinds and only complexity and semantic have a form here, so
adaptive and quality must not be handed an editor that would write auto_router_config onto a
deployment storing its settings elsewhere. ORIGIN: a config.yaml row reports db_model false and
the API refuses it whatever its strategy (PATCH /model/{id}/update 404s, POST /model/delete
400s). Capability is derived per capability rather than as one editable flag, because the
constraints differ: editing needs an editor, deleting removes a row by id and never reads its
config, so a DB-created adaptive router stays deletable. Both axes live in
add_model/auto_router_strategies.ts as a declarative table, one record per strategy, so a fifth
strategy is a table row rather than another branch. That also retired four copies of "is this a
complexity router", one of which was written twice in a row in model_info_view.
Creation narrows to the complexity router, which the UI calls Auto-Router v2; the semantic
option was already badged "to be deprecated" in the picker, so the picker goes away along with
the semantic submit path and its validation helper. Existing semantic routers stay editable.
The edit modal mounted ComplexityRouterConfig without the keyword, escalation and
semantic-matching handlers, so those sections never rendered and could only be set at create
time. It now hydrates them from the stored config, and the five keys become managed only when a
caller supplies that state, so a caller rendering no such control still carries them through. A
component-level round-trip test covers it: a payload-builder test cannot see a hydration bug.
A complexity tier is str | list[str] on the backend, and the UI carried three readers of that
rule, one of which dropped a pinned string. They collapse into one owner,
add_model/complexity_router_tiers.ts.
An interactive oauth2 MCP server created with explicit endpoint URLs and no issuer served
400 "authorization url is not configured" from /authorize about a minute after creation,
with the admin's endpoints intact in the row the whole time (#34985). Discovery wrote its
trust-on-first-use issuer into the same column an admin writes, so the next registry build
read the gateway's own output back as an admin pin, anchored the server to RFC 8414
section 3.3, and discarded the stored endpoint columns; one transient metadata fetch
failure then had nothing to serve, and the reload fast path pinned the broken entry until
an unrelated config write
The core of the fix is a deletion. The gateway no longer writes discovery results anywhere:
the OAuth columns and credentials.scopes carry admin intent alone, and everything discovery
learns lives on the in-memory registry entry, as the existing carry-forward already
assumes. With no gateway write there is no value whose provenance a later build can
misread, so the accidental anchoring cannot be expressed
Deleting the write cannot fix a row a released version already stamped, which still reads
as pinned, so a one-time startup heal clears those stamps. The signal is necessarily a
heuristic: updated_by records only the most recent writer and no audit trail says which
field it touched. A row is therefore healed only on the full signature of the defect, which
is discovery as the last writer plus an issuer plus at least one configured endpoint column
that anchoring is actively discarding; rows with an issuer but no configured endpoints are
left alone, since for them both paths resolve from the same upstream document. Every heal
logs the cleared value so an admin who pinned deliberately can re-pin, and the heal records
its own actor, which makes it idempotent
The reload fast path exempts servers missing an endpoint their flow needs, so failed
discovery retries on the normal reload cadence rather than waiting for a config write. Flow
requirements are read through effective_oauth2_flow, the column-first shape-fallback judge
every flow decision uses, so a legacy null-flow M2M row is classified exactly as the
request path classifies it instead of re-discovering forever; a dcr_bridge server with no
configured client needs its registration endpoint for the relay arm, and an entra_obo
server needs a scope, both of which discovery can supply. Retries back off per server,
doubling from one reload cadence to a fifteen-minute cap, so a permanently unresolvable
server cannot re-run the RFC 9728 to 8414 chain and re-log its warning every cycle forever
Deployments with store_model_in_db unset or false loaded MCP servers exactly once at
startup, leaving that retry with no driver, so they now refresh the registry on the same
reload interval. That job deliberately calls a reload-only entry point rather than the
startup composite, keeping the one-time oauth2_flow backfill and issuer heal out of a
recurring path
Losing the persisted trust-on-first-use issuer also means the issuer column no longer
changes underneath the OAuth token identity, so user tokens are purged only when an admin
actually edits the server
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): link organization teams to their team detail pages
On the organization info page the teams shown for an org were plain
badges, so walking to a team meant copying its id and finding it by
hand on the teams page
Team badges now link to /teams?team=<team_id>, which opens that team's
detail page directly since #35112. Adds a shared BadgeLink (a badge
rendered as a real anchor with modifier-aware client-side navigation,
so cmd-click opens a new tab) and a teamDetailHref builder for reuse by
future entity links
* fix(ui): format BadgeLink, split its modifier-click chain, and size it up
prettier wanted the Badge props wrapped, and local/no-long-condition-chain
flagged the four-way modifier-click guard; the guard is now two named
conditions. Linked badges also render slightly larger (text-sm, roomier
padding) than plain badges so clickable entries stand out
* feat(ui): size org model badges to match the linked team badges
BadgeLink's href is now optional; without one it renders the same
enlarged plain badge (no pointer, no hover), so the org page's model
badges share the component and the size while staying non-clickable
* feat(ui): deep link organization detail page via ?org= query param
The organizations page kept the selected organization in React state, so
an org detail page had no URL: it could not be shared, bookmarked, or
opened from another page, and the browser back button dropped you out of
the page instead of closing the detail view
Adds useOrgDetailRouting reading ?org= (same pattern as the api-keys,
models, logs, and teams deep links) and derives the open organization in
OrganizationsPanel from the URL
* fix(ui): reset org edit mode on plain row selection and type test mocks
Greptile P1: with the selected org now URL-derived, browser Back leaves
the detail view without running onClose, so a stale editOrg=true made
the next plain row click open on the Settings tab. Reset the flag on
row selection, matching the teams page
Greptile P2: type the panel test's captured table and detail-view props
from the real components instead of any
The teams page kept the selected team in React state, so a team detail
page had no URL: it could not be shared, bookmarked, or opened from
another page, and the browser back button dropped you out of the page
instead of closing the detail view
Adds useTeamDetailRouting reading ?team= (same pattern as the api-keys,
models, and logs deep links) and derives the open team in Teams.tsx from
the URL. TeamInfo now also derives team-admin rights from the fetched
team data, so team admins arriving via a deep link are not stuck with a
read-only view
* feat(ui): shareable log links via log_id query param on the logs page
Clicking a log row now writes ?log_id=<request_id> to the URL, closing the
drawer removes it, and loading the logs page with ?log_id= opens the drawer
for that log. When the log is not in the loaded page, it is fetched by
request_id (the backend already drops the date window for id lookups), so
links keep working for logs of any age. Drawer open state derives from the
URL, mirroring the models page ?model= pattern.
* fix(ui): close the log drawer on browser back after opening via session id
Session opens now write ?session_id= to the URL instead of holding local
state, so back removes both params and the drawer closes (Greptile P1).
Session views become shareable links as a side effect. In-drawer log
switching now replaces the history entry instead of pushing, so back
always closes the drawer in one step rather than replaying every viewed
log.
* fix(proxy): scope /spend/logs/session/ui to the requesting user's visible logs
Non-admin callers now only receive session rows they could already see on
/spend/logs/ui: their own logs plus logs of teams where they hold the
spend-logs permission. Previously any authenticated user could read any
session's log metadata by id, which shareable ?session_id= links made
trivial to trigger. Admin views are unchanged. Also, clicking a log row
now clears a lingering ?session_id= from the URL so the drawer shows the
clicked log instead of a stale session (Greptile P1).
* feat(ui): chart failed requests as their own series on the cache dashboard
Spend logs for failed requests are stored with an empty call_type, so the
Cache Hits vs API Requests chart lumped them into an Unknown bar that read
as normal LLM API traffic. The activity query now also returns a per-group
failed_rows count (status = 'failure') and the dashboard charts it as a
third stacked series, so failures are visibly separate from successful
requests and cache hits. The chart data transform moves into a pure
summarizeCacheActivity helper with unit tests; header stats keep their
existing semantics (cache hit ratio still counts failures in the
denominator).
* refactor(ui): move cache dashboard aggregation server-side with a typed response
The /global/activity/cache_hits endpoint previously returned raw per
(key, call_type, model) spend-log aggregates typed as LiteLLM_SpendLogs
(wrong), and the dashboard reduced them in the browser: grouping by
call_type, relabeling empty call_type as Unknown, and computing the stat
card totals. All of that now happens server-side. The SQL groups per
call_type and splits cache hits vs successful vs failed requests, a new
cache_activity module validates rows into Pydantic models and computes
totals plus the key-alias/model filter options, and the endpoint declares
a real response_model so schema.d.ts types it correctly. The dashboard
consumes it through a typed $api react-query hook (filters ride the
query key and are applied in SQL instead of the browser), the hand-rolled
summarizeCacheActivity transform and the adminGlobalCacheActivity fetch
helper are deleted, and the refresh button now actually refetches.
The endpoint is UI-internal (hidden from the public swagger), so the
response reshape is not a public API break.
The card variant used viewport breakpoints (md:grid-cols-2 lg:grid-cols-3)
but every card usage sits in a one-third-width grid cell, so on desktop the
narrow card still rendered three internal columns of roughly 100px each and
the text spilled out of its boxes. Switch to Tailwind container queries so
the internal column count follows the card's own width
The aggregate gateway DCR flow ends in a 303 to the client's loopback
redirect_uri. When the MCP client runs on a browserless machine (EC2,
SSH box, container) the user authorizes from a browser on another
machine, so the 303 dereferences the wrong loopback and the code never
reaches the client.
The connect banner now offers manual delivery for loopback clients: the
finish form posts delivery=manual and /authorize/complete renders the
callback URL on a no-store page instead of redirecting. The user pastes
it into the client (Claude Code v2.1.191+ accepts a pasted callback URL)
or fetches it from the client machine's terminal. Manual codes keep the
same sealing, PKCE binding, and single-use guard, with a 5 minute
expiry instead of 2 to survive the copy-paste hop; the used-code marker
TTL derives from the code's own remaining lifetime so the single-use
property holds for the full 5 minutes. The default redirect path is
unchanged.
Resolves LIT-4863
* Fix cache leakage card layout to keep date picker on right and prevent content overlap
Removes flex-wrap and mt-3 to ensure date picker stays pinned to the right side of the card header regardless of zoom level, preventing it from covering card content below
* Remove overflow-hidden from Card to allow dropdowns and overlays to display fully
Fixes date picker dropdown being clipped when opened in cards like the Cache Leakage Card. By removing overflow-hidden from the Card container, popovers, dropdowns, and other overflow content can now display properly without being clipped by the card boundaries.
* Make cache leakage card descriptions consistent with line clamping
Adds line-clamp-2 to ensure both 'by model' and 'by virtual key' cards maintain consistent height. Removes conditional anthropic-specific text that caused height variations between dimensions.
* fix(ui): validate default team values in Default User Settings
The Default User Settings form accepted any free-text team id, and the
proxy persisted it without checking the team exists. New users were then
silently never added to the default team because the consume-time 404
from team_member_add was swallowed at debug level.
Backend: PATCH /update/internal_user_settings now rejects unknown and
duplicate team ids with a 400 naming them, before any persistence or
team budget side effects. Team-add failures in _add_user_to_team now log
at ERROR with user and team ids.
UI: DefaultUserSettings rewritten as a shadcn + react-hook-form + zod
form following the org-settings pattern. The team id free-text input is
replaced with a searchable server-backed team picker, so only existing
teams can be selected; zod blocks empty and duplicate rows. The shared
deriveErrorMessage helper now unwraps the HTTPException detail.error
shape so backend validation errors surface readably in toasts.
* fix(ui): restore read-only view with Edit Settings toggle on default user settings
Parity with the pre-migration form: the tab renders a read-only summary
of the saved defaults, Edit Settings opens the RHF form, Cancel discards
pending edits and returns to the summary, and a successful save returns
to the summary showing the new values. Model sentinel labels in the
summary are derived from ModelSelect's now-exported special values
instead of duplicating the strings.
* refactor(ui): rename MODEL_SELECT_SPECIAL_VALUES_ARRAY to MODEL_SENTINEL_OPTIONS
* fix(ui): move Edit Settings into the card header action slot
gitpython arrives transitively through mlflow-skinny; re-resolved with uv so the
lock moves that one package only. brace-expansion is a dev-only transitive dep
already pinned in the dashboard 'overrides' block, so the pin is bumped
alongside the lockfile to keep the change durable across reinstalls.
5.0.8 narrows its engines range from '18 || 20 || >=22' to '20 || >=22'; the
dashboard already requires node >=20.9.0 and every CI job pins node 20, so
nothing loses support.
Three fixes from an adversarial review of this branch, each at the owning
seam rather than the report site.
The flush retried DB_CONNECTION_ERROR_TYPES, which includes ReadTimeout.
A ReadTimeout is the committed-but-unacked case: the review reproduced the
engine abandoning the transaction open on the pooled connection, the retry
stacking its statements into it, and one commit applying both increment
sets while the flush reports success. The retry now covers only
ConnectError, the one failure that proves the statements never reached the
database; post-send failures drop the batch with an error log. The
docstring no longer claims an idempotency the pattern does not have. The
same hazard exists in the untouched daily spend writer and is left for its
own change.
get_tool_calls_from_response read choices[0] only, so a tool invoked in a
later choice of an n>1 response earned spend but never reached the rollup,
the index, or the registry. Choice scope is now an explicit parameter:
accounting passes include_all_choices=True because every choice costs
money; guardrails keep the primary-choice default because they rebuild the
primary assistant message. First multi-choice fixtures in the suite pin
both scopes.
maxBarSize=64 had been added to the shared BarChart unconditionally,
resizing every existing consumer. It is now a prop; only the tool spend
charts opt in. The legend flex-wrap changes stay global because clipping
overflow was a defect, not a preference.
`/customer/aliases` shipped two days ago and has not been in a release, so its
wire contract is still free to change. This lands it on the control-plane
contract before that stops being true, since after a release the path, the param
names and the envelope would all need a permanent legacy adapter
The endpoint becomes `GET /management/v1/spend_logs/end_users`. It is a facet,
the distinct values one column takes over a filtered query on a resource, not an
entity collection; naming it after `customers` implied it listed the end-user
table when it actually reads spend logs, which is a different row set. Serving it
under the parent resource means its filters are the parent's filters, so the
dropdown offers exactly the values the logs table can show without two endpoints
having to keep agreeing on that
Contract changes: `size` becomes `page_size`, `search` becomes `q`, the window
moves from flat `start_date` / `end_date` to `filter[startTime][gte]` / `[lte]`,
and the body becomes `{data, meta, links}`. Unknown query params are now a 400
rather than being silently dropped, because an ignored filter over-returns data.
Errors are RFC 9457 problem documents on this prefix only; every other route
keeps the shape its callers already parse
`links` is what makes the rest deferrable. The dashboard hook follows the
server's `links.next` instead of computing `page + 1`, so moving this to cursor
pagination later changes the links and nothing the client does. That matters
because the inner scan is a sliding window, so offset paging can currently skip
or repeat an end user across pages; the fix is a follow-up, and the hypermedia
means it will not be a breaking one
Cursor mode, `sort`, `include`, ETag / `If-None-Match` and the generic `ListSpec`
framework are all deliberately out of scope here. They are additive or internal,
so none of them needs to beat the release
The Team dropdown popup is pinned to the trigger width via
w-(--anchor-width) and clips its overflow, while Base UI's ItemText
wrapper is flex-1 shrink-0 with min-width: auto, so it sizes itself to
the full nowrap label and simply overflows the popup. Teams without a
team_alias render their 36-char id, so those options were sliced
mid-character with no ellipsis.
Clears min-width: auto off the text wrapper and truncates the label at
the call site. The underlying gap is in the shared Select primitive,
which any long-labelled select in the dashboard will hit; that is left
for a separate change.
GET /v1/tool/spend served the Cost Optimization card with two raw queries
over LiteLLM_SpendLogToolIndex x LiteLLM_SpendLogs on every dashboard load;
the totals query's driving scan was all of SpendLogs in the window. Both
per-request tables reach 1M+ rows at customer scale, so the card cost
O(traffic) per view and had to be capped at 30 days.
The index writer also mined proxy_server_request.tools, i.e. tools DECLARED
in the request body, attributing each request's full spend to tools that
never ran; and all non-MCP mining ran against payload fields that are '{}'
unless store_prompts_in_spend_logs is enabled, so non-MCP coverage silently
depended on a privacy setting.
Now the spend writer builds a ToolUsageTransaction at request time from
invoked tools only, resolved by the shared get_tool_calls_from_response
normalizer so every response surface (chat completions, Responses API,
Anthropic Messages) is covered; the tool registry's response arm delegates
to the same owner. Transactions queue beside the spend-log queue and the
flush job writes index rows plus a new LiteLLM_DailyToolSpend rollup
(date, tool_name PK) in one transaction, retrying connection errors with
backoff (a failed batch commits nothing, so the retry cannot double-count)
and dropping the batch with an error log on anything else.
The endpoint aggregates in SQL: by_tool is the top TOOL_SPEND_TOP_TOOLS
tools by spend via group_by and daily covers only those tools, so the
response is bounded by days x TOOL_SPEND_TOP_TOOLS regardless of range or
tool-name cardinality; the 30-day clamp is gone. total_spend is dropped
from the response; it was never rendered and its deduplicated semantics
are not computable from a rollup. Spend-log retention deliberately does
not touch the rollup, so tool spend history outlives per-request rows.
