Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
The attempt row now prices the real arm (the payload's response_cost plus its own
routing classifier when it routed) beside the shadow arm (completion plus the
classifier cost the routing decision writes back), and flags turns litellm's
response cache served. A per-leg funnel table counts the eligible requests that
produced no row (lost the sampling dice, unjudgeable shape, concurrency shed),
so results can weigh judged rows against the traffic they stand for. Job results
gain per-slice and overall arm spends plus the coverage counts, the budget gates
charge the shadow arm's classifier spend against max_budget, and the dashboard
shows the measured cost comparison beside the win rate
Resolves LIT-6358
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The search tool create and edit forms both built a payload carrying
api_base, timeout and max_retries read off form values that neither
zod schema declares, so all three were always undefined. Drop them.
JSON.stringify omits undefined-valued keys, so the request body on
the wire is unchanged.
SearchToolLiteLLMParams and SearchToolInfo in the page's types.tsx
were hand-rolled with a [key: string]: any index signature, which is
why a param could go missing from a form with nothing complaining.
SearchToolLiteLLMParams is now the generated OpenAPI component and
neither type carries an index signature, so the payload builder can
only set params the backend declares.
Also remove a stray ", ]" text node that rendered as visible garbage
next to the connection test dialog's Close button.
* feat(ui): edit the auto-router tier set with custom classifier-defined tiers
The editor over the model layer beneath it. An Edit tiers button turns the tier
list into an editor: a tier takes a name, a classifier definition, and models,
between two and eight rows. Restore defaults resets to the built-in four rather
than stacking them on top. Keyword rules follow a rename, an orphaned rule
blocks the save, and both forms dry-run the exact payload against the backend
validator before writing.
The edit modal hydrates a stored custom set into rows, and an untouched
open-and-save round-trips byte-identically, per-model reasoning efforts
included. A form that never opens the editor submits the same bytes as before.
The cost-optimization tier chart renders arbitrary tier names: the guard that
returned no models for a non-built-in name is gone, and the fixed four-color
array gives way to the shared cycle.
* refactor(ui): extract tier editor sections to clear new lint warnings
* test(ui): drop narration comments per repo convention
* fix(ui): default editingTiers so the build's type check passes
* fix(ui): restore the mid-dry-run submit guard and its regression tests
* feat(spend): report prompt caching savings as total and gateway-attributed
`prompt_caching_savings_spend` credited every cached request, including caching a
client asked for with its own `cache_control` and caching a provider does implicitly,
so the number overstated what the gateway had any hand in.
Gating that column in place would have fixed the overstatement by changing what the
column means, leaving rows written before the change saying "all caching savings" and
rows after saying "gateway-injected only" with nothing to tell them apart, and forcing
a decision about rewriting history. It also breaks the cache-leakage estimate on the
dashboard, whose numerator would be gated while its denominator, the cached token
counts, would not, so the rate it extrapolates from would be quietly diluted.
Report both instead. `prompt_caching_savings_spend` keeps meaning every net dollar
caching saved, which is what a customer means by "what did caching save me", and the
new `gateway_injected_caching_savings_spend` carries the subset litellm caused by
injecting the breakpoints itself. Both are derived from the same marker, so this
changes what is done with it rather than how it is obtained.
The attributed figure is normally the smaller of the two, being a subset of the same
requests, but not always: a request that writes cache it never reads has negative net
savings, and excluding such a request can lift the attributed figure above the total.
Also stops the marker riding into a fallback leg. The fallback rebuild spread the
failed attempt's metadata forward, so a deployment that injected nothing inherited the
marker and was credited anyway, which silently restored the very overstatement this
separates out.
* fix(bedrock): credit gateway caching where the tool cachePoint is placed (#38478)
The savings marker records breakpoints litellm placed, and a tool_config
injection point becomes one only in the converse transform, and only when the
request carries tools. The prompt hook cannot see either condition, so marking
on the point's presence credited request shapes that cached nothing, while
Bedrock tool caching the gateway did cause went uncredited.
Record it at the placement site instead. The marker's reader also resolves its
bucket by value now: litellm_params declares litellm_metadata as None on every
request, so asking the shared name resolver named a bucket that was not there
and the mark was dropped.
Model chips on the team info and virtual key info pages (overview and settings tabs) now link to /models-and-endpoints?model_group=<name>, which the All Models tab reads through a new nuqs-backed model group filter. Grant sentinels such as all-proxy-models stay plain badges. The selected group is also sent as the server-side search so the matching deployments are fetched even when they are beyond the first page.
The dashboard's dark theme left a chunk of the bundled provider logos
unreadable: 25 of them are pure black marks on a transparent background,
so on a near-black surface they disappeared entirely, and another 11 are
dark multicolor marks drawn for a white page.
This adds the seam the rest of the work hangs off: a per-asset treatment
manifest in logoTreatments.ts, and a Logo component that applies the
treatment it names. Two treatments exist today. "invert" flattens a mark
to solid white with brightness(0) invert(1), which is what the vendor's
own white mark looks like for a pure-black transparent glyph. "plate"
puts a white surface behind the mark so it reads exactly as it does on a
light page.
Both are dark-only, and only assets named in the manifest are touched, so
light mode is unchanged and the other 96 bundled logos keep rendering
byte for byte as they do today. The className an untreated logo receives
is passed through verbatim rather than routed through cn(), so even the
class string is unchanged.
The split between invert and plate was measured per asset, not guessed:
luminance, saturation and alpha coverage sampled off a canvas render. Two
assets that look monochrome, aiml_api and repelloai, carry a light
knockout inside dark artwork, so inversion would flatten the knockout
into the mark and erase it. They get a plate instead, and a test pins
that.
Six assets whose artwork is an opaque dark box (aim_logo, aim_security,
deepgram, jina, lakeraai, openmeter) are deliberately left untreated. A
plate cannot show through an opaque image, so the only honest fix for
them is a replacement asset.
Two loading/empty transitions the disabled empty state got wrong.
A selection made in a range that had options survives a move to a range that
has none, and it still scopes the data below, so disabling the combobox
outright took away the only control that could clear it. Disable it only when
there is nothing selected to clear.
The customer list defaulted to an empty array while its query was in flight,
so the filter announced a range with no customers before anything had been
read. Leave it undefined until the query resolves, as the tag list now does.
Changing the date range left the previous range's tags in state until the
new request landed, so the filter either offered tags that range no longer
has or, when the old range was empty, stated "no tags" about a range nobody
had measured yet.
Stamp the fetched list with its range key and select it during render, the
same way the request tiles above already guard against a superseded range.
Clearing in the effect would be a render too late.
The usage page seeded its tag list as an empty array, so between first
paint and the tag request landing the filter had a resolved-looking empty
list and showed "No tags with usage in this range" for a scope nobody had
measured yet.
Seed it as null instead, which the filter already treats as unresolved and
hides, so the empty state only appears once the list has actually come back.
The tag usage filter was removed from the DOM whenever the tag list came
back empty, so an internal user whose traffic all runs through team keys
saw a blank Tag Usage panel with no filter and no explanation. Their scope
is legitimately empty, but the page gave them no way to tell that apart
from a broken or gated feature.
Render the filter whenever the entity list has resolved, disabling it and
swapping in an entity-specific empty message when there are no options.
A still-unresolved list keeps hiding the control as before.
An MCP server behind an API gateway needs two credentials on one request: the
gateway's own token on a private header, and a separate bearer on Authorization
for the server behind it. Every arm that minted or held a token hardcoded
Authorization, and the conflict rule then dropped the operator's static
Authorization to make room, so the second credential never arrived.
ApiKeyConfig already modelled this as header_name plus value_prefix behind a
header() method. Extend that carrier to the four minted-token configs, have each
resolver arm ask its config which header to use instead of naming one, and drop
only the header the resolved credential is about to occupy.
Operators set it per server via upstream_token_header, plumbed through
config.yaml, the credentials blob, the management API and the admin form, on the
M2M, token-exchange, authorization-code and ID-JAG arms. It is non-secret so it
stays plaintext and round-trips on admin reads. Unset keeps today's behaviour.
Moving a credential off Authorization means it stops inheriting what Authorization
gets for free, so the slot now carries those protections itself. httpx drops
Authorization when a redirect crosses origin and keeps every other header, so a
custom slot is dropped by the client on the same condition, mirroring httpx's own
scheme/host/port rule with an agreement test that fails if the two ever diverge.
The v1 path also mirrors the v2 conflict rule, so an injected header cannot shadow
the credential the gateway resolved for that slot.
Which header a credential occupies, and what counts as being that header, was
answered independently in nine places by four hand-rolled comparisons. same_header,
has_header and without_header in litellm/types/mcp.py are now the one owner, shared
by both MCP stacks, and the client derives its slot once instead of three times.
The header name reaches egress verbatim, so the RFC 7230 grammar lives in one
place and is checked where servers are built: a bad value fails the config load
and the management API returns 400, rather than raising while a spec is built
and emptying the aggregate tool list for every other server. A blank means unset,
matching what the endpoint already accepts.
The shared SelectContent wrapper defaulted alignItemWithTrigger to true,
which puts Base UI's positioner into item-aligned mode and places the
popup so the active item sits on top of the trigger. In that mode the
side and sideOffset the wrapper passes two lines above are ignored, and
the popup reports data-side="none".
The overlap only becomes visible once the items are tall enough to
matter, which is why the autorouter Template picker shows it clearly:
its options are three-line cards, so the popup covers both the select
box and its own label.
No call site in the dashboard asked for item-aligned mode. 21 of them
across 15 files already passed alignItemWithTrigger={false} by hand to
undo the default, and the remaining 127 inherited the bug. Flipping the
default makes side and sideOffset live, so collision handling works and
a select with no room below now flips above the trigger rather than
covering it. The 21 hand-written opt-outs are deleted as redundant.
/v2/model/info returns llm_router.model_list, which carries no defined order: the DB
read has no order_by and an edited deployment is popped and re-appended. The Auto
routers table rendered that order verbatim behind a ten-row first page, so on a proxy
with more than ten auto routers a router created moments ago was drawn wherever the
API happened to return it, in practice last, and read as never created
Adopt the ordering the rest of the dashboard already uses, with the two cases this
table has and its siblings do not. created_at is enterprise-gated and config.yaml
routers never carry one, so seeding created_at desc alone leaves every comparison
tied on a non-premium proxy and the fix a no-op. The column now declares
sortUndefined last, which table-core applies before the desc flip so undated rows
stay last in both directions, and the row emits undefined rather than null so that
branch is reachable at all. Name is the secondary key, giving the undated block a
defined order too
Page size is deliberately unchanged: it exposes the missing order rather than
causing it
The savings card carried four numbers in two stacked halves: the headline
saving with its delta on the left over the two spend rows, and avg saved per
session on the right. Give the headline the whole left half, move the two
spend rows into a rail on the right, and drop avg saved per session into the
metric row below as its first tile, with the session count as an inline hint.
Each spend row stays a description list so assistive tech keeps the label to
value association, with the shadcn Separator between the two rows. Both hero
columns are minmax(0,1fr) so a large total wraps instead of overflowing the
card, which also fixes the clipping the old 1fr columns already had. Metric
grows one optional hint slot so the new tile reuses the same presenter as its
three siblings.
* feat(complexity_router): heuristic-first classifier chaining
Adds classifier_type 'heuristic_first', which scores locally on every request and
only calls the LLM classifier for traffic the scorer could not place at or below
heuristic_first_max_tier. A request short-circuits when the scorer landed at or
below the threshold and produced at least one signal; everything else escalates.
The signal requirement is load-bearing. A prompt where no dimension fires scores
exactly 0.0, which is under simple_medium, so the score-to-tier mapping calls it
SIMPLE by default rather than by evidence, and that is about half of general
traffic. Gating on the tier alone would route it to the cheapest model without
ever consulting the classifier.
Introduces uses_llm_classifier as the single owner of 'does this router call the
classifier model', replacing the classifier_type == 'llm' comparisons in the
config validator, the prompt prebuild, the health dependency graph, the
routing-test authorizer, and six dashboard sites.
* fix(complexity_router): reuse the heuristic verdict on classifier failure, load the threshold on edit
Three review findings, one push.
The heuristic-first fallback re-scored the prompt after a classifier failure,
which the README already documented as a reuse. The outcome computed before
escalation is now handed to the failure path, so the scorer runs once per request.
The edit modal never hydrated heuristic_first_max_tier, while save rebuilds every
managed key from form state, so opening a heuristic-first router and saving it
dropped a field the proxy requires. The dropdown's display fallback hid it. Both
are fixed, and the hydration is extracted into a pure function so a test can pin
the invariant: every managed key present in a stored config survives an untouched
open-and-save. That test also covers every field added later.
Classifier radio labels lost their em dashes, per the repo writing convention.
The badges flagged UI that shipped a while ago, so they no longer tell
anyone anything. Dropped all four render sites: the Settings and Admin
Settings items in the left nav, the UI Settings tab in the admin panel,
and the Submitted MCPs tab.
The NewBadge component stays so the next genuinely new surface can use
it again. BetaBadge and the "hide new badges" account toggle are
untouched, since that toggle still gates BetaBadge.
Virtual Keys, Budgets, Projects, Access Groups, Guardrails Monitor and Cost
Optimization all move onto the shared PageHeader, matching the Teams page.
That empties LegacyPageHeader, so it and its test are deleted.
Each page now uses its own sidebar icon, so the nav and the page agree:
Virtual Keys keeps KeyRound and Budgets keeps Wallet, Projects picks up
Folder and Access Groups picks up Boxes, and Guardrails Monitor swaps the
indigo Shield for the sidebar's HeartPulse. Cost Optimization keeps
PiggyBank but drops its hardcoded size and stroke, which PageHeader owns.
Control rows follow the spec instead of each page inventing one. Virtual
Keys had its create button rendered as a sibling below the header, Budgets
hand-rolled a row with a bottom border that closed the header off, and
Projects and Access Groups sat their button next to the title. All four now
pass primaryAction. Guardrails Monitor's date picker moves out of the parent
and joins Export Data in utilities. Cost Optimization's tabs move into the
tabs slot with the standard 22px spacing.
Page insets go to p-8 with a 24px gap to content, replacing p-6 px-12,
p-6, mx-4 and py-2.
Every page test now asserts its heading, subtext and sidebar icon. Swapping
any of the six icons fails its suite.
Regression LIT-6143 (the policy Flow Builder painting its guardrail dropdown
underneath a position: fixed shell at z-index 1000) was one instance of a
class of bug: pages picking their own z-index numbers above the portalled
popup layer. This removes the class.
- globals.css defines the only z-index values in the dashboard as Tailwind
utilities: z-raised, z-chrome, z-sticky, z-sticky-pinned, z-floating,
z-overlay, z-popup; every numeric, arbitrary and inline z-index across
src is migrated onto them and tailwind-merge learns the tokens
- new local/no-ad-hoc-z-index ESLint rule bans z-<n>, z-[...], z-(...) and
inline zIndex everywhere, and reserves z-popup for the portalled
primitives in components/ui (and the DataTable menus)
- the Flow Builder renders in the dashboard content area instead of as a
fixed full-screen overlay, so it has no stacking level at all
- the guardrail content-filter Add keyword / Add pattern / Custom pattern
dialogs drop the leftover z-[1100] (renamed from ABOVE_ANTD_MODAL when
antd was removed) that hid their own Action select and pattern combobox
behind the dialog, the same bug as LIT-6143
The edit form's Authorize & Fetch Token button built its temporary OAuth
session payload without issuer, authorization_url, token_url, or
registration_url, unlike the create form's equivalent payload builder. The
backend's temporary-session endpoint builds its ephemeral server purely from
that payload, so any admin-configured OAuth endpoints on an existing server
were silently dropped, endpoint discovery fell back to (and failed against)
the plain server url, and Authorize & Fetch Token 400'd with "authorization
url is not configured" even though the saved server had those fields filled
in. Add the four missing fields to the edit form's temporary payload builder,
mirroring the create form.
The drilldown now self-dismisses when refetched activity has no failures
for its call_type, instead of holding a selection the chart no longer
shows. groupErrorBuckets is rewritten as pure filter/map/sort over the
already-grouped SQL rows.
/global/activity/cache_hits now returns an error_breakdown: failed spend
logs bucketed per call_type by error code and error class, read from
metadata->error_information. Clicking a red failed-requests segment on
the cache activity chart opens a per-code bar chart; hovering a bar
lists the error classes behind that code.
* refactor(ui): install the shadcn field primitive
`components/shared/form/field.tsx` was the upstream base-vega `field` source
living outside `components/ui/`. It exported the same ten symbols as upstream,
so `npx shadcn add` could never update it and it had already drifted: its
`FieldLabel` was missing the hover and focus-visible ring utilities upstream
now ships for labels that wrap a nested field.
Install the primitive and point the 77 importers at it. The copy is deleted
rather than kept as a wrapper because it added nothing beyond `forwardRef`,
which React 19 makes unnecessary since `ref` arrives as an ordinary prop.
`field.test.tsx` moves next to the primitive with no edits to its contents,
and its nineteen tests, ref assertions included, pass against the generated
file. That is the evidence the swap is behaviour-preserving.
Two nested-field call sites pick up the upstream hover and focus-visible
styling that the stale copy had been missing.
(cherry picked from commit 947f7fa674c83bfc57f43ad8bfc89c894da947a2)
* test(ui): cover the nested-field interaction cues FieldLabel had lost
The stale copy of `field` was missing the hover, focus-visible and disabled
selectors upstream applies to a label that wraps a nested field, so installing
the primitive restored them with nothing asserting they stay.
Assert the class contract rather than the rendered effect. jsdom evaluates
neither `:has()` nor `:focus-visible`, and Tailwind is not compiled under
vitest, so a test that clicked or tabbed would pass on an element with no
styling at all. Checking the utilities are present is the assertion that
actually fails when they go missing, which is the way they were lost before.
Verified by stripping the four selectors from the primitive: both tests fail,
and both pass once it is restored.
(cherry picked from commit 5a5dbf64270d9d1285dbc4a7af76bb3d927778a8)
The dashboard used `cva@1.0.0-beta.4` with the object-argument API behind
`@/lib/cva.config`, while shadcn emits `class-variance-authority` with the
positional API. Every `shadcn add` of a cva-based primitive therefore needed a
hand fix-up before it compiled, which meant `components/ui/` could never match
a fresh CLI run and `shadcn add <name> --diff` reported the whole file as
changed instead of showing real upstream drift.
Swap the dependency, and regenerate `badge`, `button`, `button-group`,
`input-group` and `tabs` straight from the base-vega registry so they are now
byte-identical to the CLI output plus prettier.
Two primitives could not be regenerated because they are local code rather
than registry items, so they move out of `components/ui/`: `sidebar` (203
lines against upstream's 730, and only `leftnav` consumes it) and `meter`
(no registry entry at all, it wraps Base UI's Meter).
The customisations that were baked into the regenerated files move to
wrappers, following the rule that `components/ui/` holds CLI output and
anything on top of it lives outside:
- badge carried info, success and warning variants that duplicated the
existing `StatusBadge` tone map, so its five call sites now use
`StatusBadge`, which gains an optional `className`
- input-group's addon focuses `[data-slot=input-group-control]` rather than
upstream's `input`, which matters because the chat composer puts a textarea
there. That handler now sits at the one call site that needs it
`cx` keeps its previous twMerge behaviour. It came from the old
`defineConfig({hooks: {onComplete: twMerge}})`, and CVA's own `cx` is plain
clsx, so pointing it at `cn` avoids silently dropping conflict resolution in
the six files that use it.
`Sidebar.test.tsx` covers the failure mode this migration can hide: passing
the object form to the positional API is accepted by clsx and renders the
literal class string "base variants defaultVariants", so the component loses
every style while the type checker and the existing suite stay green.
The playground message bubble painted its fill, border and avatar circle from
inline hex values, so in dark mode both bubbles stayed near-white while the text
inherited the dark foreground: the message body was unreadable. The MCP-events
placeholder bubble in ChatUI carried the same three fills.
They move onto the tokens the rest of the sweep already uses, so the assistant
surface is bg-card over border-border and the user surface is the info tint at
the same weight the other selected-state surfaces take. Light mode keeps the
same colour family it had.
The regression test asserts the token classes and that no inline style survives
on either surface, which is the exact shape the bug took.
* fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited
A tpm_limit or rpm_limit of 0 is a hard block on the backend (every request 429s) and only null means unlimited, but the team and organization views rendered both as "Unlimited" (and a team-member limit of 0 as "No Limit") because every display site used a falsy || fallback. The team member edit dialog also seeded its form with `tpm_limit || null`, so opening Edit Member on a member stored with 0 and clicking Save sent null to /team/member_update and silently turned the hard block into unlimited
Every limit display site in TeamInfo, organization_view, the organizations list cell and the team members table now uses a nullish check, and both member form seeding paths keep 0 for max_budget_in_team, tpm_limit and rpm_limit. Regression tests cover each site and the existing memberFormValues test that asserted 0 -> null is flipped to assert 0 survives
Resolves LIT-5760
* test(ui): assert a stored 0 member limit survives an untouched save
The EditMembership integration test named the old 0 -> null collapse as the expected payload, so the related-tests CI job went red once the form kept 0. It now asserts 0 survives and only the empty budget_duration collapses to null. The TeamMemberTab fixture is built with a map instead of mutating the nested membership
PR #36130 added a KNOWN_MODEL_MODES guard to isModelCompatibleWithEndpoint
that hides any model whose mode isn't in the ModelMode enum, to keep
rerank/ocr/batch/etc. models out of chat-style endpoints. mode: completion
(legacy text-completion models) wasn't in that enum, so it got caught by
the same guard and disappeared from every endpoint, including chat, where
it routes fine.
Add ModelMode.COMPLETION and map it to EndpointType.CHAT like the other
chat-compatible modes.
* feat(ui): add per-key Savings tab to key detail page
Adds a "Savings" tab to the key detail view, showing the same four metrics
and time-series chart as the proxy-wide Cost Optimization view, but scoped
to a single API key.
For org admins, the tab shows the key's full savings across all requests.
Non-admins see only their own requests on the key, with a scope note
explaining the limitation.
Root cause: userDailyActivityCall and userDailyActivityAggregatedCall
never forwarded an api_key query parameter to the backend, even though
both handlers already accept and filter by it.
Changes:
- networking.tsx: Add optional apiKey param to both daily activity call
wrappers (appended to variadic options tuple for backward compatibility).
- costOptimizationUtils.ts: Extract shared metrics helpers (compressionOf,
cachingOf, autorouterOf, savedTokensOf, cacheHitRatio) and shortDate
so both UsageTab and KeySavingsTab use the same formulas and prevent
divergence.
- useDailyActivityRange.ts: Refactor into useScopedDailyActivityRange(
accessToken, scope: {userId, apiKey?}) for reuse-by-parameter unbundling.
Role resolution stays at the entry point (useDailyActivityRange), not in
a scoped caller. Update test expectations for new 6-arg tuple.
- UsageTab.tsx: Simplify by importing extracted helpers and SummaryCard
component instead of defining them inline. No behavioral change.
- key_info_view.tsx: Insert "Savings" tab trigger between "Overview" and
"Settings"; wire TabsContent to new KeySavingsTab component with lazy
mounting (no keepMounted) to defer daily-activity fetch until tab opened.
- NEW: components/shared/SummaryCard.tsx — Shared presenter for four-tile
summary row (label + value + hint + optional info popover). Extracted
from UsageTab so both surfaces show identical tile layout without CSS
divergence.
- NEW: components/templates/KeySavingsTab.tsx — Per-key view with admin/
non-admin scope branching, empty-state messaging, same chart toggles
and info popovers as UsageTab.
- NEW: components/templates/KeySavingsTab.test.tsx — 7 tests covering mount,
loading state, empty state, scoping, and scope-note visibility.
Authorization: No new permission check. Both backends gate api_key filter
by the same user role check that governs the request itself. Non-admins
must send their own user_id and can only see their own keys.
Tests: 6121 pass (1 pre-existing failure unrelated to this change).
Prior art / collision note:
- PR #37570 (budgets tab) lands in same TabsList hunks as "Savings" tab,
but different tab names so conflict trivial if both merge.
- PR #37659 (my own) adds progress/cancelled/cancel to DailyActivityRange,
but this PR uses stable three-field interface from staging.
* fix(ui): scope spend view by the backend's admin-view contract, not all_admin_roles
Greptile flagged org admin handling on the key savings tab. The live bug it
described does not fire today: useAuthorized supplies session-role labels and
all_admin_roles only carries the raw org_admin spelling, so an org admin was
already scoped. That safety was accidental, so replace the predicate with
spendScopeUserId / hasProxyWideSpendView in utils/roles.ts, mirroring the
backend's user_api_key_has_admin_view (proxy admin and admin viewer only, org
admin excluded in both spellings), and use it in both useDailyActivityRange
and KeySavingsTab
Reclassify the KeySavingsTab render test as an integration test per the
repo's unit/integration split, move scope-resolution coverage to roles.test.ts
as a full role matrix, use real session-role values instead of raw ones, and
assert tile totals against non-empty metrics. Replace the nested ternary in
the chart body (frontend-lint error) with flat conditional rendering
* fix(ui): show auto-router savings as the fourth key-savings tile
Cache hit rate had displaced auto-router savings from the fourth slot,
diverging from the org-wide Cost Optimization page's tile order. Match
it: Total / Compression / Prompt caching / Auto-router, with cache hit
rate as a fifth tile.
* fix(ui): drop cache hit rate from the key savings tiles
Keep the four tiles this page is meant to show: total, compression,
prompt caching, and auto-router savings.
* fix(ui): stop an empty api_key from widening a key-scoped activity read
The paginated and aggregated daily-activity wrappers disagreed on an
empty filter value: the paginated one appended it, the aggregated one
coerced it to undefined with || and dropped it. Since the aggregated
call is the one tried first, an empty key hash would have silently
turned a key-scoped read into a proxy-wide one and reported every
key's savings as this key's. Use ?? so both send the filter through
and it matches nothing instead.
* style(ui): satisfy prettier and the inline-object lint rule in key savings tests
* refactor(ui): drop the cacheHitRatio extraction left over from the removed tile
* fix(ui): pass daily-activity filters raw so both transports agree at the null boundary
* refactor(ui): share the savings tiles and totals between both surfaces
The per-key Savings tab and the proxy-wide Cost Optimization tab carried a byte-identical
four-tile block, three long metric-definition strings included, and five identical useMemo
totals. Both now render SavingsTiles and total through useSavingsTotals, so the donut cannot
slice numbers the tile above it disagrees with.
* docs(ui): say request, not mount, in the savings tab comment
The comment claimed mounting eagerly would fire the rollup sweep, which reads as a claim about
the bundle. Only the request is deferred; the module ships with the key page either way.
* test(ui): pin the daily-activity args array against the real caller signatures
The sibling unit test mocks networking, so it checks the positional array against itself and
stays green when the array and a networking signature drift apart. Swapping user_id and api_key
in the aggregated signature alone passes there and fails here on user_id=hash-abc.
* style(ui): hoist the daily-activity query options out of the call argument
The four-property object literal tripped local/no-large-inline-object-arg. The violation predates
this branch, which only moved the line into the annotated range, and the rule count drops 550 to 549.
* fix(ui): drive auto-router usage from the shared cost-optimization time picker
* fix(ui): extend a live-ending benchmarks range to the current UTC day
Per-model budgets were three separate things pretending to be one. The
enforcement check, the post-call increment and the info endpoints each derived
their own cache key, so a budget could refuse traffic at 429 while /key/info
reported zero usage, and a Bedrock model id never matched a budget keyed on the
bare family name. /user/new echoed a model_max_budget back and stored an empty
dict, and nothing enforced a user-scoped per-model budget at all.
One owner now builds the counter key from the configured budget model, and
enforcement, the increment and the info endpoints all read it. Bedrock ids
resolve through the model-cost map. Auth carries the user's budget onto the
token on every branch that reaches the spend hook, including JWT and
auto-registration. Native passthrough attaches the three budget metadata keys
its StandardLoggingUserAPIKeyMetadata does not carry, so /anthropic/... and
/bedrock/... traffic is counted and capped like /v1/chat/completions.
The dashboard gains the per-model budget editor it never had, on the key create,
key edit and internal-user edit forms. It is read-only without an enterprise
license, matching the write gate the proxy already enforces, and an untouched
budget is left out of an update so an unrelated edit cannot trip that gate.
The editor hydrates from either BudgetConfig spelling, since model_max_budget is
a plain dict that the proxy stores exactly as the client sent it, and it carries
through the fields it does not model. Without both, editing one model would drop
another model row entirely and silently discard its tpm_limit and rpm_limit.
/user/info refreshes its local copy of the user field by field after a save, so
model_max_budget joins that list. Left out, a saved cap read back as the old one
when the form was reopened, and clearing the row to recover would then wipe the
value that had actually persisted.
A zero-dollar cap is the strictest limit expressible, not the absence of one,
so it is enforced rather than skipped on falsiness, spend exactly at the cap is
refused the way every sibling budget check already refuses it, and a counter
that was never written reads as zero spend rather than as unknown. The usage
endpoints read every counter in one batched lookup, so a large model_max_budget
cannot fan out into one concurrent cache call per configured model.
Every auth path honours the same zero-cost skip flag, so none of them can refuse
a free request that another serves. The custom-auth helper gains the flag it
never had, which also changes its pre-existing key and end-user checks.
The compaction summary gate checks the user scope alongside the key and end-user
ones. This file propagates all three budgets into the summary subrequest, so
enforcing only two let compaction increment a counter it could not be refused by.
Custom auth attaches the user's budget to the token unconditionally, since the
post-call spend hook reads it there: gating the attach on the same condition as
enforcement left the counter uncharged whenever the request was not itself
enforceable. An entry that will not validate is skipped rather than raised on,
so one malformed scope cannot abort every other scope's increment or turn a
config typo into a 500.
The edit forms re-seed the budget editor when a different key or user is loaded.
Its rows are seeded once and cannot re-read their own value prop, so without
this a save wrote the previously loaded record's budgets onto the current one.
Only the built-in provider pass-through routes carry the budget metadata.
get_model_from_request deliberately resolves no model for a user-defined
pass-through, since its body is forwarded verbatim and names an upstream model,
so attaching there would charge a counter nothing on that route can refuse.
team_allowed_routes and admin_allowed_routes only matched exact strings or named route groups, so a whole prefix of pass-through endpoints had to be listed route by route in config. Match trailing-wildcard patterns with the same helper the key-level allowed_routes check uses, so "/prefix/*" covers endpoints registered later.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Model-Specific Limits rows now carry Input TPM and Output TPM, and a
limit the operator removes is sent as an explicitly empty map so
/project/update actually drops it instead of leaving the stored quota
enforced behind a UI that shows it gone.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The hook swallowed errors into the console, so an admin flipping the switch without
STORE_MODEL_IN_DB saw nothing happen and got no reason why. Adds the missing hook tests.
