The author does not want fetchAvailableModelsForTeam fanning out a second request,
so it goes back to the single /models call it was before. The team path carries no
capability metadata again, which is what it did prior to this branch, and the effort
options still come from /model_group/info on the non-team path.
get_model_info answers supports_reasoning None both for a model absent from
the map, which the router registers under a synthesized entry, and for a
mapped model that simply is not a reasoning model. Reading both as "adds no
levels" let one custom deployment wipe every level its mapped siblings agreed
on.
The synthesized entry carries no mode, which every real map entry for a
routable model does, so an unset flag with no mode now resolves to unknown and
never narrows its group. A group that genuinely shares no level still
advertises none, and the dashboard drops the effort control for it instead of
offering levels routing would refuse.
The chat-completions gate only ever owned xhigh. Widening it to max and ultra
made gpt-5.6 answer 400 on requests litellm itself converts to /v1/responses,
where max is valid, because the gate runs before the bridge decision. No map
entry asserts either flag, so the widened gate could only ever reject.
An empty per-group intersection now falls back to the capability-blind level
list in the dashboard, matching what the picker showed before the field
existed, and ModelGroupInfo tolerates whatever shape an operator writes under
supported_reasoning_efforts instead of failing the whole /model_group/info
response.
* fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): type the arbitrary-uid image test fixture
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Six defects in the RunwayML video provider:
- transform_video_create_request hardcoded /image_to_video, so text-to-video 400'd and video-to-video was unreachable; the endpoint is now selected from the inputs present (promptVideo/videoUri, promptImage, or text only)
- get_error_class raised instead of returning, turning a provider 4xx into a proxy 500 APIConnectionError; it now returns a RunwayMLError
- VideoObject.progress was typed int while Runway sends a 0..1 float, 500'ing status polls while RUNNING; it is now scaled to a 0..100 percent
- custom per-deployment pricing stored under litellm_metadata was ignored for video; the deployment model_info lookup now checks both metadata keys
- stale cost-map entries (gen3a_turbo, gen4_aleph) were removed and current models added, with output_cost_per_second_480p/_4k tier keys plumbed through the model-info and router types
- video cost now falls back to Runway's estimatedCost from the create response when no custom pricing is configured, and custom pricing always wins over it
Fixes#36483
Resolves the test-file conflict by keeping both sides, extends the
finish-reason gate to trace-bearing metadata events so guardrail trace
chunks keep their pre-regression delta shape, parametrizes the
regression test over tool-call, mixed, and reasoning streams, and
repairs the one ant-design icon usage the lucide-react migration left
behind in skill_detail.tsx (semantic conflict on the base branch)
The playground message bubble painted its fill, border and avatar circle from
inline hex values, so in dark mode both bubbles stayed near-white while the text
inherited the dark foreground: the message body was unreadable. The MCP-events
placeholder bubble in ChatUI carried the same three fills.
They move onto the tokens the rest of the sweep already uses, so the assistant
surface is bg-card over border-border and the user surface is the info tint at
the same weight the other selected-state surfaces take. Light mode keeps the
same colour family it had.
The regression test asserts the token classes and that no inline style survives
on either surface, which is the exact shape the bug took.
Regenerating a key from the key info page left the ?key= query param on the
old hash, so dismissing the dialog or reloading landed on a key that no longer
exists and the page rendered "Key not found".
Two defects had to line up. POST /key/{key}/regenerate returns the rotated
hash in token_id and leaves token null, but RegenerateKeyModal read
response.token || response.key_id, neither of which the endpoint populates, so
it always reported the old hash back to its parent. And KeyInfoView's
onKeyDataUpdate prop had no caller anywhere in the tree: VirtualKeysTable owns
the ?key= param and mounts the view but never passed it, so even a correct
hash went nowhere.
VirtualKeysTable now handles the update by pointing ?key= at the rotated hash
and refetching. KeyInfoView holds that callback until the regenerate dialog is
dismissed rather than firing it on the API response, because swapping the
selected key mid-dialog unmounts the view and tears down the one-time
plaintext key before the user can copy it.
* fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited
A tpm_limit or rpm_limit of 0 is a hard block on the backend (every request 429s) and only null means unlimited, but the team and organization views rendered both as "Unlimited" (and a team-member limit of 0 as "No Limit") because every display site used a falsy || fallback. The team member edit dialog also seeded its form with `tpm_limit || null`, so opening Edit Member on a member stored with 0 and clicking Save sent null to /team/member_update and silently turned the hard block into unlimited
Every limit display site in TeamInfo, organization_view, the organizations list cell and the team members table now uses a nullish check, and both member form seeding paths keep 0 for max_budget_in_team, tpm_limit and rpm_limit. Regression tests cover each site and the existing memberFormValues test that asserted 0 -> null is flipped to assert 0 survives
Resolves LIT-5760
* test(ui): assert a stored 0 member limit survives an untouched save
The EditMembership integration test named the old 0 -> null collapse as the expected payload, so the related-tests CI job went red once the form kept 0. It now asserts 0 survives and only the empty budget_duration collapses to null. The TeamMemberTab fixture is built with a map instead of mutating the nested membership
The tooltip popup is an inline-flex row, so the four sibling blocks passed as a fragment laid out side by side in four columns. Wrap them in a single flex-col container instead.
The inline code samples also used bg-muted, which is defined against the page surface, not the inverted tooltip surface, so they rendered as near-white chips carrying near-white text. Tint them from the popup's own token instead.
The virtual key shown after creating a key sits in a div with a
hardcoded #f8f8f8 inline background, so in dark mode the box keeps
the light background while the key text inherits the light foreground
color, leaving the key nearly unreadable. Swap the inline styles for
the bg-muted and text-foreground tokens, which resolve per theme.
PR #36130 added a KNOWN_MODEL_MODES guard to isModelCompatibleWithEndpoint
that hides any model whose mode isn't in the ModelMode enum, to keep
rerank/ocr/batch/etc. models out of chat-style endpoints. mode: completion
(legacy text-completion models) wasn't in that enum, so it got caught by
the same guard and disappeared from every endpoint, including chat, where
it routes fine.
Add ModelMode.COMPLETION and map it to EndpointType.CHAT like the other
chat-compatible modes.
Drops a pyright suppression, getattr string access, and bare dict annotations from the model_max_budget code, and trims a comment referencing its own PR.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add per-key Savings tab to key detail page
Adds a "Savings" tab to the key detail view, showing the same four metrics
and time-series chart as the proxy-wide Cost Optimization view, but scoped
to a single API key.
For org admins, the tab shows the key's full savings across all requests.
Non-admins see only their own requests on the key, with a scope note
explaining the limitation.
Root cause: userDailyActivityCall and userDailyActivityAggregatedCall
never forwarded an api_key query parameter to the backend, even though
both handlers already accept and filter by it.
Changes:
- networking.tsx: Add optional apiKey param to both daily activity call
wrappers (appended to variadic options tuple for backward compatibility).
- costOptimizationUtils.ts: Extract shared metrics helpers (compressionOf,
cachingOf, autorouterOf, savedTokensOf, cacheHitRatio) and shortDate
so both UsageTab and KeySavingsTab use the same formulas and prevent
divergence.
- useDailyActivityRange.ts: Refactor into useScopedDailyActivityRange(
accessToken, scope: {userId, apiKey?}) for reuse-by-parameter unbundling.
Role resolution stays at the entry point (useDailyActivityRange), not in
a scoped caller. Update test expectations for new 6-arg tuple.
- UsageTab.tsx: Simplify by importing extracted helpers and SummaryCard
component instead of defining them inline. No behavioral change.
- key_info_view.tsx: Insert "Savings" tab trigger between "Overview" and
"Settings"; wire TabsContent to new KeySavingsTab component with lazy
mounting (no keepMounted) to defer daily-activity fetch until tab opened.
- NEW: components/shared/SummaryCard.tsx — Shared presenter for four-tile
summary row (label + value + hint + optional info popover). Extracted
from UsageTab so both surfaces show identical tile layout without CSS
divergence.
- NEW: components/templates/KeySavingsTab.tsx — Per-key view with admin/
non-admin scope branching, empty-state messaging, same chart toggles
and info popovers as UsageTab.
- NEW: components/templates/KeySavingsTab.test.tsx — 7 tests covering mount,
loading state, empty state, scoping, and scope-note visibility.
Authorization: No new permission check. Both backends gate api_key filter
by the same user role check that governs the request itself. Non-admins
must send their own user_id and can only see their own keys.
Tests: 6121 pass (1 pre-existing failure unrelated to this change).
Prior art / collision note:
- PR #37570 (budgets tab) lands in same TabsList hunks as "Savings" tab,
but different tab names so conflict trivial if both merge.
- PR #37659 (my own) adds progress/cancelled/cancel to DailyActivityRange,
but this PR uses stable three-field interface from staging.
* fix(ui): scope spend view by the backend's admin-view contract, not all_admin_roles
Greptile flagged org admin handling on the key savings tab. The live bug it
described does not fire today: useAuthorized supplies session-role labels and
all_admin_roles only carries the raw org_admin spelling, so an org admin was
already scoped. That safety was accidental, so replace the predicate with
spendScopeUserId / hasProxyWideSpendView in utils/roles.ts, mirroring the
backend's user_api_key_has_admin_view (proxy admin and admin viewer only, org
admin excluded in both spellings), and use it in both useDailyActivityRange
and KeySavingsTab
Reclassify the KeySavingsTab render test as an integration test per the
repo's unit/integration split, move scope-resolution coverage to roles.test.ts
as a full role matrix, use real session-role values instead of raw ones, and
assert tile totals against non-empty metrics. Replace the nested ternary in
the chart body (frontend-lint error) with flat conditional rendering
* fix(ui): show auto-router savings as the fourth key-savings tile
Cache hit rate had displaced auto-router savings from the fourth slot,
diverging from the org-wide Cost Optimization page's tile order. Match
it: Total / Compression / Prompt caching / Auto-router, with cache hit
rate as a fifth tile.
* fix(ui): drop cache hit rate from the key savings tiles
Keep the four tiles this page is meant to show: total, compression,
prompt caching, and auto-router savings.
* fix(ui): stop an empty api_key from widening a key-scoped activity read
The paginated and aggregated daily-activity wrappers disagreed on an
empty filter value: the paginated one appended it, the aggregated one
coerced it to undefined with || and dropped it. Since the aggregated
call is the one tried first, an empty key hash would have silently
turned a key-scoped read into a proxy-wide one and reported every
key's savings as this key's. Use ?? so both send the filter through
and it matches nothing instead.
* style(ui): satisfy prettier and the inline-object lint rule in key savings tests
* refactor(ui): drop the cacheHitRatio extraction left over from the removed tile
* fix(ui): pass daily-activity filters raw so both transports agree at the null boundary
* refactor(ui): share the savings tiles and totals between both surfaces
The per-key Savings tab and the proxy-wide Cost Optimization tab carried a byte-identical
four-tile block, three long metric-definition strings included, and five identical useMemo
totals. Both now render SavingsTiles and total through useSavingsTotals, so the donut cannot
slice numbers the tile above it disagrees with.
* docs(ui): say request, not mount, in the savings tab comment
The comment claimed mounting eagerly would fire the rollup sweep, which reads as a claim about
the bundle. Only the request is deferred; the module ships with the key page either way.
* test(ui): pin the daily-activity args array against the real caller signatures
The sibling unit test mocks networking, so it checks the positional array against itself and
stays green when the array and a networking signature drift apart. Swapping user_id and api_key
in the aggregated signature alone passes there and fails here on user_id=hash-abc.
* style(ui): hoist the daily-activity query options out of the call argument
The four-property object literal tripped local/no-large-inline-object-arg. The violation predates
this branch, which only moved the line into the annotated range, and the rule count drops 550 to 549.
* fix(ui): drive auto-router usage from the shared cost-optimization time picker
* fix(ui): extend a live-ending benchmarks range to the current UTC day
Per-model budgets were three separate things pretending to be one. The
enforcement check, the post-call increment and the info endpoints each derived
their own cache key, so a budget could refuse traffic at 429 while /key/info
reported zero usage, and a Bedrock model id never matched a budget keyed on the
bare family name. /user/new echoed a model_max_budget back and stored an empty
dict, and nothing enforced a user-scoped per-model budget at all.
One owner now builds the counter key from the configured budget model, and
enforcement, the increment and the info endpoints all read it. Bedrock ids
resolve through the model-cost map. Auth carries the user's budget onto the
token on every branch that reaches the spend hook, including JWT and
auto-registration. Native passthrough attaches the three budget metadata keys
its StandardLoggingUserAPIKeyMetadata does not carry, so /anthropic/... and
/bedrock/... traffic is counted and capped like /v1/chat/completions.
The dashboard gains the per-model budget editor it never had, on the key create,
key edit and internal-user edit forms. It is read-only without an enterprise
license, matching the write gate the proxy already enforces, and an untouched
budget is left out of an update so an unrelated edit cannot trip that gate.
The editor hydrates from either BudgetConfig spelling, since model_max_budget is
a plain dict that the proxy stores exactly as the client sent it, and it carries
through the fields it does not model. Without both, editing one model would drop
another model row entirely and silently discard its tpm_limit and rpm_limit.
/user/info refreshes its local copy of the user field by field after a save, so
model_max_budget joins that list. Left out, a saved cap read back as the old one
when the form was reopened, and clearing the row to recover would then wipe the
value that had actually persisted.
A zero-dollar cap is the strictest limit expressible, not the absence of one,
so it is enforced rather than skipped on falsiness, spend exactly at the cap is
refused the way every sibling budget check already refuses it, and a counter
that was never written reads as zero spend rather than as unknown. The usage
endpoints read every counter in one batched lookup, so a large model_max_budget
cannot fan out into one concurrent cache call per configured model.
Every auth path honours the same zero-cost skip flag, so none of them can refuse
a free request that another serves. The custom-auth helper gains the flag it
never had, which also changes its pre-existing key and end-user checks.
The compaction summary gate checks the user scope alongside the key and end-user
ones. This file propagates all three budgets into the summary subrequest, so
enforcing only two let compaction increment a counter it could not be refused by.
Custom auth attaches the user's budget to the token unconditionally, since the
post-call spend hook reads it there: gating the attach on the same condition as
enforcement left the counter uncharged whenever the request was not itself
enforceable. An entry that will not validate is skipped rather than raised on,
so one malformed scope cannot abort every other scope's increment or turn a
config typo into a 500.
The edit forms re-seed the budget editor when a different key or user is loaded.
Its rows are seeded once and cannot re-read their own value prop, so without
this a save wrote the previously loaded record's budgets onto the current one.
Only the built-in provider pass-through routes carry the budget metadata.
get_model_from_request deliberately resolves no model for a user-defined
pass-through, since its body is forwarded verbatim and names an upstream model,
so attaching there would charge a counter nothing on that route can refuse.
Resolves the three conflicts against the JSON provider registry refactor. The
hardcoded api.scx.ai base-url branch in get_llm_provider_logic.py is dropped in
favour of the generic JSONProviderRegistry.get_by_base_url lookup, which reads
the same base_url and api_key_env from providers.json and additionally honours
an explicitly passed api_key. constants.py and types/utils.py keep both the
cognition and scx-ai entries added on either side.
team_allowed_routes and admin_allowed_routes only matched exact strings or named route groups, so a whole prefix of pass-through endpoints had to be listed route by route in config. Match trailing-wildcard patterns with the same helper the key-level allowed_routes check uses, so "/prefix/*" covers endpoints registered later.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Model-Specific Limits rows now carry Input TPM and Output TPM, and a
limit the operator removes is sent as an explicitly empty map so
/project/update actually drops it instead of leaving the stored quota
enforced behind a UI that shows it gone.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Cognition serves an OpenAI-compatible /v1/chat/completions endpoint, so it has been onboarded as
custom_llm_provider: openai. That books its traffic as OpenAI, which means OpenAI-specific cost
discounts and provider-level reporting apply to it.
Registers cognition through the JSON provider registry: a providers.json entry with
COGNITION_API_KEY and COGNITION_API_BASE, LlmProviders.COGNITION, the constants.py provider lists,
cost map entries for swe-1.6 and swe-1.7, the provider endpoints matrix, the dashboard provider
fields, and tests. JSON providers can now also be resolved from their base url alone, so an
api_base pointing at a known provider no longer falls through to an unresolved provider.
Entering a custom model name on the Add Model form crashed the whole page
to "This page couldn't load" (React error #185, maximum update depth
exceeded), taking the provider credential fields down with it, so the
model could never be created.
ConditionalPublicModelName kept a `tableKey` counter and bumped it from
an effect on every run to force the mappings table to remount. That was
harmless under antd, whose useWatch handed back the stored array. React
Hook Form's useWatch returns a fresh array each render, so the effect's
dependency changed every render, the effect bumped state again, and the
render loop never settled.
The table is driven by its `data` prop, so the remount counter buys
nothing: drop it, key the effects off the selection contents rather than
the array identity, and write model_mappings only when they actually
change. The two `react-hooks/set-state-in-effect` suppressions on this
file, which were recording exactly this bug, go with it.
The hook swallowed errors into the console, so an admin flipping the switch without
STORE_MODEL_IN_DB saw nothing happen and got no reason why. Adds the missing hook tests.
* fix(ui): surface the paginated fallback on Cost Optimization
The page streamed its fallback silently: useDailyActivityRange dropped
the hook's progress and cancel fields and CacheLeakageCard only showed
a loading state while empty. Extract the Usage page's fetch banner into
a shared PaginationStatusAlerts component, render it above the tabs,
and note on the cache leakage tables when pages are still arriving.
* fix(ui): gate the cache leakage streaming note on isFetchingMore only
loading also covers a fresh aggregated request over the previous
range's rows, where pagination copy mislabels stale data. Drop the
redundant component comment flagged against the repo comment policy.
The Experimental badge sat in the top bar next to the icon, which read as if the
whole theme control were experimental and cost toolbar width for a caveat that
only applies once. It moves into the menu as a Beta tag on the Dark entry, where
it labels exactly the choice it is about and is visible before the choice is made
rather than only after.
* feat(ui): add a light/dark/system theme toggle
The dashboard already carried a full `.dark` palette, dark-aware surfaces and a
dark logo variant, but nothing ever put the `dark` class on the document, so
none of it could be reached. next-themes now owns that class: it reads the
stored choice, falls back to the OS preference, and stamps the class from an
inline script before first paint so there is no light flash on load.
The toggle is a three-way System / Light / Dark control in the account menu,
in both the sidebar menu and the older navbar dropdown, so it is reachable from
the gateway dashboard, chat and the model hub alike.
useIsDarkMode watched the root element with a MutationObserver purely to answer
a question next-themes now answers directly, so it goes, and useSyntaxTheme
reads resolvedTheme instead. The toaster follows the resolved theme too.
* feat(ui): move the theme control to the top bar and default to light
The toggle now lives in the header toolbar of both shells, the gateway
dashboard's DashboardHeader and the older full-width Navbar, where it replaces
the placeholder comment that had been holding its spot. It reads better there
as a single icon button with a System / Light / Dark menu than as a segmented
row buried in the account popover, so the account menus lose their theme row.
Dark mode is still being rolled out, so an install that has never touched the
control now stays light instead of following the OS. System is still a choice,
just no longer the default. While dark is active the toolbar carries a small
Experimental badge, so nobody mistakes an unstyled surface for a bug.
* fix(ui): serve the dark logo in the legacy navbar too
The sidebar already paired its logo with a dark variant, but the full-width
navbar kept a single light-only image. That did not matter while dark mode was
unreachable; now that the toggle sits in that shell's own top bar, the white
JPEG slab lands on a dark bar. It gets the same two-image swap the sidebar uses,
and a test that pins the pairing so the two shells cannot drift apart again.
* feat(ui): let admins supply a dark-mode variant of their custom logo
A deployment branded through UI_LOGO_PATH got its light artwork on the
dark sidebar, and there was nothing an admin could set to change that.
Adds UI_LOGO_PATH_DARK, exposed as the logo_url_dark theme setting and a
second field on the UI theme page. /get_image now walks an ordered list
of candidates for the requested theme and serves the first usable one:
the dark logo, then the light logo, then the bundled default.
Falling through rather than failing is the point. An admin who never
sets a dark logo keeps their own light one instead of reverting to
LiteLLM's, and a dark logo that goes missing later degrades to their
light logo rather than dropping their branding entirely.
* fix(ui): recover from a dark logo the browser cannot load
A dark logo given as an http(s) URL is loaded by the browser straight
from the sidebar, so it never passes through the proxy's fallback chain.
A URL that 404s left a broken image where the admin's light logo should
have been, while the same logo given as a local path fell back cleanly.
The sidebar now remembers the dark URL that failed and drops to the light
logo, matching how the proxy resolves an unusable dark logo and how the
provider Logo component already handles a broken image.
* feat(proxy): add POST /auto_router/validate_config to dry-run the complexity-router write gate
* refactor(proxy): scope the validation endpoint name to the complexity router
* fix(proxy): give the complexity-router validate route the same audience as /model/new
* feat(proxy): gate auto-router dry runs like the write they rehearse
* chore(proxy): dedupe the validate route's self_managed_routes entry
* test(proxy): fold the dry-run route reachability check into the model-new audience parity test
* fix(proxy): scope test_routing's configured check to models the caller can use
* chore(ui): regenerate schema.d.ts for the scoped configured-check description
