Commit graph

4814 commits

Author SHA1 Message Date
tin-berri
31a67561ab
feat(complexity_router): bound the classifier context block, not each turn in it (#38145)
The LLM classifier capped every prior turn at 200 characters independently, so a
785 character turn was cut even when the whole block it belonged to was 353
characters. A character budget now bounds the block: turns are taken newest first
and quoted whole while they fit, older turns are dropped whole once it runs out,
and only the turn straddling the boundary is cut. The per-turn cap stays as an
optional clamp for operators who set it deliberately, defaulting to unset.
2026-08-24 22:56:00 -07:00
Mateo Wang
da91d4b6c9
Merge pull request #38119 from BerriAI/litellm_bing_grounding_search_provider
feat(search): add Grounding with Bing Search (bing_grounding) as a search provider
2026-08-24 19:33:23 -07:00
ryan-crabbe-berri
a5c81e525b refactor(ui): derive drilldown validity and drop Map mutation in bucket grouping
The drilldown now self-dismisses when refetched activity has no failures
for its call_type, instead of holding a selection the chart no longer
shows. groupErrorBuckets is rewritten as pure filter/map/sort over the
already-grouped SQL rows.
2026-08-24 17:56:22 -07:00
ryan-crabbe-berri
0de0e1cef6 feat(ui): add error-code drilldown for failed requests on caching page
/global/activity/cache_hits now returns an error_breakdown: failed spend
logs bucketed per call_type by error code and error class, read from
metadata->error_information. Clicking a red failed-requests segment on
the cache activity chart opens a per-code bar chart; hovering a bar
lists the error classes behind that code.
2026-08-24 15:45:45 -07:00
yuneng-jiang
6147b3ce6e
refactor(ui): install the shadcn field primitive (#38126)
* refactor(ui): install the shadcn field primitive

`components/shared/form/field.tsx` was the upstream base-vega `field` source
living outside `components/ui/`. It exported the same ten symbols as upstream,
so `npx shadcn add` could never update it and it had already drifted: its
`FieldLabel` was missing the hover and focus-visible ring utilities upstream
now ships for labels that wrap a nested field.

Install the primitive and point the 77 importers at it. The copy is deleted
rather than kept as a wrapper because it added nothing beyond `forwardRef`,
which React 19 makes unnecessary since `ref` arrives as an ordinary prop.

`field.test.tsx` moves next to the primitive with no edits to its contents,
and its nineteen tests, ref assertions included, pass against the generated
file. That is the evidence the swap is behaviour-preserving.

Two nested-field call sites pick up the upstream hover and focus-visible
styling that the stale copy had been missing.

(cherry picked from commit 947f7fa674c83bfc57f43ad8bfc89c894da947a2)

* test(ui): cover the nested-field interaction cues FieldLabel had lost

The stale copy of `field` was missing the hover, focus-visible and disabled
selectors upstream applies to a label that wraps a nested field, so installing
the primitive restored them with nothing asserting they stay.

Assert the class contract rather than the rendered effect. jsdom evaluates
neither `:has()` nor `:focus-visible`, and Tailwind is not compiled under
vitest, so a test that clicked or tabbed would pass on an element with no
styling at all. Checking the utilities are present is the assertion that
actually fails when they go missing, which is the way they were lost before.

Verified by stripping the four selectors from the primitive: both tests fail,
and both pass once it is restored.

(cherry picked from commit 5a5dbf64270d9d1285dbc4a7af76bb3d927778a8)
2026-08-24 14:30:32 -07:00
Mateo Wang
0f596a5145
Merge pull request #38129 from BerriAI/litellm_autorouter_dropdown_registry
fix(auto-router): list configured auto-routers in the usage picker before they have traffic
2026-08-24 14:23:09 -07:00
yuneng-jiang
47c988e05c
refactor(ui): move the dashboard onto class-variance-authority (#38125)
The dashboard used `cva@1.0.0-beta.4` with the object-argument API behind
`@/lib/cva.config`, while shadcn emits `class-variance-authority` with the
positional API. Every `shadcn add` of a cva-based primitive therefore needed a
hand fix-up before it compiled, which meant `components/ui/` could never match
a fresh CLI run and `shadcn add <name> --diff` reported the whole file as
changed instead of showing real upstream drift.

Swap the dependency, and regenerate `badge`, `button`, `button-group`,
`input-group` and `tabs` straight from the base-vega registry so they are now
byte-identical to the CLI output plus prettier.

Two primitives could not be regenerated because they are local code rather
than registry items, so they move out of `components/ui/`: `sidebar` (203
lines against upstream's 730, and only `leftnav` consumes it) and `meter`
(no registry entry at all, it wraps Base UI's Meter).

The customisations that were baked into the regenerated files move to
wrappers, following the rule that `components/ui/` holds CLI output and
anything on top of it lives outside:

- badge carried info, success and warning variants that duplicated the
  existing `StatusBadge` tone map, so its five call sites now use
  `StatusBadge`, which gains an optional `className`
- input-group's addon focuses `[data-slot=input-group-control]` rather than
  upstream's `input`, which matters because the chat composer puts a textarea
  there. That handler now sits at the one call site that needs it

`cx` keeps its previous twMerge behaviour. It came from the old
`defineConfig({hooks: {onComplete: twMerge}})`, and CVA's own `cx` is plain
clsx, so pointing it at `cn` avoids silently dropping conflict resolution in
the six files that use it.

`Sidebar.test.tsx` covers the failure mode this migration can hide: passing
the object form to the positional API is accepted by clsx and renders the
literal class string "base variants defaultVariants", so the component loses
every style while the type checker and the existing suite stay green.
2026-08-24 14:10:31 -07:00
tin-berri
85d5ac2b5c
feat(ui): add Gemini Family auto-router preset (#38138)
Adds the `gemini_family` bundled template to the auto-router tab, a
heuristic-classifier preset alongside the existing Anthropic and OpenAI
family presets.

Tiers ascend in cost across the Gemini lineup:
  SIMPLE     gemini-2.5-flash-lite    $0.10 / $0.40
  MEDIUM     gemini-3.1-flash-lite    $0.25 / $1.50
  COMPLEX    gemini-3.7-flash         $0.75 / $3.75
  REASONING  gemini-3.1-pro-preview   $2.00 / $12.00

Uses concrete model ids rather than Google's `gemini-*-latest` aliases.
Those aliases hot-swap to the newest release of their variation (stable,
preview or experimental) with only a two-week notice, while their rows in
model_prices_and_context_window.json are pinned at 2.5-generation rates,
so a swap onto a 3.x model would bill at the stale price and silently
undercount auto-router spend. A pin test asserts no tier resolves to a
`-latest` alias and that all four rungs are distinct.
2026-08-24 13:28:45 -07:00
Mateo Wang
d0da90ee6d
Merge pull request #38115 from BerriAI/litellm_fix_runwayml_video_provider
fix(runwayml): route every generation endpoint and fix video cost tracking
2026-08-24 12:35:59 -07:00
Tin Chi Lo
26e47aea32 fix(auto-router): list configured auto-routers in the usage picker before they have traffic 2026-08-24 15:28:20 -04:00
devin-ai-integration[bot]
a1134755ca
fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp (#37982)
* fix(ui): boot the UI image as an arbitrary uid by anchoring nginx writes under /tmp

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): type the arbitrary-uid image test fixture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-24 11:57:36 -07:00
mateo-berri
671a454baa Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bing_grounding_search_provider 2026-08-24 11:53:45 -07:00
Mateo Wang
a626170c89
Merge pull request #36806 from BerriAI/litellm_bedrock_converse_no_trailing_empty_chunk
fix(bedrock): stop emitting an empty assistant delta after the finish_reason chunk
2026-08-24 11:37:04 -07:00
mateo-berri
6407a66375 fix(runwayml): route every generation endpoint and fix video cost tracking
Six defects in the RunwayML video provider:

- transform_video_create_request hardcoded /image_to_video, so text-to-video 400'd and video-to-video was unreachable; the endpoint is now selected from the inputs present (promptVideo/videoUri, promptImage, or text only)
- get_error_class raised instead of returning, turning a provider 4xx into a proxy 500 APIConnectionError; it now returns a RunwayMLError
- VideoObject.progress was typed int while Runway sends a 0..1 float, 500'ing status polls while RUNNING; it is now scaled to a 0..100 percent
- custom per-deployment pricing stored under litellm_metadata was ignored for video; the deployment model_info lookup now checks both metadata keys
- stale cost-map entries (gen3a_turbo, gen4_aleph) were removed and current models added, with output_cost_per_second_480p/_4k tier keys plumbed through the model-info and router types
- video cost now falls back to Runway's estimatedCost from the create response when no custom pricing is configured, and custom pricing always wins over it

Fixes #36483
2026-08-24 11:23:03 -07:00
Mateo Wang
9bcc00b1f1
Merge pull request #33310 from BerriAI/litellm_google_interactions_cost
fix(interactions): track cost and spend for Google Interactions API requests
2026-08-24 11:18:58 -07:00
mateo-berri
03a676995a feat(search): add Grounding with Bing Search (bing_grounding) as a search provider 2026-08-24 11:08:24 -07:00
mateo-berri
e0511e9384 Merge branch 'litellm_internal_staging' into litellm_bedrock_converse_no_trailing_empty_chunk
Resolves the test-file conflict by keeping both sides, extends the
finish-reason gate to trace-bearing metadata events so guardrail trace
chunks keep their pre-regression delta shape, parametrizes the
regression test over tool-call, mixed, and reasoning streams, and
repairs the one ant-design icon usage the lucide-react migration left
behind in skill_detail.tsx (semantic conflict on the base branch)
2026-08-24 10:37:32 -07:00
ryan-crabbe-berri
4913b2a3ca
Merge pull request #33514 from ozolam/litellm_fix_skills_marketplace_commands_v2
fix(UI): correct skill install command and marketplace setup UX
2026-08-24 10:17:25 -07:00
yuneng-jiang
3fb1009f81
fix(ui): make playground chat bubbles theme-aware (#37978)
The playground message bubble painted its fill, border and avatar circle from
inline hex values, so in dark mode both bubbles stayed near-white while the text
inherited the dark foreground: the message body was unreadable. The MCP-events
placeholder bubble in ChatUI carried the same three fills.

They move onto the tokens the rest of the sweep already uses, so the assistant
surface is bg-card over border-border and the user surface is the info tint at
the same weight the other selected-state surfaces take. Light mode keeps the
same colour family it had.

The regression test asserts the token classes and that no inline style survives
on either surface, which is the exact shape the bug took.
2026-08-24 10:13:24 -07:00
yuneng-jiang
7113685a76
fix(ui): repoint the key detail URL to the rotated hash after regenerating (#37968)
Regenerating a key from the key info page left the ?key= query param on the
old hash, so dismissing the dialog or reloading landed on a key that no longer
exists and the page rendered "Key not found".

Two defects had to line up. POST /key/{key}/regenerate returns the rotated
hash in token_id and leaves token null, but RegenerateKeyModal read
response.token || response.key_id, neither of which the endpoint populates, so
it always reported the old hash back to its parent. And KeyInfoView's
onKeyDataUpdate prop had no caller anywhere in the tree: VirtualKeysTable owns
the ?key= param and mounts the view but never passed it, so even a correct
hash went nowhere.

VirtualKeysTable now handles the update by pointing ?key= at the rotated hash
and refetching. KeyInfoView holds that callback until the regenerate dialog is
dismissed rather than firing it on the API response, because swapping the
selected key mid-dialog unmounts the view and tears down the one-time
plaintext key before the user can copy it.
2026-08-24 10:12:55 -07:00
yuneng-jiang
5b1c142c6e
fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited (#37916)
* fix(ui): render team and org tpm/rpm limits of 0 as 0 instead of Unlimited

A tpm_limit or rpm_limit of 0 is a hard block on the backend (every request 429s) and only null means unlimited, but the team and organization views rendered both as "Unlimited" (and a team-member limit of 0 as "No Limit") because every display site used a falsy || fallback. The team member edit dialog also seeded its form with `tpm_limit || null`, so opening Edit Member on a member stored with 0 and clicking Save sent null to /team/member_update and silently turned the hard block into unlimited

Every limit display site in TeamInfo, organization_view, the organizations list cell and the team members table now uses a nullish check, and both member form seeding paths keep 0 for max_budget_in_team, tpm_limit and rpm_limit. Regression tests cover each site and the existing memberFormValues test that asserted 0 -> null is flipped to assert 0 survives

Resolves LIT-5760

* test(ui): assert a stored 0 member limit survives an untouched save

The EditMembership integration test named the old 0 -> null collapse as the expected payload, so the related-tests CI job went red once the form kept 0. It now asserts 0 survives and only the empty budget_duration collapses to null. The TeamMemberTab fixture is built with a map instead of mutating the nested membership
2026-08-24 10:12:52 -07:00
yuneng-jiang
6db5a5d660
fix(ui): restore the public model name tooltip layout in the add model flow (#37986)
The tooltip popup is an inline-flex row, so the four sibling blocks passed as a fragment laid out side by side in four columns. Wrap them in a single flex-col container instead.

The inline code samples also used bg-muted, which is defined against the page surface, not the inverted tooltip surface, so they rendered as near-white chips carrying near-white text. Tint them from the popup's own token instead.
2026-08-24 10:12:46 -07:00
yuneng-jiang
5f56be3294
fix(ui): theme the created-key box so it follows dark mode (#37985)
The virtual key shown after creating a key sits in a div with a
hardcoded #f8f8f8 inline background, so in dark mode the box keeps
the light background while the key text inherits the light foreground
color, leaving the key nearly unreadable. Swap the inline styles for
the bg-muted and text-foreground tokens, which resolve per theme.
2026-08-24 10:12:16 -07:00
yucheng-berri
d447be15b9
feat(newrelic): per-team New Relic trace routing via team callbacks (#37603)
Some checks failed
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-08-22 19:13:48 -07:00
yuneng-jiang
1c421f3578
fix(ui): keep completion-mode models in the playground chat dropdown (#37954)
PR #36130 added a KNOWN_MODEL_MODES guard to isModelCompatibleWithEndpoint
that hides any model whose mode isn't in the ModelMode enum, to keep
rerank/ocr/batch/etc. models out of chat-style endpoints. mode: completion
(legacy text-completion models) wasn't in that enum, so it got caught by
the same guard and disappeared from every endpoint, including chat, where
it routes fine.

Add ModelMode.COMPLETION and map it to EndpointType.CHAT like the other
chat-compatible modes.
2026-08-22 11:36:24 -07:00
mateo-berri
9906770e41 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_google_interactions_cost
# Conflicts:
#	litellm/constants.py
#	litellm/interactions/main.py
#	litellm/litellm_core_utils/litellm_logging.py
#	litellm/litellm_core_utils/llm_cost_calc/usage_object_transformation.py
#	litellm/proxy/hooks/proxy_track_cost_callback.py
#	litellm/proxy/management_endpoints/credential_migration.py
#	tests/test_litellm/litellm_core_utils/test_litellm_logging.py
#	tests/test_litellm/test_cost_calculator.py
#	ui/litellm-dashboard/src/lib/http/schema.d.ts
2026-08-22 10:39:45 -07:00
mateo-berri
138b0da21f chore(dashboard): regenerate api types for the files list params 2026-08-21 17:33:30 -07:00
Mateo Wang
3029f7eb84
Merge pull request #34752 from SouthernCrossAI/litellm_scx_ai_provider
feat(providers): add SCX.ai as a JSON-configured OpenAI-compatible provider
2026-08-21 17:21:46 -07:00
tin-berri
0c50286a55
feat(ui): add per-key Savings tab to key detail page (#37693)
* feat(ui): add per-key Savings tab to key detail page

Adds a "Savings" tab to the key detail view, showing the same four metrics
and time-series chart as the proxy-wide Cost Optimization view, but scoped
to a single API key.

For org admins, the tab shows the key's full savings across all requests.
Non-admins see only their own requests on the key, with a scope note
explaining the limitation.

Root cause: userDailyActivityCall and userDailyActivityAggregatedCall
never forwarded an api_key query parameter to the backend, even though
both handlers already accept and filter by it.

Changes:

- networking.tsx: Add optional apiKey param to both daily activity call
  wrappers (appended to variadic options tuple for backward compatibility).

- costOptimizationUtils.ts: Extract shared metrics helpers (compressionOf,
  cachingOf, autorouterOf, savedTokensOf, cacheHitRatio) and shortDate
  so both UsageTab and KeySavingsTab use the same formulas and prevent
  divergence.

- useDailyActivityRange.ts: Refactor into useScopedDailyActivityRange(
  accessToken, scope: {userId, apiKey?}) for reuse-by-parameter unbundling.
  Role resolution stays at the entry point (useDailyActivityRange), not in
  a scoped caller. Update test expectations for new 6-arg tuple.

- UsageTab.tsx: Simplify by importing extracted helpers and SummaryCard
  component instead of defining them inline. No behavioral change.

- key_info_view.tsx: Insert "Savings" tab trigger between "Overview" and
  "Settings"; wire TabsContent to new KeySavingsTab component with lazy
  mounting (no keepMounted) to defer daily-activity fetch until tab opened.

- NEW: components/shared/SummaryCard.tsx — Shared presenter for four-tile
  summary row (label + value + hint + optional info popover). Extracted
  from UsageTab so both surfaces show identical tile layout without CSS
  divergence.

- NEW: components/templates/KeySavingsTab.tsx — Per-key view with admin/
  non-admin scope branching, empty-state messaging, same chart toggles
  and info popovers as UsageTab.

- NEW: components/templates/KeySavingsTab.test.tsx — 7 tests covering mount,
  loading state, empty state, scoping, and scope-note visibility.

Authorization: No new permission check. Both backends gate api_key filter
by the same user role check that governs the request itself. Non-admins
must send their own user_id and can only see their own keys.

Tests: 6121 pass (1 pre-existing failure unrelated to this change).

Prior art / collision note:
- PR #37570 (budgets tab) lands in same TabsList hunks as "Savings" tab,
  but different tab names so conflict trivial if both merge.
- PR #37659 (my own) adds progress/cancelled/cancel to DailyActivityRange,
  but this PR uses stable three-field interface from staging.

* fix(ui): scope spend view by the backend's admin-view contract, not all_admin_roles

Greptile flagged org admin handling on the key savings tab. The live bug it
described does not fire today: useAuthorized supplies session-role labels and
all_admin_roles only carries the raw org_admin spelling, so an org admin was
already scoped. That safety was accidental, so replace the predicate with
spendScopeUserId / hasProxyWideSpendView in utils/roles.ts, mirroring the
backend's user_api_key_has_admin_view (proxy admin and admin viewer only, org
admin excluded in both spellings), and use it in both useDailyActivityRange
and KeySavingsTab

Reclassify the KeySavingsTab render test as an integration test per the
repo's unit/integration split, move scope-resolution coverage to roles.test.ts
as a full role matrix, use real session-role values instead of raw ones, and
assert tile totals against non-empty metrics. Replace the nested ternary in
the chart body (frontend-lint error) with flat conditional rendering

* fix(ui): show auto-router savings as the fourth key-savings tile

Cache hit rate had displaced auto-router savings from the fourth slot,
diverging from the org-wide Cost Optimization page's tile order. Match
it: Total / Compression / Prompt caching / Auto-router, with cache hit
rate as a fifth tile.

* fix(ui): drop cache hit rate from the key savings tiles

Keep the four tiles this page is meant to show: total, compression,
prompt caching, and auto-router savings.

* fix(ui): stop an empty api_key from widening a key-scoped activity read

The paginated and aggregated daily-activity wrappers disagreed on an
empty filter value: the paginated one appended it, the aggregated one
coerced it to undefined with || and dropped it. Since the aggregated
call is the one tried first, an empty key hash would have silently
turned a key-scoped read into a proxy-wide one and reported every
key's savings as this key's. Use ?? so both send the filter through
and it matches nothing instead.

* style(ui): satisfy prettier and the inline-object lint rule in key savings tests

* refactor(ui): drop the cacheHitRatio extraction left over from the removed tile

* fix(ui): pass daily-activity filters raw so both transports agree at the null boundary

* refactor(ui): share the savings tiles and totals between both surfaces

The per-key Savings tab and the proxy-wide Cost Optimization tab carried a byte-identical
four-tile block, three long metric-definition strings included, and five identical useMemo
totals. Both now render SavingsTiles and total through useSavingsTotals, so the donut cannot
slice numbers the tile above it disagrees with.

* docs(ui): say request, not mount, in the savings tab comment

The comment claimed mounting eagerly would fire the rollup sweep, which reads as a claim about
the bundle. Only the request is deferred; the module ships with the key page either way.

* test(ui): pin the daily-activity args array against the real caller signatures

The sibling unit test mocks networking, so it checks the positional array against itself and
stays green when the array and a networking signature drift apart. Swapping user_id and api_key
in the aggregated signature alone passes there and fails here on user_id=hash-abc.

* style(ui): hoist the daily-activity query options out of the call argument

The four-property object literal tripped local/no-large-inline-object-arg. The violation predates
this branch, which only moved the line into the annotated range, and the rule count drops 550 to 549.
2026-08-21 14:50:02 -07:00
tin-berri
9821b451e3
fix(ui): drive auto-router usage from the shared cost-optimization time picker (#37871)
* fix(ui): drive auto-router usage from the shared cost-optimization time picker

* fix(ui): extend a live-ending benchmarks range to the current UTC day
2026-08-21 13:49:08 -07:00
Yassin Kortam
7da34e8aed
fix(proxy): make per-model budgets track spend, enforce, and report the same counter (#37736)
Per-model budgets were three separate things pretending to be one. The
enforcement check, the post-call increment and the info endpoints each derived
their own cache key, so a budget could refuse traffic at 429 while /key/info
reported zero usage, and a Bedrock model id never matched a budget keyed on the
bare family name. /user/new echoed a model_max_budget back and stored an empty
dict, and nothing enforced a user-scoped per-model budget at all.

One owner now builds the counter key from the configured budget model, and
enforcement, the increment and the info endpoints all read it. Bedrock ids
resolve through the model-cost map. Auth carries the user's budget onto the
token on every branch that reaches the spend hook, including JWT and
auto-registration. Native passthrough attaches the three budget metadata keys
its StandardLoggingUserAPIKeyMetadata does not carry, so /anthropic/... and
/bedrock/... traffic is counted and capped like /v1/chat/completions.

The dashboard gains the per-model budget editor it never had, on the key create,
key edit and internal-user edit forms. It is read-only without an enterprise
license, matching the write gate the proxy already enforces, and an untouched
budget is left out of an update so an unrelated edit cannot trip that gate.

The editor hydrates from either BudgetConfig spelling, since model_max_budget is
a plain dict that the proxy stores exactly as the client sent it, and it carries
through the fields it does not model. Without both, editing one model would drop
another model row entirely and silently discard its tpm_limit and rpm_limit.

/user/info refreshes its local copy of the user field by field after a save, so
model_max_budget joins that list. Left out, a saved cap read back as the old one
when the form was reopened, and clearing the row to recover would then wipe the
value that had actually persisted.

A zero-dollar cap is the strictest limit expressible, not the absence of one,
so it is enforced rather than skipped on falsiness, spend exactly at the cap is
refused the way every sibling budget check already refuses it, and a counter
that was never written reads as zero spend rather than as unknown. The usage
endpoints read every counter in one batched lookup, so a large model_max_budget
cannot fan out into one concurrent cache call per configured model.

Every auth path honours the same zero-cost skip flag, so none of them can refuse
a free request that another serves. The custom-auth helper gains the flag it
never had, which also changes its pre-existing key and end-user checks.

The compaction summary gate checks the user scope alongside the key and end-user
ones. This file propagates all three budgets into the summary subrequest, so
enforcing only two let compaction increment a counter it could not be refused by.

Custom auth attaches the user's budget to the token unconditionally, since the
post-call spend hook reads it there: gating the attach on the same condition as
enforcement left the counter uncharged whenever the request was not itself
enforceable. An entry that will not validate is skipped rather than raised on,
so one malformed scope cannot abort every other scope's increment or turn a
config typo into a 500.

The edit forms re-seed the budget editor when a different key or user is loaded.
Its rows are seeded once and cannot re-read their own value prop, so without
this a save wrote the previously loaded record's budgets onto the current one.

Only the built-in provider pass-through routes carry the budget metadata.
get_model_from_request deliberately resolves no model for a user-defined
pass-through, since its body is forwarded verbatim and names an upstream model,
so attaching there would charge a counter nothing on that route can refuse.
2026-08-21 09:47:52 -07:00
bhuvan2134686
aef09aca18 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_scx_ai_provider
Resolves the three conflicts against the JSON provider registry refactor. The
hardcoded api.scx.ai base-url branch in get_llm_provider_logic.py is dropped in
favour of the generic JSONProviderRegistry.get_by_base_url lookup, which reads
the same base_url and api_key_env from providers.json and additionally honours
an explicitly passed api_key. constants.py and types/utils.py keep both the
cognition and scx-ai entries added on either side.
2026-08-21 17:27:49 +10:00
Mateo Wang
66a6a09706
Merge pull request #37743 from BerriAI/litellm_cognition_provider_identity
feat(cognition): give Cognition its own provider identity
2026-08-20 18:38:23 -07:00
devin-ai-integration[bot]
c74e9e75f9
feat(ui): support project input and output TPM limits (#37676)
The Model-Specific Limits rows now carry Input TPM and Output TPM, and a
limit the operator removes is sent as an explicitly empty map so
/project/update actually drops it instead of leaving the stored quota
enforced behind a UI that shows it gone.

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-20 17:43:52 -07:00
Mateo Wang
02e67cd715
Merge pull request #35181 from BerriAI/litellm_block_unpriced_models
feat(proxy): add admin toggle to block requests for models without pricing
2026-08-20 17:32:28 -07:00
mateo-berri
e00301703f feat(cognition): give Cognition its own provider identity
Cognition serves an OpenAI-compatible /v1/chat/completions endpoint, so it has been onboarded as
custom_llm_provider: openai. That books its traffic as OpenAI, which means OpenAI-specific cost
discounts and provider-level reporting apply to it.

Registers cognition through the JSON provider registry: a providers.json entry with
COGNITION_API_KEY and COGNITION_API_BASE, LlmProviders.COGNITION, the constants.py provider lists,
cost map entries for swe-1.6 and swe-1.7, the provider endpoints matrix, the dashboard provider
fields, and tests. JSON providers can now also be resolved from their base url alone, so an
api_base pointing at a known provider no longer falls through to an unresolved provider.
2026-08-20 17:16:53 -07:00
yuneng-jiang
cb89c7aa8f
fix(ui): stop the Add Model mapping table from looping the page (#37741)
Entering a custom model name on the Add Model form crashed the whole page
to "This page couldn't load" (React error #185, maximum update depth
exceeded), taking the provider credential fields down with it, so the
model could never be created.

ConditionalPublicModelName kept a `tableKey` counter and bumped it from
an effect on every run to force the mappings table to remount. That was
harmless under antd, whose useWatch handed back the stored array. React
Hook Form's useWatch returns a fresh array each render, so the effect's
dependency changed every render, the effect bumped state again, and the
render loop never settled.

The table is driven by its `data` prop, so the remount counter buys
nothing: drop it, key the effects off the selection contents rather than
the array identity, and write model_mappings only when they actually
change. The two `react-hooks/set-state-in-effect` suppressions on this
file, which were recording exactly this bug, go with it.
2026-08-20 16:42:49 -07:00
devin-ai-integration[bot]
22e8b45c68
feat(proxy): add maximum_health_check_retention_period to bound the health-check table (#37681)
* feat(proxy): add health check retention cleanup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): drop redundant health-check assertion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): share cleanup budget across retention groups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): clarify cleanup group deadlines

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-20 16:01:16 -07:00
mateo-berri
3c44f8d926 fix(ui): surface toggle failures on the block-unpriced-models setting
The hook swallowed errors into the console, so an admin flipping the switch without
STORE_MODEL_IN_DB saw nothing happen and got no reason why. Adds the missing hook tests.
2026-08-20 15:47:13 -07:00
tin-berri
cb4eb82249
feat(ui): per-model reasoning effort in the complexity tier editor (#37673)
* feat(ui): per-model reasoning effort in the complexity tier editor

* feat(ui): gate the effort control on model group reasoning support
2026-08-20 15:10:35 -07:00
tin-berri
2dcd453860
feat(shadow_eval)!: gate the per-key budget on dollar spend instead of turns (#37555) 2026-08-20 14:55:21 -07:00
tin-berri
60e03bedcf
fix(ui): surface the paginated fallback on Cost Optimization (#37659)
* fix(ui): surface the paginated fallback on Cost Optimization

The page streamed its fallback silently: useDailyActivityRange dropped
the hook's progress and cancel fields and CacheLeakageCard only showed
a loading state while empty. Extract the Usage page's fetch banner into
a shared PaginationStatusAlerts component, render it above the tabs,
and note on the cache leakage tables when pages are still arriving.

* fix(ui): gate the cache leakage streaming note on isFetchingMore only

loading also covers a fresh aggregated request over the previous
range's rows, where pagination copy mislabels stale data. Drop the
redundant component comment flagged against the repo comment policy.
2026-08-20 14:55:00 -07:00
mateo-berri
5301872093 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_block_unpriced_models 2026-08-20 14:42:03 -07:00
github-actions[bot]
6bd21fad70 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_block_unpriced_models
# Conflicts:
#	litellm/proxy/auth/auth_checks.py
#	tests/test_litellm/proxy/auth/test_auth_checks.py
#	tests/test_litellm/proxy/management_endpoints/test_cost_tracking_settings.py
#	ui/litellm-dashboard/src/app/(dashboard)/cost-tracking/_components/cost_tracking_settings.tsx
#	ui/litellm-dashboard/src/lib/http/schema.d.ts
2026-08-20 13:43:12 -07:00
ryan-crabbe-berri
787edb123f
refactor(ui): mark dark as beta in the theme menu, not the toolbar (#37680)
The Experimental badge sat in the top bar next to the icon, which read as if the
whole theme control were experimental and cost toolbar width for a caveat that
only applies once. It moves into the menu as a Beta tag on the Dark entry, where
it labels exactly the choice it is about and is visible before the choice is made
rather than only after.
2026-08-20 13:29:43 -07:00
ryan-crabbe-berri
933e28d900
feat(ui): add a light/dark/system theme toggle to the top bar (#37669)
* feat(ui): add a light/dark/system theme toggle

The dashboard already carried a full `.dark` palette, dark-aware surfaces and a
dark logo variant, but nothing ever put the `dark` class on the document, so
none of it could be reached. next-themes now owns that class: it reads the
stored choice, falls back to the OS preference, and stamps the class from an
inline script before first paint so there is no light flash on load.

The toggle is a three-way System / Light / Dark control in the account menu,
in both the sidebar menu and the older navbar dropdown, so it is reachable from
the gateway dashboard, chat and the model hub alike.

useIsDarkMode watched the root element with a MutationObserver purely to answer
a question next-themes now answers directly, so it goes, and useSyntaxTheme
reads resolvedTheme instead. The toaster follows the resolved theme too.

* feat(ui): move the theme control to the top bar and default to light

The toggle now lives in the header toolbar of both shells, the gateway
dashboard's DashboardHeader and the older full-width Navbar, where it replaces
the placeholder comment that had been holding its spot. It reads better there
as a single icon button with a System / Light / Dark menu than as a segmented
row buried in the account popover, so the account menus lose their theme row.

Dark mode is still being rolled out, so an install that has never touched the
control now stays light instead of following the OS. System is still a choice,
just no longer the default. While dark is active the toolbar carries a small
Experimental badge, so nobody mistakes an unstyled surface for a bug.

* fix(ui): serve the dark logo in the legacy navbar too

The sidebar already paired its logo with a dark variant, but the full-width
navbar kept a single light-only image. That did not matter while dark mode was
unreachable; now that the toggle sits in that shell's own top bar, the white
JPEG slab lands on a dark bar. It gets the same two-image swap the sidebar uses,
and a test that pins the pairing so the two shells cannot drift apart again.
2026-08-20 12:58:29 -07:00
yuneng-jiang
122675c309
feat(ui): let admins supply a dark-mode variant of their custom logo (#37662)
* feat(ui): let admins supply a dark-mode variant of their custom logo

A deployment branded through UI_LOGO_PATH got its light artwork on the
dark sidebar, and there was nothing an admin could set to change that.

Adds UI_LOGO_PATH_DARK, exposed as the logo_url_dark theme setting and a
second field on the UI theme page. /get_image now walks an ordered list
of candidates for the requested theme and serves the first usable one:
the dark logo, then the light logo, then the bundled default.

Falling through rather than failing is the point. An admin who never
sets a dark logo keeps their own light one instead of reverting to
LiteLLM's, and a dark logo that goes missing later degrades to their
light logo rather than dropping their branding entirely.

* fix(ui): recover from a dark logo the browser cannot load

A dark logo given as an http(s) URL is loaded by the browser straight
from the sidebar, so it never passes through the proxy's fallback chain.
A URL that 404s left a broken image where the admin's light logo should
have been, while the same logo given as a local path fell back cleanly.

The sidebar now remembers the dark URL that failed and drops to the light
logo, matching how the proxy resolves an unusable dark logo and how the
provider Logo component already handles a broken image.
2026-08-20 12:48:55 -07:00
tin-berri
135f234e89
feat(proxy): add POST /auto_router/validate_complexity_router_config to dry-run the complexity-router write gate (#37409)
* feat(proxy): add POST /auto_router/validate_config to dry-run the complexity-router write gate

* refactor(proxy): scope the validation endpoint name to the complexity router

* fix(proxy): give the complexity-router validate route the same audience as /model/new

* feat(proxy): gate auto-router dry runs like the write they rehearse

* chore(proxy): dedupe the validate route's self_managed_routes entry

* test(proxy): fold the dry-run route reachability check into the model-new audience parity test

* fix(proxy): scope test_routing's configured check to models the caller can use

* chore(ui): regenerate schema.d.ts for the scoped configured-check description
2026-08-20 19:48:32 +00:00
tin-berri
79cac36564
fix(ui): keep keyword tier rules that target operator-defined tiers when hydrating the edit modal (#37413) 2026-08-20 12:34:38 -07:00
yuneng-jiang
edbb3429a3
feat(ui): serve a dark-mode variant of the LiteLLM logo (#37656)
The bundled logo is a JPEG, so it carries no alpha and its white
background renders as a bright slab against a dark sidebar. Making it
transparent alone would not be enough either: the wordmark is near-black
and would disappear on dark.

Adds logo_dark.png, derived from the light logo. The sky-blue disc and
train are kept as they are behind a circular alpha mask, and the
wordmark's antialiasing is un-flattened from white into straight alpha
and repainted in the dark theme's own foreground colour. Both files are
1000x257, so swapping between them cannot shift the sidebar header.

/get_image gains a theme query param. The default response is byte for
byte what it was, and a logo configured through UI_LOGO_PATH is served
unchanged in both themes, since custom logos have no dark variant yet.
2026-08-20 11:26:10 -07:00