Commit graph

5084 commits

Author SHA1 Message Date
tin-berri
e35ee4e5fa
feat(router): independent, default-on deployment affinity for the auto-router (#36146) 2026-08-08 13:02:29 -07:00
Yuneng Jiang
1a40a67394
fix: stabilize generated user role ordering 2026-08-08 12:54:23 -07:00
mubashir1osmani
3725233736 Merge remote-tracking branch 'berri/litellm_internal_staging' into litellm_playground_shadcn
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
# Conflicts:
#	ui/litellm-dashboard/src/components/llm_calls/fetch_models.tsx
2026-08-08 12:33:13 -07:00
devin-ai-integration[bot]
cfd64d45a8
fix(ui): show team BYOK models in team fallback settings (#36241)
* fix(ui): show team BYOK models in team fallback settings

Team router settings loaded fallback options from /model_group/info, which resolves models without a team, so a team's own BYOK deployments were never selectable in its own fallback config. Load the team-scoped listing when a team id is present.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): ignore stale team model responses in router settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): use react-query for fallback model listing in router settings accordion

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-08-08 19:28:57 +00:00
mubashir1osmani
f2e3b6a568 fix(ui): drop null guardrail names from MultiSelect options 2026-08-08 12:25:41 -07:00
Yuneng Jiang
0d7f7c689a
test: repair stale CircleCI contracts 2026-08-08 12:19:29 -07:00
mubashir1osmani
757d5a4dcd fix(ui): apply coy theme after code props on ReasoningContent 2026-08-08 12:19:17 -07:00
mubashir1osmani
4aff515a8e fix(ui): cast syntax highlighter theme through unknown 2026-08-08 12:15:09 -07:00
mubashir1osmani
38e1fe8ffd fix(ui): extract chat handlers and type MCP tool pool 2026-08-08 12:11:33 -07:00
mubashir1osmani
01d55b9d0f fix(ui): guard null voice selection from shadcn Select 2026-08-08 12:07:36 -07:00
mubashir1osmani
d78a016bc5 fix(ui): green playground shadcn CI after staging merge
Normalize MCP description nullability for MultiSelect, preserve model
selection via functional setState, and update endpoint/vector-store
selector tests for the shadcn combobox API
2026-08-08 12:01:44 -07:00
mubashir1osmani
0302283eba chore: merge litellm_internal_staging into playground shadcn stack 2026-08-08 11:56:47 -07:00
mateo
f668c10609 chore(ui): regenerate dashboard api types for ModelInfo pricing fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 03:25:09 +00:00
mateo
96a8b7f488 chore(ui): regenerate dashboard api types for tier_turns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 02:09:53 +00:00
ryan-crabbe-berri
2a9aac7004
fix(ui): let access groups be a team's only model source, with hover provenance (#36234)
* feat(proxy): return per-group model provenance on /team/info

/team/info now carries access_group_details, one entry per resolved access
group with its id, name, and model list, so the UI can attribute each
inherited model to the group granting it. The batch resolver returns the
access group rows keyed by id instead of a stringly dict of lists, and the
team member budget helper returns a copy instead of mutating its parameter.
Type discipline and basedpyright budgets ratchet down accordingly.

* feat(ui): allow group-only teams and show model provenance on hover

Team create and edit no longer require a model selection: an empty
selection is saved as the no-default-models sentinel, never as a bare
empty list, since an empty team model list means unrestricted access.
The team info Models card now renders every badge with a hover tooltip
naming how the team got that model: directly, via named access groups,
or both, and group-granted badges stay visible when the direct list is
empty or a sentinel.

* refactor(proxy): dedupe access group ids and return copies instead of mutating

Duplicate access_group_ids no longer amplify the /team/info response: ids
collapse order-preserving before provenance is built, pinned by a regression
test. The resolver returns a model_copy rather than mutating its parameter,
and the team create call sends a new object instead of reassigning
formValues.models. Budgets ratchet down further with the mutation removal.
2026-08-07 17:45:50 -07:00
ryan-crabbe-berri
8db2fbaad0
feat(ui): show user email or alias in usage data export (#36232)
* feat(ui): resolve user email/alias in usage export instead of raw user id

* test(ui): cover email/alias resolution in usage export data builders

* chore(ui): drop explanatory comment per repo comment policy
2026-08-07 22:53:59 +00:00
mateo
2f70f4edba build(deps): bump nanoid to 3.3.17 in the dashboard lockfile
GHSA-2v37-7h3g-55p8 (CVE-2026-67213) rates 8.2 against nanoid 3.3.16 and reds osv-scan on every PR into staging. Custom generators can loop indefinitely when size is zero, so a caller that passes through a zero size hangs the process.

nanoid is transitive through the dashboard's toolchain and 3.3.17 is a patch release that published 08-03, so it already clears the .npmrc min-release-age=3 guard. The lock diff is the version, resolved url, and integrity hash for that one package.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-07 21:30:09 +00:00
mateo
493eb9054e chore(ui): regenerate schema.d.ts for the /key/info docstring update
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-07 19:13:12 +00:00
ryan-crabbe-berri
527dc0a8bb
feat(proxy): add apply_user_budget_to_team_keys opt-in (#36102)
* feat(proxy): add apply_user_budget_to_team_keys opt-in

PR #32005 made a user's personal max_budget apply to their team-scoped keys
too, and PR #35271 reverted the whole thing (behavior plus the
skip_user_budget_on_team_key opt-out) because that flipped the default for
everyone. This brings the behavior back the other way round: default is
unchanged, and general_settings.apply_user_budget_to_team_keys opts a
deployment into charging the key owner's personal budget on team keys.

The flag reaches all three personal-budget gates so an opted-in deployment
enforces consistently: the read-time check in common_checks, the optimistic
reservation counter in _get_budget_counters, and the _PROXY_MaxBudgetLimiter
pre-call hook. It is also in the /config/list allowed args and, unlike the
reverted flag, in the _update_general_settings propagation allowlist, so the
Admin UI General Settings toggle actually takes effect at runtime; an explicit
YAML value still wins over the DB value on reload.

get_config_list's allowed_args moves to a module-level frozen mapping of
field name to type string, dropping 18 LIT002 violations and rebuilding one
less dict per request.

* style(proxy): drop explanatory comments from the budget flag paths
2026-08-07 15:40:13 +00:00
LHMQ878
55b52970ea fix(proxy): preserve OpenAI WS query params and provider auth
Forward realtime model query string, keep OPENAI_API_KEY (forward_headers=False),
satisfy ruff strict gates, sync dashboard OpenAPI types, and cover the behavior in tests.
2026-08-07 11:09:27 +08:00
yuneng-jiang
4e5495e1bd
Merge pull request #36147 from BerriAI/litellm_bump_js_yaml_4_3_1
build(deps): bump h2 to 4.4.1 and js-yaml to 4.3.1
2026-08-06 19:10:07 -07:00
ryan-crabbe-berri
429a5dc430
fix(ui): allow clearing a key's budget reset from the Edit Key form (#36140) 2026-08-06 18:46:28 -07:00
bhuvan2134686
1e24f93d39 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_scx_ai_provider 2026-08-07 11:26:05 +10:00
bhuvan2134686
8aa9d3dfe5 feat(models): swap SCX.ai catalog to GLM-5.2 and Qwen3.8 Max
Replaces the five launch models with the two that SCX.ai now leads on.
Both are live on api.scx.ai and both were verified against it for tool
calling, json_object and json_schema output, reasoning, prompt caching,
and, for Qwen3.8 Max, image input

Pricing follows SCX's published USD rates. GLM-5.2 lands at $0.55/M input
and $1.9255/M output, tracking the recent GLM-5.2 market repricing;
Qwen3.8 Max at $1.815/M and $5.4461/M sits under the only other seller of
that model, and is the first Qwen3.8 Max entry in the catalog

Also corrects a metadata bug the removed entries carried: they set
max_tokens equal to max_input_tokens, conflating the context window with
the output cap. Both new entries declare a max_output_tokens of 131072,
which is what the endpoint's own validator enforces

The Add Model placeholder moves to scx-ai/GLM-5.2 now that MiniMax-M2.7
is no longer in the catalog
2026-08-07 11:22:01 +10:00
Yuneng Jiang
0253154780
build(deps-dev): bump js-yaml to 4.3.1
Closes GHSA-5p4m-2wfm-xmqj (CVSS 7.5), flagged by osv-scan against
ui/litellm-dashboard/package-lock.json. js-yaml is pinned by an exact
npm override, so the override and the lock move together.

Dev-only dependency: js-yaml reaches the tree through eslintrc, knip
and @redocly/openapi-core, none of which ship in the built dashboard.

4.3.1 published 2026-07-31, clear of the 3-day min-release-age cooldown.
2026-08-06 18:14:26 -07:00
tin-berri
7da891a42a
fix(ui): match auto-router preset models against wildcard-expanded model groups (#36111) 2026-08-06 17:47:05 -07:00
mubashir1osmani
5102b9c0d8 fix(ui): ignore stale playground model loads on key switch
Cancel in-flight model fetches when the key or source changes so an older
response cannot overwrite modelInfo. Drop the inverted endpoint-filter
assertion; filtering coverage lands in the next stack PR
2026-08-06 15:33:30 -07:00
mubashir1osmani
bb5d9a199a feat(ui): migrate ChatUI off Ant Design and Tremor
Replace ChatUI cards, inputs, dialogs, popovers, MCP selects, uploads,
tooltips, and icons with shadcn/Base UI and Lucide. Update ChatUI tests
to drive searchable combobox controls instead of Ant Design selectors
2026-08-06 14:33:39 -07:00
mubashir1osmani
ea654d11a5 feat(ui): migrate playground tabs and message icons to shadcn
Replace Tremor playground page tabs with Base UI tabs and swap remaining
message bubble and attachment renderer icons to Lucide
2026-08-06 14:26:15 -07:00
mubashir1osmani
f5d98c0b8c feat(ui): migrate playground chat controls toward shadcn
Continue the Playground Chat Ant Design/Tremor migration: shared MultiSelect,
upload validation with semantic file inputs, collapsible message widgets, and
AdditionalModelSettings on Base UI controls
2026-08-06 14:25:11 -07:00
Mateo Wang
0acca3e86a
Merge pull request #24548 from mpcusack-altos/fix/bedrock-batch-credential-fields
fix(router): include Bedrock batch/S3 fields and model in deployment credentials
2026-08-05 22:39:39 -07:00
tin-berri
34fc8d2ee7
fix: expired-miss share over all measured turns + cost-optimization tab labels (#36037)
* fix(ui): make the expired-miss stat row a focusable tooltip trigger

* fix: auto-router expired-miss percentage and cost-optimization tab labels

- change expired-miss percentage denominator from return-to-tier misses to
  all measured turns (same_model + first_visit + return_to_tier). when
  auto-routers flip tiers rapidly within TTL, return-to-tier turns become
  hits and disappear from the miss count; the old metric reported only the
  rare failure population. the new metric contextualizes that population as
  a share of overall coverage
- rename usage tab from 'Usage' to 'Overall'
- rename auto-router-usage tab from 'Auto-Router Usage' to 'Auto-Router'
- update component and unit tests to match new semantics
2026-08-05 22:34:55 -07:00
tin-berri
86890654c5
fix(proxy): include today's UTC bucket when a daily activity range ends at the caller's current day (#36051)
* fix(proxy): include today's UTC bucket when a daily activity range ends at the caller's current day

* fix(proxy): gate the current-UTC-day extension behind an opt-in param sent by the cost optimization dashboard

* fix(ui): label cost optimization savings dates as UTC days
2026-08-05 22:33:54 -07:00
Michael Cusack
3d275d97fe fix(router): return model and Bedrock batch fields in deployment credentials
get_deployment_credentials_with_provider dropped s3_region_name,
s3_encryption_key_id, and aws_batch_role_arn because
CredentialLiteLLMParams never declared them, and it never returned the
deployment's model, so proxy batch creation against Bedrock failed with
"LiteLLM doesn't support custom_llm_provider=bedrock for 'create_batch'"
or "AWS IAM role ARN is required" (#25104)

Provider-only file and batch calls keep their no-model contract:
get_team_provider_credentials strips the model key so a provider-scoped
request is not pinned to an arbitrary matching deployment
2026-08-05 21:49:31 -07:00
Mateo Wang
ba91768146
Merge pull request #35925 from BerriAI/litellm_tier_aware_reasoning_token_cost
fix(cost): bill reasoning tokens at the service tier output rate
2026-08-05 21:09:55 -07:00
tin-berri
7c621b3141
fix(auto-router): accept every reminder marker pair a harness emits (#36029)
* fix(auto-router): accept every reminder marker pair a harness emits

reminder_markers held one (open, close) pair, so a harness that wraps
injected context differently per agent type only got the slice of traffic
using the configured envelope stripped. Every other agent type kept hitting
the original bug: its reminder-only turn never stripped to empty, won
"newest human ask", and the harness blob got classified in place of the
real question, choosing the tier and therefore the spend.

The field now takes a list of ReminderMarkerPair, following the
KeywordTierRule pattern already in this file so each pair validates itself
and errors point at reminder_markers.N.close rather than a bare index.

Blocks from different pairs can nest, which the gap construction could not
handle: resuming the kept text at an inner block's end walks back inside
the enclosing block and leaks its remainder. Running the block ends through
a maximum collapses nested and overlapping spans without a separate merge
pass, and stays linear in block count, which a fold over a growing tuple
of merged spans would not.

A single pair's ends already increase, so the maximum is the identity and
the default path is byte-identical: verified against the shipped function
over 200k generated inputs, and every existing reminder test passes
unchanged. The prior single-pair config shape is rejected loudly at
startup and at /model/new rather than silently stripping nothing.

* docs(auto-router): document reminder_markers in the complexity router README

* chore(ui): regenerate dashboard API types for the reminder_markers shape

---------

Co-authored-by: Abhimanyu Kapur <38531241+akapur99@users.noreply.github.com>
2026-08-05 21:03:36 -07:00
mateo-berri
1b30b1bc20 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_tier_aware_reasoning_token_cost
# Conflicts:
#	litellm/types/utils.py
2026-08-05 20:45:34 -07:00
tin-berri
ece652f6a7
feat(ui): add the auto-router usage tab to cost optimization (#35995) 2026-08-06 02:15:35 +00:00
Yuneng Jiang
6e434c926b
Merge branch 'litellm_internal_staging' into litellm_dead_locals_5_8 2026-08-05 18:07:23 -07:00
Yuneng Jiang
ce5c4c1bf9
refactor(ui): drop dead locals and unused React state across the dashboard
Removes declarations nothing reads, along with the writes that fed them, so
the remaining code says what it actually does.

Where a declaration was dead but its initializer had a real effect, the call
survives and only the binding goes: spies stay installed, renders still run,
and every awaited request keeps its await. Pure computations are deleted
whole rather than left as statements that build a value and throw it away.

Dead useState pairs are removed outright instead of being elided to
const [, setX], which would keep a hook and every write to a value nothing
reads. Three chains turned out to be dead end to end and are removed with
their fetches: the tool detail team list, the Teams MCP access group load,
and the user dashboard proxy settings load.

ColumnMeta's declaration merging in columnMeta.ts and view_logs/table.tsx is
a false positive; TypeScript requires those type parameters to match the
upstream signature exactly, so both get a scoped suppression instead.
2026-08-05 18:07:17 -07:00
yuneng-jiang
624fa11d71
Merge pull request #36025 from BerriAI/litellm_dead_locals_3_tests_and_destructures
refactor(ui): drop unreferenced locals from tests and narrow destructures
2026-08-05 17:54:12 -07:00
Yuneng Jiang
888f911133
refactor(ui): drop unreferenced locals from tests and narrow destructures
Third and fourth slices of the sweep, combined because they raise nearly the
same question and neither changes what runs.

Nine test files plus one source file lose symbols whose only mention was
their own declaration. Ten more narrow a destructure to the keys actually
read, so `const { accessToken, userRole, userId: userID, premiumUser } =
useAuthorized()` keeps only `accessToken`. Aliases are preserved as written.

ignoreRestSiblings stays on so the omit idiom `const { tags, ...rest } =
metadata` is left alone; dropping `tags` there would fold it back into rest.

ToolDetail is held back again. Its unread binding only looks like a plain
deletion on the first pass, because the dead useMemo still reads it; one more
pass exposes a useQuery that issues a real request. That belongs with the
slices that get QA'd.

Part of LIT-5162.
2026-08-05 17:46:39 -07:00
yuneng-jiang
c2c795fad5
Merge pull request #35821 from BerriAI/litellm_dead_locals_2_components
refactor(ui): drop unreferenced locals from shared dashboard components
2026-08-05 17:44:38 -07:00
ryan-crabbe-berri
f2690aa60e
fix(ui): opening a project now pushes ?project= so back and deep links work (#36001)
* fix(ui): drive project detail selection from the ?project= url param

Opening a project kept selectedProjectId in useState, so the URL never changed; the detail view could not be linked or reloaded and browser Back skipped past the Projects page entirely.

Selection now lives in the ?project= query param via nuqs with history: push, matching how Teams, Organizations and Virtual Keys already work.

* fix(ui): project detail close replaces history to match the other detail pages

Adopts the close semantics from PR #36013 so browser Back after an
in-page close leaves the Projects page instead of reopening the
dismissed detail; the close test now pins the replace mode
2026-08-05 17:31:22 -07:00
Yuneng Jiang
80627c0477
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_dead_locals_2_components
# Conflicts:
#	ui/litellm-dashboard/src/components/add_model/add_auto_router_tab.tsx
2026-08-05 17:29:54 -07:00
yuneng-jiang
1b059f472d
Merge pull request #35819 from BerriAI/litellm_dead_locals_1_app_routes
refactor(ui): drop unreferenced locals from dashboard route components
2026-08-05 17:28:15 -07:00
ryan-crabbe-berri
1dab133d24
fix(ui): link project page keys to their virtual key detail (#36002) 2026-08-05 17:26:13 -07:00
ryan-crabbe-berri
6a2e4e6c36
fix(ui): sync projects list page index to ?page= so back and reload keep the page (#36003)
* fix(ui): sync projects list page index to ?page= so back and reload keep the page

Paging the Projects list only moved TanStack's internal page index, so the URL
never changed: reload dropped you on page 1, browser Back left the page entirely,
and the page could not be shared.

The page index now comes from a nuqs ?page= query state with history: "push".
Pagination stays controlled off that value and the footer writes the URL
directly, because TanStack resets its page index whenever the data array
identity changes; letting it own the state would clear a deep-linked page as
soon as the projects query resolved. A page outside the current row set falls
back to page 1, which covers both a hand-typed ?page=99 and a search that
narrows the list below the current page.

* fix(ui): carry page_size in the url so restored history entries show the same rows

Greptile flagged that a history entry restoring ?page=N under a changed
local page size displays different projects than it originally showed.
Page size now rides the same query string via useQueryStates, size
changes reset the page inside a single history entry, and values outside
the offered options fall back to the default
2026-08-05 17:25:42 -07:00
tin-berri
55e666a05f
feat(complexity_router): report LLM classifier cost per request via routing_decision and x-litellm-classifier-cost header (#36015) 2026-08-05 16:27:32 -07:00
tin-berri
265945dfcd
feat(ui): match auto-router preset models against deployments' underlying model IDs (#35972)
The bundled presets only became selectable when an admin's public model_group
names matched the preset's hardcoded model names. model_name is admin-arbitrary,
so renamed deployments (my-claude-fast, bedrock-opus) left both presets greyed
out. Resolve preset models against each deployment's litellm_params.model and
model_info.base_model from /v2/model/info via a normalized ID join, and prefill
the admin's registered group names. Resolves LIT-5225
2026-08-05 15:24:54 -07:00