Commit graph

4874 commits

Author SHA1 Message Date
Yuneng Jiang
c5c98cf706
test(ui): characterise GuardrailsOverview before the shadcn migration
Covers the header and export action, all five summary metric cards, the
table toolbar heading, the evaluation settings modal wiring, the busy
state and the request failure message. Every assertion is role, title or
text based so it holds against both the antd markup and its shadcn
replacement, letting the migration commit land without editing this file
2026-08-13 14:56:58 -07:00
Yuneng Jiang
28032eb4c9
test(ui): decouple usage-route tests from antd and Tremor markup
Rewrites the usage route's library-coupled assertions to role and text
queries so they characterise behaviour rather than the widget library that
happens to render it. Every test here is green against the current antd and
Tremor components and is meant to survive a migration unedited.

Drops the wholesale vi.mock of antd, @ant-design/icons and @tremor/react in
UsagePageView, UsageViewSelect and activity_metrics, and drives the real
controls instead. Two of those mocks were hiding behaviour: activity_metrics
asserted panel order through a heading role that only existed because the
mock faked an h2, and the Tremor tab stubs flattened panel selection away
entirely.

Replaces DOM-shape lookups with anchors that do not move: a chart is found
from its own heading rather than a fixed wrapper depth, an active panel is
identified by the inactive markers both tab libraries set, and select options
are matched by text since antd's real options carry no option role.

Adds the missing characterisation test for team_multi_select, and covers the
mount contracts the route depends on: Tremor keeps every tab panel mounted,
and antd Collapse keeps a section mounted once it has been expanded, which is
what preserves the view-mode state a model section owns.
2026-08-13 14:23:26 -07:00
mubashir1osmani
2a7f5f270c Merge remote-tracking branch 'berri/litellm_internal_staging' into litellm_playground_shadcn
# Conflicts:
#	ui/litellm-dashboard/eslint-suppressions.json
2026-08-13 13:54:11 -07:00
Yuneng Jiang
8ada34fe2c
test(ui): characterise guardrails route behaviour before shadcn migration
Rewrite the PatternModal test off antd class selectors onto role and text
queries, and add characterisation tests for the guardrails components that
had none: the keyword modal, the content filter display and configuration,
the guardrail garden and the custom code modal.

Also covers two behaviours the migration must preserve: the action and
severity dropdowns in the content filter tables, and test playground state
surviving a tab switch away and back.

Every assertion here passes against the current antd and Tremor components
so the same file can prove the shadcn versions unedited.
2026-08-13 13:49:46 -07:00
Yuneng Jiang
1dc0ea3d11
Merge branch 'litellm_internal_staging' into litellm_shadcn_logs_drawer_header_0813 2026-08-13 13:33:00 -07:00
Yuneng Jiang
9f947a7406
Merge branch 'litellm_internal_staging' into litellm_/quirky-mcnulty-e432c4 2026-08-13 13:23:51 -07:00
Yuneng Jiang
2f7602b028
Merge branch 'litellm_internal_staging' into litellm_/eloquent-wu-3ab1d5 2026-08-13 13:16:48 -07:00
yuneng-jiang
c9b543dbfe
Merge pull request #36737 from BerriAI/litellm_/gifted-colden-355244
refactor(ui): migrate SimpleMessageBlock and SimpleToolCallBlock to shadcn
2026-08-13 13:16:22 -07:00
tin-berri
d8fda675cc
feat: pre-adoption shadow eval for the auto-router (blind pairwise judge, derived state) (#36587) 2026-08-13 13:15:45 -07:00
Yuneng Jiang
b71e7a3213
Merge branch 'litellm_internal_staging' into litellm_/gifted-colden-355244 2026-08-13 13:07:51 -07:00
mubashir1osmani
f3f8c48b1d style(ui): format new playground tests and drop suppressions this stack fixed
Prettier flagged the two test files added while fixing review findings.
Migrating these components off Ant Design also retired the lint suppressions
they carried, so prune those 15 entries and leave the unrelated ones for the
PRs that made them stale.
2026-08-13 12:55:43 -07:00
yuneng-jiang
b8577516d4
Merge branch 'litellm_internal_staging' into litellm_/quirky-mcnulty-e432c4 2026-08-13 12:51:53 -07:00
yuneng-jiang
21dbd3381e
Merge branch 'litellm_internal_staging' into litellm_/eloquent-wu-3ab1d5 2026-08-13 12:51:51 -07:00
yuneng-jiang
229d258664
Merge branch 'litellm_internal_staging' into litellm_/gifted-colden-355244 2026-08-13 12:51:50 -07:00
yuneng-jiang
6f9ee6e28f
Merge branch 'litellm_internal_staging' into litellm_/interesting-meitner-e88370 2026-08-13 12:51:48 -07:00
Yuneng Jiang
81f324dc15
Merge branch 'litellm_internal_staging' of github.com:BerriAI/litellm into litellm_shadcn_logs_drawer_header_0813 2026-08-13 12:51:37 -07:00
Yuneng Jiang
dfdafbf89b
fix(ui): keep the tools panel mounted so a tool's expanded detail survives
antd's Collapse kept the panel mounted once opened, so a tool a user had
expanded stayed expanded after closing and reopening Tools. Base UI renders
only the open branch, so the migration silently reset every ToolItem.

The regression test passes against the antd original, fails against the
migration without keepMounted, and passes with it.
2026-08-13 12:50:56 -07:00
Yuneng Jiang
b4092f476f
test(ui): find the section copy button by role instead of the antd icon
InputCard and OutputCard located SectionHeader's copy button by querying for a
descendant with aria-label="copy", which is the antd CopyOutlined icon. That
selector reaches into SectionHeader's internals, so migrating it off antd left
copyButton undefined and failed four tests.

getByRole("button", { name: /copy/i }) is green against both the antd and the
shadcn SectionHeader, verified by running these two files against each.
2026-08-13 12:47:45 -07:00
mubashir1osmani
8fd97803ce Merge remote-tracking branch 'berri/litellm_playground_shadcn' into litellm_playground_shadcn 2026-08-13 12:44:01 -07:00
mubashir1osmani
f4fb4e0bea fix(ui): label the virtual key source and unstick a cancelled model load
The key source trigger rendered the stored value, so the playground showed
session and custom instead of Current UI Session and Virtual Key. Name the
selected option on the trigger.

Clearing the key while models were loading left the selector disabled for
good: the in-flight load skips its reset once cancelled, and the branch that
handles an empty key returned without clearing the loading flag, so nothing
put it back. Clear it on that path too.
2026-08-13 12:43:25 -07:00
mubashir1osmani
3fdacfa6f9
fix(ui): restore playground model filtering by endpoint (#36130)
* fix(ui): restore playground model filtering by endpoint

Bring back the prior Chat model dropdown filter (including chat models
on responses/anthropic/interactions and image models on image_edits), and
map mode realtime so the realtime endpoint only lists compatible models

* fix(ui): exclude unknown model modes from playground endpoint filters

Modes outside ModelMode (batch, rerank, ocr, etc.) must not collapse to
chat-compatible, or conversational endpoints surface unusable models

* feat(ui): add shared vercel-style playground chat composer (#36131)

* feat(ui): adopt vercel-style chat composer for playground

Replace the compact single-line input with a PromptInput-style composer:
taller auto-growing textarea, rounded card shell, footer tools, and
stop button while a request is in flight

* style(ui): strengthen playground chat composer border and shadow

Make the shared chat input stand out with a fuller border, layered
shadow, and a slightly stronger focus ring

* fix(ui): size chat composer textarea with CSS field-sizing

Drop direct el.style.height mutation in favor of field-sizing:content

* fix(ui): keep the chat composer out of a nested form and focus its textarea

The composer wrapped everything in a native form, so MCP mode nested Ant
Design's tool-arguments form inside it, which is invalid HTML and let Enter
hit either form. The footer also relied on InputGroupAddon focusing the first
input in the group, which is the hidden file input from the attach controls
rather than the message textarea.

Drop the outer form and submit from the send button directly, and have the
addon focus the element marked as the group's control.

* refactor(ui): reuse the endpoint compatibility check when a model is picked

The endpoint guard added upstream duplicated the compatibility families this
PR introduces, so point it at isModelCompatibleWithEndpoint instead. Filtering
also means an incompatible model is no longer offered for an endpoint, so the
test that picked one now asserts it is absent.

* fix(ui): match the image-edit model mode the backend actually sends

model_prices_and_context_window.json labels these models image_edit, but the
mode enum spelled it image_edits, so once unknown modes started being filtered
out every image-edit model vanished from the playground, /v1/images/edits
included. The endpoint key keeps its own spelling.

The compatibility tests stubbed getEndpointType with a hand-written map that
repeated the same wrong spelling, which is how this stayed hidden, so they now
run against the real mapping.
2026-08-13 12:40:42 -07:00
mubashir1osmani
26194b85a4 fix(ui): keep the playground endpoint, editable number fields, and drag-to-upload
Picking a model reset the endpoint from its mode unconditionally, so choosing
a chat model while on /v1/responses, /v1/messages or interactions bounced the
playground to /v1/chat/completions. Only switch when the current endpoint
cannot serve the picked model.

The temperature and max-token boxes parsed and clamped on every keystroke, so
a decimal lost its point and clearing the field snapped to a bound. They are
now text fields with a numeric input mode that hold what was typed and clamp
on blur; the sliders beside them still give the stepped control.

The image-edit and transcription areas invited a drag but had no drop
handlers after the Ant Design Dragger came out, so drops did nothing. Wire
drop through the same validation the file picker uses.
2026-08-13 12:26:50 -07:00
tin-berri
5f2986a1f3
feat(complexity_router): calibrate the classifier rubric with worked examples, selectable per router (#36578)
* feat(complexity_router): calibrate the classifier rubric with worked examples

The built-in rubric stated its tier boundaries as prose alone, and prose
calibrated to consumer chat puts "non-trivial code, multi-step technical work"
at the top of the scale. That is the median request in developer and agent
traffic, so ordinary engineering read as top-tier and the router paid for the
most expensive model on it.

Adds calibration examples to the rubric, selected by a new
classifier_llm_config.rubric preset. The agentic preset (now the default)
anchors routine installs, builds, multi-file edits, and standard debugging at
MEDIUM; the chat preset omits those anchors for deployments serving only
conversational traffic. Both share the same tier criteria, the trust-boundary
paragraph, and the context-window closing line, so this moves where the
boundary sits without changing the taxonomy.

Both presets render byte-identical to the strings a prompt sweep scored, and a
test pins that, so the measured accuracy describes what a router sends.

* feat(ui): pick the classifier rubric preset on an auto-router

Adds a Rubric dropdown to the auto-router's classification panel, so the
agentic and chat presets are selectable rather than config-file only. The
prompt editor prefills from the selected preset, since prefilling agentic text
for a router on chat would show examples its classifier never receives.

The picker is disabled while a custom prompt is set, and the payload builder
drops the preset in that case: a custom prompt is the classifier's whole system
role, so the backend rejects the two together. The builder records the default
preset explicitly, so a later change to which preset is default cannot silently
move an existing router.

* fix(complexity_router): mark an unchosen rubric preset with None, not model_fields_set

The mutual-exclusion check read model_fields_set to tell an explicit preset
from the default. That flag does not survive serialization, and this config is
dumped and handed straight back to ComplexityRouter by /auto_router/test_routing,
where a dump re-states every field. So a custom-prompt classifier saved fine and
then failed validation on preview, rejecting on the second pass what it accepted
on the first.

The preset is now optional, with None meaning the default, matching how None
already means the built-in rubric for system_prompt on the same model. The
default lives in one place, DEFAULT_RUBRIC_PRESET, resolved where the prompt is
assembled. The dashboard stops sending a copy of the default it displays, so a
router nobody configured follows the default rather than pinning today's value,
and UI-built routers behave the same as hand-written config.

Regenerates schema.d.ts, which was left stale by an earlier description edit.

* feat(complexity_router): grandfather existing routers onto the uncalibrated rubric

An unset preset now means LEGACY, the rubric exactly as it shipped before
calibration examples existed, so upgrading cannot move the tier decisions or the
bill of a router that is already running. Config-file routers get this for free
since they name no preset, and a stored config that never had one reads the same
way.

New routers still get the calibrated rubric: switching a classifier to LLM
stamps the agentic preset, because a classifier being configured for the first
time has no prior tier behaviour to preserve. The picker offers legacy so an
existing router's state is representable and opening the form cannot silently
upgrade it.

Each preset is pinned byte-identical to the text the prompt sweep scored,
legacy included, which is what proves an existing router's prompt did not move.

Also collapses the preset data from a NamedTuple with group wrappers and
per-preset frozensets into plain text blocks in a MappingProxyType, matching how
the tier criteria next to it are already stored: 21 lines of prompt text no
longer cost 190 lines of constructors. Tiers are format placeholders so
tier_labels still reach the examples.

* refactor(complexity_router): name the field classification_rubric

`rubric` alone did not say what it selects, and the field sits beside
`system_prompt`, which genuinely is the whole classification prompt. The name
now says which of the two an operator is reaching for: the rubric the built-in
prompt is assembled from, not the prompt itself.

Renames the config field, the query param, the enum, and the dashboard label to
match, and moves the preset text to classification_rubrics.py.

* test(ui): set the preset the mutual-exclusion case is meant to drop

The rename left classification_classification_rubric in the custom-prompt case,
so its input never carried a preset and the assertion held for the wrong reason:
it proved an absent preset stays absent, not that a set one is dropped. A
normalizer that forwards the preset whenever one is set passed with the typo and
fails without it.

tsc reports the typo as TS2353; the earlier sweep grepped for the source file
and not the test, so it went unseen.

* test(ui): scope the role-gate assertions to each page's own endpoint

The memory, workflows, and guardrails-monitor page tests asserted that a denied
role fires no request at all. Their names, and the assertion on the very next
line, say the intent is narrower: the page must not fetch its own data.

Resolving whether a caller is an org admin goes through /organization/list for
every role, since deciding org-admin-for-any-org needs the list, and the route
scopes rows per caller. That legitimate request fails a blanket no-fetch
assertion, so all three files went red on staging for a reason unrelated to
what they test.

Drops the blanket assertion and keeps the scoped one. Bypassing the gate in
memory/page.tsx still fails five tests, so the narrower assertion continues to
catch a genuinely broken gate.

* fix(complexity_router): document that an unset rubric keeps the legacy prompt

The field said 'Leave unset for agentic' while an omitted rubric resolves to
LEGACY, so the OpenAPI schema an operator reads promised calibrated routing
where they got the uncalibrated one.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 12:22:20 -07:00
mubashir1osmani
136ef345da Merge remote-tracking branch 'berri/litellm_internal_staging' into litellm_playground_shadcn
# Conflicts:
#	ui/litellm-dashboard/src/app/(dashboard)/playground/components/chat_ui/ChatUI.test.tsx
#	ui/litellm-dashboard/src/app/(dashboard)/playground/components/chat_ui/ChatUI.tsx
#	ui/litellm-dashboard/src/components/policies/PolicySelector.tsx
2026-08-13 11:28:42 -07:00
Daniel Meismer
80f49024a3 chore(ui): bump nanoid to 3.3.18
Update the transitive lockfile entry to the first patched 3.x release so OSV no longer reports GHSA-2v37-7h3g-55p8.

Generated with AI

Co-Authored-By: Codex
2026-08-13 13:14:17 -04:00
Daniel Meismer
d794b61347 chore(ui): bump nanoid to 3.3.18
Update the transitive lockfile entry to the first patched 3.x release so OSV no longer reports GHSA-2v37-7h3g-55p8.

Generated with AI

Co-Authored-By: Codex
2026-08-13 13:09:47 -04:00
Yuneng Jiang
bc9005c149
refactor(ui): migrate SectionHeader and ToolsSection to shadcn 2026-08-13 10:04:38 -07:00
Devin AI
630a1d0b69 build(deps): bump nanoid to 3.3.18 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 16:38:24 +00:00
Yuneng Jiang
eb23dc2e81
test(ui): pin SectionHeader and ToolsSection behaviour before migration 2026-08-13 09:37:43 -07:00
yuneng-jiang
9d069f21dc
refactor(ui): migrate guardrails content tables to shared DataTable (#36708)
* test(ui): characterize guardrails content tables

* refactor(ui): migrate guardrails content tables to shared DataTable

* test(ui): verify shared guardrails table integration
2026-08-13 09:21:00 -07:00
yuneng-jiang
b8241d6664
refactor(ui): migrate guardrails monitor table to shared DataTable (#36709)
* test(ui): characterize guardrails monitor table

* refactor(ui): migrate guardrails monitor table to shared DataTable
2026-08-13 09:20:48 -07:00
yuneng-jiang
bfa34e2b10
refactor(ui): migrate usage tables to shared DataTable (#36707)
* test(ui): decouple usage table test from antd

* refactor(ui): migrate usage tables to shared DataTable

* test(ui): preserve data utility exports in usage tests
2026-08-13 09:20:45 -07:00
Daniel Meismer
19eae00d71 fix(ui): make per-user usage filter searchable
Reuse the Global Usage user search and pagination behavior in the Per User report, including empty-result handling.

Co-Authored-By: Codex
2026-08-13 12:16:36 -04:00
Daniel Meismer
297fe272ec feat: scope request log user filter
Add a bounded spend-log user facet for the Request Logs picker and
intersect explicit user filters with the caller's own and permitted-team
scope.

Co-Authored-By: Codex
2026-08-13 11:50:43 -04:00
Daniel Meismer
151bdbb2a9 test(ui): cover user filter pagination
Co-Authored-By: Codex
2026-08-13 11:21:48 -04:00
Daniel Meismer
c019ce53e3 feat(ui): add user ID request log filter
Co-Authored-By: Codex
2026-08-13 11:15:45 -04:00
yassin
6346497498 fix(ui): add nvidia riva to the model provider list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 13:01:40 +00:00
yuneng-jiang
3d76dfc72e
refactor(ui): migrate search and user controls to shadcn (#36694)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate search and user controls

* fix(ui): restore search tool clear action
2026-08-13 06:43:30 +00:00
yuneng-jiang
fd00b98f64
refactor(ui): migrate guardrails-monitor, projects, logs to shadcn (#34606)
* test(ui): pin behaviour of guardrails-monitor, projects and logs components before migration

Adds role- and text-based characterisation tests for EvaluationSettingsModal,
GuardrailDetail and AuditLogDrawer, which had none, and moves the remaining
antd-specific assertions (.ant-spin, the icon role of an antd Spin indicator)
onto library-neutral ARIA queries. Also covers the enterprise banner on the
deleted keys and deleted teams pages, which no test reached.

All of these pass against the current antd and Tremor components.

* refactor(ui): migrate guardrails-monitor, projects and logs to shadcn

Replaces antd and Tremor with installed shadcn primitives across the files
these three routes exclusively own. Markup only, except where noted below.

Deletes AntDLoadingSpinner, an antd-only primitive living in the shadcn ui/
folder, and moves its single call site onto ui/ui-loading-spinner.

Two behaviour notes. The logs tab handler previously mapped every tab past
the first to "audit logs", so the audit panel kept polling while Deleted
Keys or Deleted Teams was on screen; each tab now reports its own value and
panels stay mounted via keepMounted. The evaluation settings dialog is
bounded to the viewport and scrolls internally, which the antd Modal got
from being top-anchored on a scrolling page.

The tests added in the previous commit pass unedited against these
components.
2026-08-13 06:43:12 +00:00
yuneng-jiang
4df421e058
refactor(ui): migrate guardrail and duration controls to shadcn (#36693)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate guardrail and duration controls

* fix(ui): preserve duration select callback shape

* fix(ui): narrow duration selection value
2026-08-13 06:42:42 +00:00
yuneng-jiang
e619106306
refactor(ui): migrate team detail controls to shadcn (#36695)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate team detail controls
2026-08-13 06:41:28 +00:00
Yuneng Jiang
266a35b88c
test(ui): cover keyboard activation of the log drawer collapse rows
The migration turned each collapse row into a real button, but the existing
tests only click, so a regression in Enter or Space activation would still
pass. Add one test per component that tabs to the row, expands with Enter and
collapses with Space, asserting visibility rather than markup.

Both fail against the antd version and pass against the migrated one.
2026-08-12 22:39:29 -07:00
Yuneng Jiang
f17143086a
refactor(ui): migrate TruncatedValue and OutputCard to shadcn
TruncatedValue swaps antd Tooltip and Typography for the shadcn Tooltip
and the shared CopyButton, so the full value now lives in the page and is
clipped with CSS rather than shortened in the text node.

OutputCard drops antd Typography for token-styled markup and folds its two
duplicated branches into one. Its border and the drawer's mono font stay on
the module's existing constants so the card still matches InputCard, which
is out of scope here.

Both files retire their no-restricted-imports suppression.
2026-08-12 22:29:57 -07:00
Yuneng Jiang
9355953414
refactor(ui): migrate SimpleMessageBlock and SimpleToolCallBlock to shadcn
Replace antd Typography and inline hex styling in the log drawer's message
and tool-call blocks with plain elements and design tokens. Both components'
existing tests pass unedited before and after.

Retires their no-restricted-imports entries from the eslint suppressions
baseline.
2026-08-12 22:27:20 -07:00
Yuneng Jiang
c25f927f03
test(ui): make the TokenFlow grouping assertion locale-independent
The assertion hard-coded en-US separators while toLocaleString follows the
host locale, so it failed under de_DE. Building the expected string the same
way keeps it deterministic everywhere and still catches a dropped
toLocaleString wherever the locale groups at all.
2026-08-12 22:23:06 -07:00
Yuneng Jiang
214a476c3e
refactor(ui): migrate TokenFlow and JsonViewer to shadcn
Drops the antd Typography import from both. TokenFlow renders inside an antd
Descriptions.Item that already sets the colour, font size, line height and
wrapping the Text wrapper restated, so a bare span is pixel-identical there.
JsonViewer's placeholder moves onto the muted-foreground token.

The characterisation tests from the previous commit are unchanged and stay
green, which is what shows the markup swap did not move behaviour.
2026-08-12 22:10:39 -07:00
Yuneng Jiang
49c697ee89
refactor(ui): migrate HistoryTree and CollapsibleMessage to shadcn
Both log-drawer collapse rows hand-rolled the same pattern: a click
handler on a plain div, hover tracked in React state, and a max-height
tween on an always-mounted panel. Move both onto the ui/collapsible
primitive with lucide chevrons, so the row is a real button that keyboard
users can reach and the open state lives in the primitive.

CollapsibleContent keeps keepMounted, which preserves the existing
contract that panel content stays in the DOM while collapsed.

Neither test file is touched: both were already role and text based, and
they pass unedited against the new markup.
2026-08-12 22:03:20 -07:00
Yuneng Jiang
89d5037780
test(ui): characterize TokenFlow and JsonViewer
Both components are shared by the logs, guardrails-monitor and tool-policies
routes and had no test. These assert on rendered text and roles only, so they
hold against antd Typography and against its replacement.
2026-08-12 21:48:51 -07:00
Yassin Kortam
50ff555d79
fix(ui): show and edit key-level router settings on a virtual key (#36674)
The virtual-key edit view never rendered the router settings stored on a
key, so fallbacks configured at creation could not be verified or changed
afterwards. The key info panel now summarises them and the edit view
embeds the router settings accordion.

The accordion is a fixed-field editor, so its value is merged over the
stored object instead of replacing it. Routing fields it cannot render,
such as tag_routing_prefix or model_group_retry_policy, survive an
unrelated edit, while a field it does own that the admin emptied still
goes out as null so the clear reaches the server. Emptying every field
sends {}, which the proxy reads as no key-level override, so the key
falls back to its team and global routing rather than being pinned to a
blob of nulls.
2026-08-12 16:47:45 -07:00
Mateo Wang
964f0755ee
Merge pull request #36495 from BerriAI/litellm_no_redis_warning_banner 2026-08-12 15:55:47 -07:00