Commit graph

46529 commits

Author SHA1 Message Date
mateo-berri
c435c25da2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr35110_itpm_otpm
# Conflicts:
#	type-discipline-budget.json
2026-08-18 16:11:12 -07:00
Mateo Wang
2903d3a02e
Merge pull request #37380 from BerriAI/litellm_cap_guardrail_usage_window
fix(guardrails): cap the date window accepted by /guardrails/usage endpoints
2026-08-18 16:01:51 -07:00
Yuneng Jiang
c2e0daa50a
fix(ui): reseed the MCP tool test form when the schema changes
The antd form reset on [form, actualSchema, tool], so it reseeded whenever
the schema changed and not only when a different tool was picked. Keying the
migrated form's remount on tool.name alone narrowed that: a same-named tool
whose schema changed would have kept its old indexed values and submitted
them under the new schema's keys, and any field the new schema added would
never get its default. The key now covers the schema content as well.

Reachability, so nobody reads more into this than is there: selectedTool is a
state snapshot set on click, so this is not reachable through the current
parent. It is a latent divergence rather than a live bug, and it is fixed
because the contract for this migration is zero functional change.

The schema default helpers move into the pure module beside the other
argument logic. They arrived carrying the original's explicit any and built
their result by mutating it; they are now typed with unknown, built by
spread, and covered directly by unit tests rather than only through a render.

Also takes the last five hardcoded neutrals onto tokens. The earlier pass
scanned the gray family only and did not see the slate ones in the tool name
chip, which had no dark variant. Counting colour utilities that have no
dark: counterpart reads 0 for this file now, against 64 before the migration.
2026-08-18 15:59:36 -07:00
Yuneng Jiang
09ad62a40d
fix(ui): stop the placeholder option clearing the picked member identity
The Base UI combobox only renders a selected value that is present in its
item list, so the port synthesizes an item for the current value when the
search results no longer contain it. That synthetic item carried an empty
user, and selecting it ran the same handler as a real result, wiping both
the email and the user id before submit.

The synthetic item now carries no user at all and the select handler
ignores it, so reselecting the value already in the field leaves both
identity fields alone. antd needed none of this: its Select renders a
value that is absent from its options.
2026-08-18 15:58:16 -07:00
Devin AI
0585c45cd9 fix(types): avoid mutable dict literal in nested cache token lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 22:52:02 +00:00
mateo-berri
5513fd032d fix(guardrails): requeue usage rollup rows dropped after retry exhaustion 2026-08-18 15:51:56 -07:00
Yuneng Jiang
c71b6ed51b
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/competent-lewin-1c8fd9 2026-08-18 15:49:12 -07:00
Yuneng Jiang
e8f698ad65
test(ui): pin the section-gated team create and update payloads
The team create and edit forms send a different set of keys depending on
which collapsible sections the user opened, because a closed section is
unmounted and its values never reach the request. Nothing covered that,
so a form rewrite could change the request body without failing a test.

Pins the exact key set the create form sends with every section closed,
the keys Additional Settings adds once opened, and that a value typed
then re-hidden is dropped while a reopened one is restored. Does the same
for the team member and search tool sections on the edit form, asserting
absence at the wire level rather than just comparing values.

Also hardens two option queries in the member modal suite onto the option
role, and lifts the duplicated mock seeding in the team info suite into
one function both blocks call.
2026-08-18 15:49:05 -07:00
mateo-berri
eb3ed6cf39 fix(guardrails): reject non-canonical date formats in usage windows 2026-08-18 15:46:09 -07:00
Mateo Wang
6c4059aacc
Merge pull request #37367 from BerriAI/litellm_lit_5527_semantic_cache_embedding_truncation
fix(caching): truncate semantic cache embedding input, send extra_body top-level
2026-08-18 15:45:30 -07:00
mateo-berri
3c34c34459 fix(proxy): guard candidate-count and batch cap coercion against float overflow 2026-08-18 15:42:02 -07:00
yuneng-jiang
657ded533c
test(ui): raise vitest test and hook timeouts for CI headroom (#37370)
* test(ui): raise vitest test and hook timeouts for CI headroom

The UI unit suite runs about 3x slower on the CI runner than locally, which
put the slowest cases right on the 30s per-test limit. TeamInfo's pass
through routes case takes ~8s locally and has been failing on staging at the
timeout across consecutive runs even though it passes reliably when run
directly.

Raise testTimeout to 60s and set hookTimeout to 30s so the current slow cases
have headroom. This is a stopgap while the suite gets split into proper tiers,
not a fix for the underlying per-test cost.

* test(ui): query agent form panels with findByRole like the rest of the file

The panel helper was the only synchronous query in add_agent_form's
integration test; every other lookup already retries via findBy. On the CI
runner the second case has been failing with "Unable to find an accessible
element with the role button and name /Cost Configuration/" against a modal
whose body had not rendered.

Make the helper retry like its siblings and await it at each call site.
2026-08-18 22:41:16 +00:00
Devin AI
645b87fae1 fix(types): map nested prompt_tokens_details.cache_creation_input_tokens to cache_write_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 22:40:27 +00:00
Yuneng Jiang
aa95809c82
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/elastic-goldstine-104755 2026-08-18 15:38:40 -07:00
Yuneng Jiang
fc32eb081a
refactor(ui): move the MCP tool test form off antd
The tool test panel drove its argument fields through an antd Form, so the
call payload was whatever rc-field-form happened to have mounted. It now runs
on react-hook-form with shadcn controls, and the payload itself lives in
toolCallArguments.ts as a pure function of the schema fields plus the entered
values.

Fields bind by index rather than by name, because an MCP tool's JSON schema
can name a property anything: a key containing a dot would be one flat key to
antd but a nested path to react-hook-form. Binding to args.0, args.1 and
zipping back to the real keys at submit time keeps the emitted arguments
identical whatever the server calls its properties.

Coercion, the blank filter, the required and JSON rules, and the params
wrapper for nested-object schemas all keep their previous behaviour, and the
neutral colours in the panel move onto tokens so it reads correctly in dark
mode.
2026-08-18 15:38:38 -07:00
mateo-berri
c9bfb7f0ab fix(guardrails): cap the date window accepted by /guardrails/usage endpoints 2026-08-18 15:37:57 -07:00
ryan-crabbe-berri
eef41c9987
refactor(ui): move the admin, SSO, SCIM, alerting and fallback forms off tremor (#37315)
* refactor(ui): move the admin, SSO, SCIM, alerting and fallback forms off tremor

Swaps the tremor Button, Card, Callout, Grid, Divider, Text, Title, TextInput, Table parts, Badge, Icon and Switch in these nine files for the shadcn layer and lucide icons, keeping antd in place. The SCIM create-token button keeps an explicit type="submit"; the two SCIM copy buttons sit outside the antd form so they stay plain buttons, and the alerting form's Enterprise Feature upsell button is a link wrapper rather than a save action, so it deliberately stays type="button" while the form keeps its own Update Settings submit. The teal login callout on the admin panel maps to the info Alert variant since no teal variant exists. Prunes the nine tremor no-restricted-imports suppressions these files no longer need.

* fix(ui): keep the alerting settings name column left aligned

tremor's TableCell hardcoded text-left, so the align="center" attribute never took effect. The shadcn cell has no text-align of its own, so translating that attribute into text-center would have centered the field name and its description for the first time.

* fix(ui): restore the CloudZero key reveal toggle and the SCIM divider gap

tremor's TextInput drew its own show/hide button whenever the type was
password and the field was not disabled, so the straight pass-through to
the plain shadcn Input silently deleted that affordance from the CloudZero
API key field. Rebuilds it with the InputGroup reveal pattern that
email_settings.tsx already uses, behind a small local control so the antd
Form.Item keeps its id, value and onChange wiring and the field still
matches its shadcn sibling in the same form.

Also puts the SCIM separator back on my-6: tremor's Divider was
"w-full mx-auto my-6", and the conversion shipped my-4, tightening that
gap by 8px on each side. The disabled SCIM token field stays a bare Input
because tremor suppressed its toggle when disabled too.
2026-08-18 22:36:35 +00:00
Yuneng Jiang
c1b2df8e4f
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/funny-cerf-33d2bd 2026-08-18 15:35:37 -07:00
Yuneng Jiang
fae700da36
refactor(ui): tokenise the access group selector and drop its stale binding note
The selector takes value and onChange, which is exactly what lets it sit
inside both an antd Form.Item and a react-hook-form FormField, so the doc
bullet naming only antd was about to describe half the truth. The props
interface already states the contract.

That bullet was also the only Form.Item match in the file, and it has twice
inflated the migration's canonical tag count, which now needs no subtraction.

Its four hardcoded colours move to tokens for the same reason as the rest of
the sweep. Its antd Select stays.
2026-08-18 15:35:35 -07:00
Yuneng Jiang
78670789a0
refactor(ui): replace the agent form's last antd Form.Item with its own field label
The agent forms moved to react-hook-form in #37357, which deliberately kept
the antd Select for the agent type picker because its dropdownRender footer
and two-tier options have no shadcn equivalent. The Form.Item wrapped around
it survived as a side effect rather than for that reason: it carries no name
and no rules, and the file renders no antd Form at all, so it bound nothing
and validated nothing.

It was also the only label in the file rendering antd's required asterisk,
while AgentFormField renders genuinely required fields without one. Moving it
to the file's own Field, FieldLabel and labelWithHint makes it match, and
gives the label a control to point at.
2026-08-18 15:35:35 -07:00
Yuneng Jiang
1b5f40b47d
refactor(ui): move the model alias manager onto design tokens and shadcn controls
The shared alias editor was light-only: 18 hardcoded palette classes across
its headings, table cells, config preview and five raw controls, so it read
grey-on-grey against the dark dashboard theme.

Its five buttons also carried no type attribute, and all three consumers
render it inside a real antd Form with an onFinish. Measured against the
pre-change file in jsdom with a liveness gate on both sides: clicking Add
Alias fired the parent's onFinish once, and a row action fired it once more.
So editing a model alias inside the create-team, team-update or create-key
form also submitted that form. shadcn Button renders type="button", which
closes the path.

Swapping the raw input and button elements for the shadcn primitives is what
makes the colours resolve, since a bare element needs the whole token set
hand-written to work in both themes.
2026-08-18 15:35:26 -07:00
Yuneng Jiang
48986da99c
test(ui): pin the placeholder branch alongside the label branch
`items` feeds `hasNullItemLabel` as well as label resolution, and that
selector decides whether `Select.Value` renders the placeholder. None of
the twenty fixed sites has a null-valued item, so their placeholders are
unchanged, but nothing pinned that.

Cover both halves: the placeholder still renders when nothing is
selected, and a null-valued entry in `items` takes over from it. The
second case is the trap to avoid when adding `items` to a Select that
relies on its placeholder.
2026-08-18 15:34:03 -07:00
Yuneng Jiang
653dc5c601
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/modest-hodgkin-5775d7 2026-08-18 15:30:37 -07:00
Yuneng Jiang
4cbceb565b
fix(ui): show select labels on the trigger instead of raw values
Base UI's Select.Value resolves an option's label only when the root
carries an `items` prop or the Value has a child. `resolveSelectedLabel`
in @base-ui/react/internals/resolveValueLabel.js falls through every
branch to `stringifyAsLabel(value)` otherwise, and `state.items` is
written only from the root's `items` prop, so the `<SelectItem>` children
rendered inside `<SelectContent>` never populate it.

A self-closing `<SelectValue />` on a root without `items` therefore
renders the raw value once something is selected. The placeholder branch
still works, so the trigger looked right until the user picked an option
and then showed `development` for Development, `LiteLLM_VerificationToken`
for Keys, `all` for All Actions, and `24h` for Daily.

Pass `items` at the 20 affected sites, using the array form the other 52
call sites already use. Where a literal option sat alongside mapped ones,
build one array and map the options over it so the labels and `items`
cannot drift.

The record-map form is avoided deliberately: `items[value]` on an object
literal reaches Object.prototype, so a dynamic value named `toString`
would resolve to a function and React would throw on it. The array form
matches with `.find` and has no prototype lookup, which matters where the
values are user-supplied model groups, team ids and key aliases.

Also replace the option lookup in CompetitorIntentConfiguration's test
helper, which searched by text and clicked the last match. That match is
now ambiguous because the trigger carries the label too, and the helper
already flaked roughly one run in six before this change.
2026-08-18 15:29:35 -07:00
Yuneng Jiang
aebdba510f
refactor(ui): move the team member search modal off antd Form
Ports user_search_modal from antd Form to react-hook-form plus the shadcn
kit, keeping the antd Modal and Alert shells. Payload parity was proven by
rendering the antd original beside the migration in one describe.each: an
untouched submit yields the same three keys with the identity fields
undefined, and picking an option yields the same email and id on both sides.

antd Select swallows Enter, so the original never submitted from a field.
The Base UI combobox does not, which added an Enter-to-submit path; the
inputs now swallow Enter and both sides measure zero submits from every
field with one from the button.
2026-08-18 15:27:50 -07:00
Mateo Wang
e75b4b1c2a
Merge pull request #37362 from BerriAI/litellm_lit_5651_bedrock_guardrail_cost
feat(guardrails): count bedrock guardrail cost against spend and budgets
2026-08-18 15:27:43 -07:00
ryan-crabbe-berri
b13fabe9c2
refactor(ui): move the cache settings and playground model selector off tremor (#37323)
* refactor(ui): move the budget, cache, cost tracking and playground forms off tremor

Swaps tremor Accordion for the Base UI Collapsible, TextInput for the shadcn Input and the two
tremor Buttons for the shadcn Button across the budget modals, cache settings, the cost tracking
add-provider and add-margin forms and the playground model selector. The accordion bodies keep
tremor's unmount-when-closed semantics, since headless-ui's Disclosure.Panel and Base UI's panel
both default to unmounting, so the antd fields inside behave exactly as before.

The two cost tracking buttons are the one deliberate behaviour change. tremor's Button renders a
bare button with no type, so inside the antd Form that wraps both components it was an implicit
submit on top of its own onClick. For the discount form that meant every click ran
handleAddProvider twice, once from onClick and once from the form's onFinish, and for the margin
form the submit did nothing at all because that Form has no onFinish. The shadcn Button forces
type="button", so the add now fires once from onClick alone and no type="submit" is added back.

The three inputs that used onValueChange now read e.target.value, and each one gained a test that
types into it and asserts the reported string, so the wiring cannot silently regress. The cache
settings suite gained a collapse contract test that the advanced sections are absent until the
section is expanded. Prunes the six no-restricted-imports suppressions these files no longer need,
each dropping from two to one for the antd import that stays.

* fix(ui): keep enter to submit on the cost tracking add forms

The shadcn Button forces type="button", so converting the two tremor buttons left both cost
tracking modals with no submit button at all. Each form still holds two fields that block
implicit submission, the provider select's search input and the value input, so pressing Enter
stopped adding anything. Both buttons get type="submit" back.

For the discount modal that alone would restore the double add the conversion had just removed,
since a submit also ran the form's onFinish, so the parent drops onFinish and the now dead
handleFormSubmit. Click and Enter both go through onClick exactly once. The margin form's parent
never had an onFinish, so restoring the submit type there is enough on its own.

Adds three cases to the cost tracking settings suite: the discount add fires once from a click,
the discount add fires once from Enter, and the margin add fires once from Enter. Dropping either
type="submit" kills the Enter cases and putting onFinish back makes both discount cases see two
calls. Also drops the two empty placeholders on the budget modals that only existed to suppress
tremor's "Type..." default.

* fix(ui): restore Enter-to-submit on the margin modal

The tremor Button rendered a bare native button, which defaults to
type="submit", so Enter in the percentage field submitted the margin
modal. The shadcn Button wraps Base UI, which defaults to type="button",
and the migration also replaced the margin modal's form element with a
plain div, so Enter went inert while the visually identical discount
modal kept working.

Give the margin modal the same form wrapper the discount modal already
has and mark its action button as the submit button. Also move the cache
settings advanced-section test into the integration file, where a test
that renders the real component tree belongs.
2026-08-18 15:26:40 -07:00
Mateo Wang
6b7adf011e
Merge pull request #37355 from BerriAI/litellm_ultrafast_service_tier_cost
fix(cost_calculator): recognize the ultrafast service tier in cost calculation
2026-08-18 15:25:53 -07:00
mateo-berri
3894455c99 test(caching): annotate new semantic cache and hosted_vllm test helpers 2026-08-18 15:20:13 -07:00
ryan-crabbe-berri
9c38d6d002
refactor(ui): move the teams page and team detail views off tremor (#37317)
* refactor(ui): move the teams page and team detail views off tremor

Swaps the tremor Accordion, Badge, Button, Card, Grid, Text, TextInput and
Title usages in Teams.tsx, TeamInfo.tsx, EditMembership.tsx and
LoggingSettings.tsx for the shadcn layer. The team model badge colour map
becomes a variant map: all-proxy, direct and access-group chips render as
secondary and no-default as outline, so the kind is now conveyed by the
tooltip rather than by hue. The LoggingSettings top decoration is drawn with
border-t-4 border-t-blue-500 and its light red Remove button becomes a ghost
button with red text. antd stays in place for this pass and the eslint
no-restricted-imports counts for the four files ratchet down by one each.

* fix(ui): keep the password reveal and model badge hues in the team views

The tremor TextInput rendered a show/hide button for every password field, so
the shadcn swap silently dropped it for the sensitive logging parameters. The
password branch now renders an InputGroup with an eye toggle, matching the
pattern email settings already uses, and a test pins the masking.

The team model chips go back to four distinct colours by way of the shared
StatusBadge, so a directly granted model still reads differently from an
access group one without hovering for the tooltip.

The hand-drawn blue accent on the logging integration card is dropped: the
tremor decoration it replaced never rendered, because the caller's own border
classes won the class merge, so the bar was new rather than preserved.

* fix(ui): drop the dead empty placeholder on the team name field

The team name input carried placeholder="" only to suppress tremor
TextInput's default "Type..." hint. shadcn Input has no default
placeholder, so the empty string does nothing and the field now relies on
its label, matching the other converted create-team fields.
2026-08-18 22:15:41 +00:00
mateo-berri
d4db4b1379 test(e2e): pin marker alias connection params staying off the routed tier 2026-08-18 15:14:05 -07:00
mateo-berri
69ea1c6599 fix(proxy): coerce batch candidate counts like the live limiter path 2026-08-18 15:04:18 -07:00
mateo-berri
ef2c30227a fix(caching): truncate semantic cache embedding input, send extra_body top-level 2026-08-18 15:03:51 -07:00
mateo-berri
b849d073e0 fix(guardrails): bill completed chunks when a later chunk fails terminally
A terminal HTTP failure partway through chunking now logs the summed usage
and cost of the ApplyGuardrail calls AWS already billed, mirroring the
blocked-chunk path.
2026-08-18 15:02:11 -07:00
yucheng-berri
0b82b087fd
feat(team-callbacks): add DELETE /team/{team_id}/callback/{callback_name} (#37331)
Removes one named callback from a team and leaves the team's other callbacks
registered and firing. Before this, the only removal route was
POST /team/{team_id}/disable_logging, which clears every callback at once, so a
tenant sharing a team could not deregister its own integration

The handler filters metadata["logging"], keeps the survivors encrypted, refreshes
the cached team so the removal applies to keys that are already live, and emits a
redacted audit row, matching what the add and disable routes do

Resolves LIT-5161
2026-08-18 14:56:45 -07:00
yassin
cf2e50077c Merge branch 'litellm_internal_staging' into devin_ai_agentcore_search
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 21:53:22 +00:00
mateo-berri
354b0c3a45 fix(guardrails): bill all chunks on mid-chunking block, strip client guardrail cost metadata, add cost map schema keys
A blocked chunk now logs the summed usage and cost of every ApplyGuardrail
call AWS billed for the logical request, not just the blocking chunk.
Client-supplied metadata.standard_logging_guardrail_information is stripped
at the proxy boundary so callers cannot forge (even negative) guardrail
cost into spend, and guardrail_information_cost ignores negative or
non-finite entry costs as defense in depth. The cost map schema test now
allows guardrail_cost_per_unit and the guardrail mode.
2026-08-18 14:52:23 -07:00
mateo-berri
e9355a7fe9 fix(proxy): hand the embeddings failure hook the post-setup request data 2026-08-18 14:49:42 -07:00
mateo-berri
96cee087be test(e2e): pin auto-router tag-split, alias pricing, heuristic scope, and Responses routing regressions 2026-08-18 14:49:12 -07:00
ryan-crabbe-berri
2059033352
refactor(ui): move the add model and credential forms off tremor (#37325)
* refactor(ui): move the add model and credential forms off tremor

TextInput becomes the shadcn Input, Text becomes a sized paragraph, the
advanced settings Accordion becomes a bordered Collapsible, and the
Team-BYOK Switch moves to the Base UI switch with onCheckedChange plus an
aria-label. That switch stays wrapped in a span so the antd Tooltip still
shows on hover while it is disabled for non-premium users, matching what
the tremor wrapper div did. provider_specific_fields keeps antd's
Input.TextArea through an AntdInput alias so its antd import stays a single
statement. Prunes the tremor no-restricted-imports suppressions these files
no longer need.

* fix(ui): keep the reveal toggle on the provider secret fields

tremor's TextInput drew its own show/hide button whenever the type was
password, and the shadcn Input is a plain native input, so every provider
secret this form renders (API keys, client secrets, and the same fields
inside the add credential modal) lost that affordance.

Puts the password branch on antd's Input.Password, which is what the other
dynamic credential forms in the dashboard already use, so the reveal comes
back and the antd Form.Item wiring stays untouched. The control chain moves
into an early-return helper, which keeps the extra branch from pushing the
file past its no-nested-ternary budget and drops that count from 5 to 3.
2026-08-18 14:47:04 -07:00
ryan-crabbe-berri
92cf577f82
refactor(ui): move the virtual key create and edit forms off tremor (#37324)
* refactor(ui): move the virtual key create and edit forms off tremor

Swaps the tremor primitives in the create-key modal, the key edit view and
their two shared field components for the in-repo shadcn layer: Accordion
becomes Collapsible, Grid/Col become grid divs, Text/Title become real
paragraphs and headings, and TextInput becomes the shadcn Input. antd stays
where it already was, so the antd Input keeps rendering the textareas and
hidden fields under the AntdInput alias.

Two behavioural notes. The key edit view's Save Changes button keeps saving
because it carries an explicit type="submit"; Base UI's button otherwise
defaults to type="button". Its Cancel button now really is type="button",
where the tremor one had no type at all and so submitted the form on top of
calling onCancel, and a test pins that.

Base UI's Collapsible panel unmounts while closed exactly like the headless
Disclosure panel tremor wrapped, so the create-key tests now open Optional
Settings before querying inside it instead of relying on a tremor mock that
flattened every accordion.

* refactor(ui): hoist the create-key collapsible header classes and keep optional settings a heading

Names the repeated Collapsible trigger and chevron class strings the way the
cost tracking conversion does, since nine copies of each lived in this one
file, wraps the Optional Settings trigger in an h3 so the section keeps a real
heading next to Key Ownership and Key Details, and drops the placeholder=""
that only ever existed to suppress tremor's default hint.
2026-08-18 14:46:41 -07:00
ryan-crabbe-berri
28266d90e7
feat(vector_stores): add Valkey as a managed vector store provider (#37002)
* feat(vector_stores): add Valkey as a managed vector store provider

Adds a valkey provider for managed vector stores, searchable via the
valkey-search module over RESP. Introduces BaseDirectVectorStoreConfig
for datastores that execute searches directly instead of building an
HTTP request, and refactors the valkey semantic cache to share the new
connection URL helper. Registered in the provider enum, router params,
proxy config registry, Admin UI Add Vector Store modal, and provider
endpoint support matrix.

* fix(vector_stores): join list queries and bound valkey socket timeouts

Review feedback: multi-string queries are now space-joined like every
other embedding-based provider instead of dropping all but the first,
and the request timeout is threaded through the direct vector store
interface into bounded socket_connect_timeout / socket_timeout values
on both redis clients so an unreachable Valkey host cannot pin proxy
workers until the OS TCP timeout.

* chore(ui): regenerate schema.d.ts for valkey vector store fields

* docs(ui): make the Valkey vector store setup note and field tooltips explicit

* feat(ui): pick the Valkey embedding model from the proxy's models like Milvus

* fix(ui): number the setup steps in the vector store provider alerts
2026-08-18 21:45:22 +00:00
mateo-berri
72960d10e9 fix(proxy): address review findings on project ITPM/OTPM quotas
- scale batch output-token reservations by the row's n / best_of candidate count
- parse client-supplied output caps defensively instead of 500ing on unparseable values
- exclude project IO descriptors from the first should_rate_limit pass when TPM
  reservation is disabled so their buckets are not double-charged
2026-08-18 14:45:05 -07:00
Yassin Kortam
3fe0201d40
fix(proxy): let org admins view their organization's usage (#37235)
An internal user who administers an organization saw an empty
Organization Usage dashboard and had to be promoted to proxy admin to
see any of it.

Two independent gates were closed on them. The route layer rejected
GET /organization/daily/activity with 401 before the handler ran, since
the route belonged to no list a non-proxy-admin can reach, and the
handler's own org-admin scoping was therefore dead code. In the
dashboard, viewOrganizationUsage was granted by session role alone, and
an org admin's session role is internal_user, so the Organization Usage
option never rendered and its data fetch stayed disabled.

The route now sits in self_managed_routes, where the handler restricts
results to organizations the caller is ORG_ADMIN of and 403s on any
other org, and viewOrganizationUsage joins the existing per-capability
org-admin allowance that already covers viewDeletedTeams.

A caller who administers no organization resolves to an empty id list
rather than to None, so the organization-alias lookup is scoped by that
same list instead of reading the whole table.

The Usage page falls back to the global view when org-admin membership
is revoked while it is open, so the selector never keeps a value it no
longer offers.
2026-08-18 14:44:36 -07:00
Yassin Kortam
1857f5d04b
fix(proxy): send SSE keepalives while a slow upstream is still silent (#37322)
A model with a long time-to-first-token leaves the proxy's response completely
idle, so any hop with an idle read timeout (AWS ALB and nginx both default to
60s) drops a connection that is perfectly healthy and would have delivered its
tokens shortly after.

The keepalive engines LiteLLM already ships wrap the response object, so they
fill a gap once the upstream has answered and then gone quiet. They cannot fill
the gap before it answers at all, and that is where the whole wait is spent:
measured against api.openai.com/v1/chat/completions with gpt-5.6 at
reasoning_effort high, the response headers and the first body byte both arrive
at 37.90s. Nothing has entered the ASGI response phase by then.

The upstream call is now raced against the keepalive interval, and when it
loses, the SSE response is opened immediately and ": ping" comments, which every
conformant SSE client ignores, fill the wire until the real response is ready to
be replayed onto it. One seam per funnel: base_process_llm_request covers every
native route, create_pass_through_route covers every passthrough route.

Committing the status line that early is the cost. A failure discovered after
the first ping reaches the client as an SSE error frame under a 200 rather than
as an HTTP error status, and LiteLLM's own x-litellm-* response headers are not
yet known. keepalive_ping_has_fired already documents the same trade-off for the
existing engines. Both are why this stays off until an operator sets
litellm_settings.sse_keepalive_ping_interval_seconds.

Separately, the passthrough relay reached neither engine even for mid-stream
gaps, which is the shape of #32491 and #24929, so the relayed bytes get the same
treatment, gated on the upstream declaring text/event-stream and only emitted
between complete frames so a binary transport (AWS event streams on /bedrock)
and a stall halfway through a frame are both left alone.

Fixes #34819
2026-08-18 14:43:01 -07:00
Yassin Kortam
49b72e14da
fix(anthropic): emit tool_use content_block_start without awaiting the next chunk (#37310)
On /v1/messages, AnthropicStreamWrapper synthesizes the content_block_start for
the first content block, queues it, then hits a bare `continue` when that same
upstream chunk's translated delta is empty. The queue is only drained at the top
of the next __next__ / __anext__, so the queued content_block_start waits for a
further upstream chunk to arrive.

An empty delta on the opening chunk is the normal tool-call shape: Bedrock
Converse's contentBlockStart carries the tool id and name with no arguments, and
OpenAI-format streams send arguments: "" on the chunk that names the function.
So a client learns a tool call started one upstream event late, and when the
provider delivers argument fragments as a trailing burst it sees nothing at all
after message_start for the whole generation.

Flush the queued event before continuing, in both the sync and async paths. The
sibling block-transition path already returns from the queue, so only the
first-block-open case changed.
2026-08-18 14:42:42 -07:00
yuneng-jiang
c180849210
refactor(ui): migrate the MCP per-user env vars, toolset and tool arguments forms to react-hook-form and shadcn (#37349)
* refactor(ui): migrate the MCP per-user env vars modal to react-hook-form and shadcn

Moves UserEnvVarsModal off the antd Form store onto react-hook-form with a
zod schema built from the server's declared per-user variables, and swaps
antd Input.Password for the shared PasswordInput.

The submit payload is unchanged: every declared variable is still sent as a
key, trimmed, with an untouched field sending an empty string. antd reset
the store from the modal's afterOpenChange; the migrated form reproduces
that by remounting on the same callback, so reopening still starts blank.

Adds UserEnvVarsModal.test.tsx, which was written against the antd original
and proven green before any production change, then re-run unedited against
the migration. Two further cases cover the reveal toggle, which antd
provided through visibilityToggle.

* refactor(ui): migrate the MCP toolset create and edit form to react-hook-form and shadcn

Moves the toolset name and description fields off the antd Form store onto
react-hook-form with a zod schema, and takes the surrounding panel onto
semantic colour tokens so the tab renders in dark mode. The purple selected
tool styling keeps its hue and gains dark variants rather than flattening
to neutral.

Payload is unchanged: create still sends toolset_name, description and
tools, an untouched description is still the empty string rather than
undefined, and the tool selection is still held outside the form. The antd
form carried no onFinish and its buttons sit outside the form element, so
the migrated form keeps submit on the footer button and neutralises its own
submit rather than introducing Enter to save.

Adds MCPToolsetsTab.test.tsx, proven green against the antd original before
any production change and re-run unedited afterwards.

* refactor(ui): migrate the MCP tool arguments form to react-hook-form and shadcn

Moves the schema-driven tool argument form off the antd Form store onto
react-hook-form. Validation moves to an explicit resolver that reproduces
antd's rules field by field, including the per-field required message and
the JSON object and array messages, and the same resolver is reused by
getSubmitValues so the imperative path and the rendered errors cannot
disagree.

getSubmitValues still rejects with a plain object carrying errorFields
rather than an Error. ChatUI branches on `err instanceof Error` to choose
its toast, so rejecting with an Error would have silently changed the
message the user sees. That is pinned by a test proven green against the
antd original with a Form.Item liveness gate, and proven red when the
rejection is switched to an Error.

Enum and boolean fields keep the antd Select, whose allowClear has no
shadcn equivalent; dropping it would remove the only way to unset an
optional enum. Everything else moves to the shadcn Input and Textarea and
onto semantic colour tokens.

Adds MCPToolArgumentsForm.test.tsx covering the string, integer, number,
boolean, object, array, nested-params and string-schema paths, written
against the antd original and re-run unedited afterwards.

* refactor(ui): drop the decorative antd Form.Item from the MCP connect guide

The connect guide rendered a single antd Form.Item with no field name and no
Form ancestor, so it registered nothing and carried no payload; it was only
supplying bottom margin. It becomes a div with the same margin class, which
removes the file's last antd Form dependency.

Also takes the guide onto semantic colour tokens so it renders in dark mode.
The blue and green callouts keep their hue and gain dark variants rather
than flattening to neutral, since the colour carries meaning there.

* chore(ui): ratchet the MCP tool arguments form lint suppressions

The react-hook-form migration removed four of the five nested ternaries
in MCPToolArgumentsForm, so lower the grandfathered count to match and
hoist the one inline object literal the budget rule flags.

* test(ui): classify the MCP modal batteries as integration tests

Both render a real component tree down to the form controls and stub only
the network boundary, which is the repo's definition of an integration
test rather than a unit test. The tool arguments battery renders a single
module in milliseconds, so it stays unsuffixed.
2026-08-18 21:41:53 +00:00
yuneng-jiang
2502776700
refactor(ui): migrate agent forms to react-hook-form and shadcn (#37357)
* refactor(ui): migrate agent forms to react-hook-form and shadcn

Move the agent create wizard and the agent detail editor off antd Form onto
react-hook-form with shadcn primitives. The two parents share three children
(agent_form_fields, dynamic_agent_form_fields, cost_config_fields), so the whole
form graph migrates in one commit.

The submit payload is unchanged. antd validates and submits only fields that are
currently mounted, and its Collapse panels mount lazily on first open and then
stay mounted, so a payload depends on which panels the user ever expanded.
react-hook-form keeps every registered value instead, so the panels track their
own mounted set and the detail editor filters the never-opened panels back out
before building the request. shouldUnregister stays off, since it drops values
for collapsed panels rather than merely excluding them from submit.

Tags, examples and forwarded header names move from antd tags-mode selects to a
combobox in the kit. Base UI clears the combobox input on blur before the blur
handler runs, so the pending text is committed from the input-clear reason,
which is what antd did when the field lost focus.

The agent type picker stays on antd Select: its popup content is not part of the
form graph, and the Base UI popup opens a macrotask later, which the existing
unit test cannot observe.

* refactor(ui): use the shared PasswordInput in the agent forms

* refactor(ui): drop the narration comments from the agent wizard
2026-08-18 14:40:42 -07:00
yuneng-jiang
3c82791968
refactor(ui): migrate the guardrail forms to react-hook-form and shadcn (#37364)
Moves the guardrail form graph off antd Form onto react-hook-form with the
shadcn field primitives. The graph migrates atomically: add_guardrail_form and
guardrail_info own the form instances, and guardrail_provider_fields,
guardrail_optional_params and LLMJudgeFields are field groups rendered inside
them, so an antd parent could not host a react-hook-form child either way.

The submit payload is unchanged. Two characterization suites, 25 cases, pin it:
each case was written against the antd original, proven green there, and passes
unedited against the migration.

Behaviour worth calling out. Nested provider fields are keyed with ":" rather
than "." so they stay flat keys the way antd stored them, since a dotted name
is a lodash path in react-hook-form and would have started shipping a nested
object. antd InputNumber clears to null and clamps on blur where a native
number input does neither, so the judge criteria weights reproduce that. The
guardrail_info submit handler is read through a ref at validation-resolution
time, matching how antd re-read onFinish, so a submit fired by the same click
that changed state still sees that state.

Two antd behaviours are preserved rather than fixed, both worth their own
follow-up: deselecting every mode blocks Next instead of falling back to the
seeded default, and a required provider-specific field is never enforced at
create time. One is fixed and disclosed: a failed validation now names the
problem instead of rendering "[object Object]", and the guardrail name label is
associated with its control, which it was not before.

MultiSelect takes an optional id so the label can point at the control.
SkipMessageSelect was duplicated verbatim in both parents and now lives in the
shared field module.
2026-08-18 14:39:36 -07:00
mateo-berri
3a4d3a01af fix(proxy): only estimate failed-request input tokens for call types whose input is countable 2026-08-18 14:36:57 -07:00