Commit graph

43876 commits

Author SHA1 Message Date
yuneng-jiang
bb388b2566
refactor(ui): migrate workflow runs to shadcn (#34370)
* test(ui): characterise the workflow runs detail drawer before migrating it

Pins the drawer's behaviour against the current antd implementation: the
metadata fields it surfaces, the timeline ordered by sequence number, the
empty-events copy, the messages section staying collapsed until opened, the
in-drawer refresh refetching, and the close control dismissing it.

Every assertion is role/text based so the same file can stay green once the
component moves off antd, without being edited.

* refactor(ui): migrate workflow runs to shadcn

Replaces the antd Drawer, Collapse, Button, Spin, Tooltip and Empty on the
Workflow Runs page with the installed Base UI primitives (Sheet, Collapsible,
Button, UiLoadingSpinner, Tooltip) and lucide icons, and moves the page's
hardcoded hex colours, fonts and geometry onto design tokens and utility
classes so the page can be themed. The only inline styles left are the gantt
bars' computed left/width, which are runtime values.

Behaviour is unchanged: the drawer's characterisation tests were written
against the antd version in the previous commit and pass here without being
edited.

Retires the file's now-unused antd no-restricted-imports suppression.
2026-07-23 10:19:04 -07:00
Tin Chi Lo
8929e09f49 fix(ui): gate the keyless connect redirect on internal-user roles
isAdminRole compares against a list that mixes raw and formatted role
strings: it holds raw org_admin but not the "Org Admin" that
formatUserRole produces, and AuthContext stores the formatted form. A
keyless org admin therefore read as a non-admin and was redirected to
the connect page.

Gate positively on internalUserRoles instead, which carries both
representations, so the redirect targets the persona it is meant for and
any role that is not unambiguously an internal user is left on the
dashboard. The shared admin list is left alone: completing it would
change org-admin access across every isAdminRole caller, which is a
roles-policy decision of its own.
2026-07-23 09:56:01 -07:00
tin-berri
712136d10d
Merge pull request #34225 from BerriAI/litellm_lit4658_oauth_discovery_logging
fix(mcp): log actionable OAuth discovery failures for misconfigured server urls
2026-07-23 09:51:31 -07:00
tin-berri
55b0046089
Merge branch 'litellm_internal_staging' into litellm_lit4658_oauth_discovery_logging 2026-07-23 09:39:11 -07:00
tin-berri
6bdb7918ea
Merge pull request #33190 from BerriAI/litellm_lit3637_session_admission
feat(mcp): admit gateway DCR session bearers at the aggregate /mcp scope
2026-07-23 09:33:59 -07:00
CrypticDriver
b43441814b test: mirror AgentCore search tests into tests/test_litellm for coverage
Coverage collection runs against the sharded tests/test_litellm tree, so
the provider tests living only in tests/search_tests were invisible to
codecov (patch coverage reported ~31% despite the suite). Mirror them as
tests/test_litellm/llms/bedrock/search/test_agentcore_search_transformation.py
and add edge-case tests (malformed MCP content blocks, SSE without a JSON
frame, notification-only streams, list request body, error-class mapping).

transformation.py line coverage: 99% (26 tests x2 trees).
2026-07-23 15:31:10 +00:00
ryan-crabbe-berri
2bfd50ed37
test(e2e): cover key budget_duration resets on personal, team, and team-member keys (#33896) 2026-07-23 07:18:35 -07:00
Mateo Wang
3bba3633c7
Merge pull request #34338 from BerriAI/litellm_lit_4313_sagemaker_chat_streaming_ttft
fix(sagemaker): forward stream events as they arrive to cut TTFT
2026-07-23 00:37:39 -07:00
Tin Chi Lo
ffa0dffbc6 refactor(mcp): trim redundant comments and dedupe admission-arm tests
Compress the security rationale in the gateway-session admission path of
user_api_key_auth_mcp.py, keeping the load-bearing "why" and dropping the
restatement, and remove a garbled dead comment in get_allowed_tools_for_server

In the tests, hoist the duplicated _team / _admitted_subject fixtures to
module-level factories and parametrize the four fail-closed session-bearer
variants into one case. No behavior change; the 294 tests in the file still pass
2026-07-23 00:24:28 -07:00
Tin Chi Lo
a78130461f feat(mcp): gateway DCR session admission at the aggregate /mcp endpoint (LIT-3637)
Admits a keyless SSO user (no virtual key) at the aggregate /mcp endpoint from a gateway DCR
session bearer, resolving team/org/SCIM/budget authorization fresh on every call.

- Aggregate DCR front door: stateless /register (sealed llm_dcrc_ client ids), SSO-backed
  /authorize + /authorize/complete, and /token minting identity-only session tokens with PKCE,
  single-use codes/flows, and rotating refresh tokens.
- Admission: a session-shaped Authorization at the aggregate scope opens via _admit_gateway_session,
  reloads the live user, and runs the centralized policy gate; failures return the RFC 9728
  invalid_token challenge. Gated on the un-forgeable, server-only mcp_admitted_user_subject marker,
  so virtual-key and JWT auth are unchanged.
- Authorization model: an admitted subject is resolved as one plain UserAPIKeyAuth per grant source
  (its own grants, plus each team it is a live roster member of), each answered by the SAME resolver
  virtual keys use, then unioned. That branch is the FIRST statement of BOTH public resolvers, so no
  single-credential prelude runs for it and a fault in a lookup it never uses cannot deny its grants. A source team counts only while it is a live grantor: roster membership, not
  blocked, and neither the team nor its owning org over budget (enforced through the SAME
  _team_max_budget_check / _organization_max_budget_check owners common_checks uses for keys).
  Each team source carries that team's own org, so the existing org
  ceiling caps it; for a keyless source the org list only ever intersects (a ceiling must not become
  a grant) and an unresolvable ceiling denies rather than silently uncapping, on both the server and
  tool axes. _roster_team_object is the single owner of "which teams count": a team whose roster no
  longer lists the user neither grants servers nor throttles, in one place.
- Rate limits: the subject is bounded by its user rpm/tpm AND by the per-server mcp_rpm_limit of
  the team a call is ATTRIBUTED to — the same single source billing charges, from the same owner. A key charges its one pinned team's bucket; a keyless
  subject has no team_id, so admission stamps each granting team's limit map onto the auth
  (server-only field, stripped from validated input like the marker) and the limiter emits that
  team's mcp_per_team descriptor. Charging every granting team instead would let one cross-team user
  drain several teams' SHARED buckets on a single call and block their other members; and a server
  the user's OWN grant reaches charges no team bucket at all, because no team provided it. Per-KEY
  MCP limits do not apply because there is no key.
- Wrapper channels: the manager-level union treats the admitted subject by the same grant model.
  The admin-role short-circuit and the absolute no_mcp_servers early-return are key-credential
  rules and never apply to it (a session bearer is a third-party client credential, not the
  dashboard, and the subject's opt-out silences only its own source). Operator-open channels
  (allow_all_keys, the user's own BYOM submissions) are owned by one operator_open_server_ids
  helper that BOTH the server union and the admitted tool resolution consult (suppress-BYOM-when-
  explicitly-scoped is a key-credential rule and never applies to the subject, whose user row
  carries the DB-default empty mcp_servers), so an open-channel
  server is default-open for tools instead of listable but uninvokable.
- Redirect URIs: one owner, validate_redirect_uri_shape, decides redirect-URI hygiene (bad scheme,
  fragment, missing host, userinfo, backslash host) and resolves allowlisted native callbacks, shared
  by DCR registration and the OAuth endpoints. Registration keeps a deliberately wider trust policy
  than validate_trusted_redirect_uri: public dynamic registration accepts any https client, and its
  controls are mandatory S256 PKCE plus the consent screen.
- Egress leak-defense: a gateway admission credential (session bearer / bridge envelope) is scrubbed
  from EVERY egress header context, anchored to the credential shape, so it can never be forwarded
  upstream and replayed.
- Single-use guard: auth-code, refresh and connect-flow claims resolve the proxy's cross-worker redis
  cache themselves rather than trusting the cache passed in, and fail CLOSED on a Redis fault instead
  of falling back to a per-worker count that a captured id could replay through another worker.
- Sign-in return_to: one shared, never-raising helper persists a safe return_to for every sign-in
  branch (SSO/Okta/generic and username/password), and every branch RESUMES through the same
  _sso_return_to_redirect the SSO callback uses, so however a deployment signs in the stored value
  is honored identically (same-origin path directly; control_plane_url via the one-time login-code
  handoff). A stale cookie is ignored rather than failing a completed sign-in.

- Budgets, both halves: ENFORCEMENT (an already over-budget team or its owning org stops being a
  grantor, in the source gate) and ACCOUNTING (a team-derived tool call is billed to the granting
  team and ITS org, so that budget accumulates and the right organization is charged). A server the
  user's own grant reaches bills the user; when several teams grant one server the pick is the
  lowest team_id, stable and auditable. Billing rides a COPY, so authorization still sees the full
  union, and it is inert when the target server cannot be resolved from the tool name.

Deferred (tracked): client-selected server scoping of the session token (LIT-4680).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 00:24:28 -07:00
Yuneng Jiang
a928db1fab
fix(ui): keep the memory detail sheet inside the viewport on narrow screens
The migrated sheet asked for a flat 720px width while its only max-width
came from the primitive's sm:-scoped rule, so below the sm breakpoint no
cap applied at all: on a 375px viewport the sheet rendered 720px wide with
its left edge at -305px, and because it is position:fixed there was no
scroll to reach the hidden content. The primitive's own w-3/4 default did
not have this problem; the fixed pixel width is what removed the guard.

Caps the width to the viewport at every breakpoint and only asks for
720px from sm up. Verified in a browser at 375px, 700px and 1280px: the
sheet is now 375, 700 and 720 wide respectively, always at left 0.
2026-07-22 23:40:19 -07:00
Mateo Wang
86eee8bd05
Merge pull request #34319 from BerriAI/litellm_anthropic_output_format_remaining_keywords
fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic
2026-07-22 23:29:37 -07:00
Tin Chi Lo
3cdd6ab9a1 test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
yuneng-jiang
f7842cdeb7
fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325)
* fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid

The non_root image baked the prisma CLI and engines under /app/.cache and used
the CLI's default (library) engine mode. Prisma stopped baking the library
engine, so `prisma migrate deploy` fell back to downloading it at startup,
which needs network egress and a writable cache. Under an arbitrary non-root
uid (OpenShift restricted-v2), an air-gapped network, or a readOnlyRootFilesystem,
that download fails and the proxy starts on an empty schema while every DB
endpoint returns 500. The migration entrypoint exits 0 on that failure, so a
default-uid `docker run` with network never surfaced it

Bake to /opt/prisma, a fixed world-readable path no cache mount shadows, and
pin PRISMA_CLI_PATH plus PRISMA_CLI_QUERY_ENGINE_TYPE=binary so the baked binary
engine is used directly, matching Dockerfile and Dockerfile.database. A
build-time guard asserts the binary query engine is present, so a future prisma
change that stops baking it fails the image build instead of silently degrading
migrations

Adds docker/test_offline_migration.sh, run from image-scan, which migrates a
fresh Postgres with no egress as a non-root uid and asserts the schema was
created, the case a default-uid `docker run` with network cannot catch

* test(docker): move the offline migration check into a gated pytest and stop pinning XDG_CACHE_HOME at the read-only bake

The offline migration check lived in docker/ as a shell script. It now lives in
tests/proxy_migration_tests/ as a pytest gated on LITELLM_IMAGE, matching the
sibling schema-migration test gated on DATABASE_URL, and image-scan invokes it
with pytest instead of bash. It also asserts the migration entrypoint's exit
code alongside the table count, so a crash or a container-startup failure fails
loudly rather than only surfacing as a low table count

Runtime XDG_CACHE_HOME pointed at /opt/prisma/.cache, which is baked a+rX with
no write, so any XDG-aware library writing a cache at runtime would be denied
for every uid. Leave it unset so it falls back to $HOME/.cache (/app/.cache,
created here and owned by the runtime uid), matching Dockerfile and
Dockerfile.database which never pin XDG at runtime. A second test guards against
a future edit pointing a cache or home var back at the read-only bake
2026-07-22 23:03:39 -07:00
Yuneng Jiang
5a3ab862ee
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/migrate-page-memory-9b2c09
# Conflicts:
#	ui/litellm-dashboard/eslint-suppressions.json
2026-07-22 22:52:48 -07:00
Yuneng Jiang
d2f872d04f
refactor(ui): migrate memory page to shadcn
Replaces the antd Drawer with ui/sheet and the antd Button, Typography and
Space usage with ui/button plus token utilities, and swaps the @ant-design
PlusOutlined icon for lucide's Plus. Toasts now go through the shared
MessageManager so the route no longer imports antd directly.

The route's tests were written against the antd components in the previous
commit and are unchanged here, so they pass on both implementations.

MemoryEditModal is left alone because it is built on antd Form; the table
already sits on the shared DataTable.
2026-07-22 22:52:15 -07:00
Yuneng Jiang
9dda5d882c
test(ui): characterise the memory page's drawer and header before migrating
Adds role/text-based coverage for MemoryDetailDrawer (which had none) and
extends MemoryView's test past the mocked table to the header, the create
modal trigger and the detail drawer round trip. Both are green against the
current antd components, so they act as an unedited regression net for the
shadcn migration that follows.
2026-07-22 22:23:45 -07:00
yuneng-jiang
baf85d7e38
refactor(ui): migrate search-tools info view to shadcn (#34323)
* test(ui): pin search-tools info view behavior before shadcn migration

Rewrite the markup-coupled copy-button assertions in SearchToolView to
role queries plus lucide icon-state, and add a role/text characterization
suite for SearchConnectionTest, which had none. Both are green against the
current antd/Tremor components so they can act as an unedited regression
net across the migration.

* refactor(ui): migrate search-tools info view to shadcn

Port the search-tools detail view and its two helpers off antd and Tremor
onto the installed shadcn primitives plus token utilities:

- SearchToolView (the info page reached by clicking a tool) now uses
  ui/button, ui/card and a plain CSS-grid header instead of Tremor
  Card/Grid/Title/Text and antd Button
- SearchToolTester swaps antd Input/Button/Spin and Tremor Card/Title for
  ui/input, ui/button and UiLoadingSpinner, with no inline styles
- SearchConnectionTest swaps antd Button/Divider/Typography and the inline
  keyframe spinner for ui/button, ui/separator and UiLoadingSpinner

Markup only; no behavior change. The list page (SearchTools) and its create
and edit forms stay on antd because they are Form-bearing and blocked until
the forms migration. Icons move from antd and heroicons to lucide. The
retired antd no-restricted-imports suppressions are pruned from the
baseline.

* fix(ui): drop redundant vertical padding in SearchToolTester card

The shadcn Card already applies py-6 and gap-6 to its flex children, so
the pt-6/pb-6/mb-6 added during the migration stacked on top of it and
roughly doubled the vertical whitespace. Keep only px-6 (Card has no
horizontal padding) and let the Card own the vertical rhythm, which
restores the original 24px spacing.

* style(ui): format SearchConnectionTest test file with prettier
2026-07-23 05:09:04 +00:00
ryan-crabbe-berri
070e19cff8
feat(organization): add RESTful PATCH /v2/organization/{organization_id} (#32350)
* fix(organization): persist cleared fields on /organization/update

Clearing an org field (the Metadata box or a TPM/RPM/max_budget limit) via PATCH /organization/update looked like it saved but reverted on refresh; the partial-update merge could not tell a cleared field from an untouched one and dropped every clear

The endpoint now decides SET vs CLEAR vs UNTOUCHED purely from which keys the raw request body carried, via a pure build_organization_update_plan. Budget nulls flow to update_budget (null clears via exclude_unset), metadata is replace-when-sent (written as {} for the non-nullable Json column), and a budget write on an org with no budget_id creates and links a budget row. This removes the exclude_none dump, both "if v is not None" filters, and the additive _update_dictionary merge

Resolves LIT-3664

* feat(organization): add RESTful PATCH /v2/organization/{organization_id}

Adds a v2 organization-update endpoint with a deterministic partial-update contract, and reverts the v1 /organization/update changes so its public behavior stays untouched

On v2 a field present in the request body is written (null/[]/{} clears, a value sets) and an omitted field is left untouched; presence is read from model_fields_set. Clearing a TPM/RPM/max_budget limit or the metadata now persists instead of being dropped as if it were never sent. Metadata is replace-when-sent and written as {} when cleared, since the org metadata Json column is non-nullable. Budget nulls flow to update_budget, and an org with no budget row gets one created and linked. The endpoint is hidden from the public Swagger docs via include_in_schema=False, and stays typed in the generated dashboard schema

Resolves LIT-3664

* test(organization): cover v2 auth guard, negative budget, and object_permission

Adds v2 endpoint tests that were missing: the real _verify_org_access path rejects a non-admin caller with 403 and writes nothing, a negative max_budget is rejected with 400 before any DB access, and a sent object_permission is passed to the upsert helper with its id linked onto the org write

Refs LIT-3664

* fix(organization): 400 on null-clear of required org fields; drop dead budget upsert

organization_alias and models are non-nullable columns, so a v2 request clearing them with null hit a 500 (NOT NULL violation) and could partially apply the budget half of the request first; the endpoint now returns a 400 with a clear message. Also removes the unreachable "create a budget when the org has none" branch from _apply_organization_budget_updates, since budget_id is a non-nullable FK and every org already has one, so the endpoint no longer needs to link a newly-created budget id

Refs LIT-3664

* fix(organization): let v2 clear object permissions when sent as null

Sending object_permission: null now detaches the org's permission by setting the nullable object_permission_id to null, instead of being a silent no-op, so the endpoint honors its documented "null clears" contract and an admin can actually revoke vector-store/MCP access. Sending a value still merges as before

Refs LIT-3664

* fix(organization): make v2 PATCH atomic, strict, and 422-consistent

Tighten the PATCH /v2/organization/{id} endpoint against standard HTTP
PATCH (RFC 5789 / RFC 7396 JSON Merge Patch) semantics:

- Apply the budget-row and org-row writes in one prisma transaction so a
  failure between them can no longer half-apply the patch (RFC 5789 requires
  a PATCH to apply atomically). The budget write is inlined as a tx-aware
  call mirroring the team-member budget path rather than the standalone
  update_budget route handler
- Set extra="forbid" on OrganizationUpdateRequestV2 so an unknown or
  misspelled key is a 422 instead of a silently dropped no-op; the contract
  is presence-driven, so swallowing unknown keys is unsafe
- Return 422 (not 400) for the hand-rolled field validations (negative
  budgets, null-clear of required organization_alias/models, invalid
  model_max_budget) so every validation failure matches the 422 that
  pydantic already returns for bad values
- Document the per-field clear tokens accurately: null clears budget limits
  and metadata, [] clears models, and organization_alias cannot be cleared

Tests cover the single-transaction write path, unknown-field rejection, the
422 status changes, and the budget_reset_at recompute.

* fix(organization): reject empty object_permission on v2 PATCH instead of silently keeping grants

object_permission is a nested merge field on PATCH /v2/organization/{id}: a
sent object merges into the existing permission row (updating one grant list
without touching the others), and null detaches it. An empty {} therefore
merged nothing and left every existing vector-store/MCP grant in place, so an
admin who sent {"object_permission": {}} to strip access silently kept it.

Reject a present-but-empty object_permission with a 422 that points the caller
at null, mirroring how the endpoint already rejects a null clear of the
required organization_alias/models. This keeps merge semantics for non-empty
payloads and does not affect the Admin UI, which only ever sends a fully
populated object or omits the field.

* fix(organization): JSON-serialize model_max_budget on the v2 budget write

model_max_budget is a Json column on the budget table. Route the budget-row
write through jsonify_object so a dict value is serialized the same way
new_budget and the org-row metadata write already do it, keeping every Json
column on this endpoint written consistently.

Raw dicts already round-trip (update_budget writes them unserialized), so this
is not a correctness fix so much as making the one Json column on the budget
path follow the same serialization as the rest of the file. Added a test that
a patched model_max_budget reaches the budget write JSON-serialized.

* refactor(organization): trim v2 docstrings and consolidate planner tests

Trim the verbose docstrings on the v2 endpoint, request model, and the two
pure helpers to the essential contract, and drop a stale line that still
referenced update_budget's exclude_unset (the budget write is inlined now).

Collapse the nine per-case planner tests into one parametrized test asserting
exact budget/org split per body, and fold the two model-validation rejection
cases into one parametrized test. Same 36 test cases run; the planner
assertions get stronger (exact-equality instead of presence/absence) and the
test additions shrink by ~85 lines.

* refactor(organization): inline the v2 update planner into the endpoint

Fold the OrganizationUpdatePlan dataclass and build_organization_update_plan
helper into update_organization_v2. The budget-vs-org split is a few dict
comprehensions built in one shot, so the extra type plus builder was more
ceremony than the job needed. Drops the now-unused dataclass/AbstractSet
imports and the isolated planner unit tests; the split is exercised end-to-end
by the endpoint tests.

* fix(organization): run v2 object permission upsert inside the update transaction

prepare_object_permission_upsert splits the shared helper's read-and-merge
step from its write so the v2 endpoint can upsert the permission row on the
same prisma transaction as the budget and org writes. Previously the upsert
ran before the transaction, so a rolled-back org write left merged grants
live on the permission row the org still pointed at. The upsert record now
pins object_permission_id, since the column's @default(uuid()) would
otherwise mint a fresh-create id different from the one linked on the org.
v1 and the team/key callers of handle_update_object_permission_common keep
their existing behavior

* fix(lint): keep the v2 org PR within the strict-rule budget

The strict gate flagged the PR's new code after the base merge: 11 UP045
Optional fields and a typing.List on OrganizationUpdateRequestV2, Dict
annotations in the new upsert helper and the TypeAdapter, and a B008 from
the v2 endpoint's Depends default. The model and helper now use pipe
unions and builtin generics, and the endpoint takes its auth dependency
via Annotated, which avoids the call-in-default pattern B008 targets

* fix(routes): expose /v2/organization on the backend component allowlist

The component-split coverage test requires every app route on a component;
the new v2 org PATCH belongs with the other management endpoints on the
backend, alongside the existing /v2/key and /v2/team prefixes

* fix(organization): clear budget_reset_at when budget_duration is cleared via v2 PATCH
2026-07-23 04:53:20 +00:00
ryan-crabbe-berri
0b0d59d62e
feat(ui): give each Models + Endpoints tab its own path (#34327)
* feat(ui): give each Models + Endpoints tab its own path

* refactor(ui): decompose Models + Endpoints into per-tab pages with URL-driven detail

Dissolve the 488-line ModelsAndEndpointsView monolith into one page per tab
under the models-and-endpoints route, with a persistent layout owning the
header, cost banner, tab bar and refresh. Each tab page owns only its own
state; shared lists come from a small useModelDashboardData hook.

Replace the stateful model/team drill-in (setSelectedModelId/setSelectedTeamId
full-page takeover) with real URL navigation: ?model=<id> and ?team=<id> render
ModelInfoView/TeamInfoView from the layout, so a model or team detail view is
now shareable, bookmarkable and back-button friendly. Removes the empty
placeholder pages from the first commit.

Swap the tab bar off phased-out tremor onto antd Tabs.

* fix(ui): render model tab panels standalone instead of Tremor TabPanel

AllModelsTab, ModelRetrySettingsTab and PriceDataManagementTab rooted their
render in a Tremor <TabPanel>, which only renders inside a Tremor <TabGroup>.
After the decomposition these panels live under antd Tabs / as route pages with
no such ancestor, so All Models (and the other two) rendered blank. Root them in
a plain container instead.

The existing component tests mocked @tremor/react (stubbing TabPanel to render
children), which hid this; add a regression test that renders with real Tremor
and asserts the content is visible standalone.

Also type visibleSlugs/TAB_LABELS with the canonical ModelTabSlug so a tab added
without a matching label is a compile error.

* fix(ui): make model/team drill-in navigation work under the /ui static mount

The drill-in close (Back to Models) and open were no-ops: the dashboard is a
static export served under /ui, a prefix the Next router (basePath "") does not
know, so a router.push to the current pathname with only the query changed is
deduped and never re-renders. Drive the ?model=/?team= overlay via real browser
navigation (window.location) so open and close reliably work; verified live.

Also address review feedback: gate the tab-permission redirect on teams/uiSettings
having loaded so a team admin hard-loading /add is not bounced to the base before
their membership resolves, and memoize getProviderFromModel on modelCostMapData so
the health tab's provider labels refresh when the cost map loads.

* fix(ui): use window.location.replace for the tab-permission redirect

router.replace is unreliable under the /ui static mount (same class of Next-router
issue that broke the drill-in back button), so the forbidden-tab redirect could
fail to fire. Use window.location.replace, which keeps the no-history semantics of
a permission redirect and is deterministic. Redirect stays gated on teams/uiSettings
having loaded.

* fix(ui): drive model/team drill-in with history.pushState for client-side nav

Switch the ?model=/?team= overlay navigation from window.location.assign to
window.history.pushState, which Next's App Router observes. This keeps navigation
client-side (no full page reload, React Query cache preserved) while still working
for the same-path query-only change that router.push cannot do under the /ui static
mount. Open, close (Back to Models) and browser Back are all verified in the built
UI. Adds unit coverage for the open/close/read behavior.
2026-07-22 21:39:33 -07:00
mateo
5bc1df5e47 test(sagemaker): assert make_sync_call maps non-200 to SagemakerError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 04:31:11 +00:00
mateo
a63884bc8d test(sagemaker): cover sync native streaming path via injectable make_sync_call
Extract the inline sync streaming post/decode into make_sync_call so it can be
exercised with an injected client, mirroring make_async_call, and add a sync
regression test that each token is forwarded after exactly one pulled frame.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 04:21:12 +00:00
mateo
27c91e6574 fix(sagemaker): forward native streaming events as they arrive to cut TTFT
Mirror the sagemaker_chat fix on the native sagemaker/ streaming path: the
sync and async completion handlers read the invocations-response-stream body
with iter_bytes(chunk_size=1024) / aiter_bytes(chunk_size=1024), so httpx
withholds bytes until 1024 accumulate and tokens arrive in gap-then-burst
waves. Drop the fixed chunk size so each decoded event is forwarded as its
bytes arrive.

Also add a boundary-agnostic decoder test proving frames reassemble correctly
regardless of where transport reads split the stream.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 04:03:25 +00:00
devin-ai-integration[bot]
ba86889f11
fix(autoroute): discover models via /v1/models so an AI-API-only key works (#34259)
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-22 20:50:14 -07:00
Mateo Wang
c1a66ac139
fix(bedrock): include type in tool_choice disable_parallel_tool_use config for Converse (#34347)
* fix(bedrock): include type in tool_choice disable_parallel_tool_use config for Converse

* fix(bedrock): let parallel_tool_calls-derived disable flag win over raw tool_choice value
2026-07-22 20:24:41 -07:00
mateo-berri
16dad256d4 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_anthropic_output_format_remaining_keywords 2026-07-22 19:59:09 -07:00
devin-ai-integration[bot]
eb2dce8771
fix(budget): resolve word-form budget_duration so it no longer silently resets daily (#34250)
* fix(budget): resolve word-form budget_duration so it no longer silently resets daily

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(budget): normalize legacy word-form budget_duration on key edit load so untouched saves stay canonical

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(budget): preserve canonical budget_duration in key update submit handler

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-22 19:58:23 -07:00
mateo-berri
75c0e12dac fix(bedrock): let parallel_tool_calls-derived disable flag win over raw tool_choice value 2026-07-22 19:52:54 -07:00
Shivam Rawat
abf18f8760
Merge pull request #33422 from BerriAI/litellm_fix_responses_reasoning_items
fix(responses): preserve reasoning through prompt hooks
2026-07-22 19:42:09 -07:00
ryan-crabbe-berri
67fce87b16
chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules (#34341)
* chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules

Wires up five error-level ESLint rules on the dashboard, grandfathering every
current offender into eslint-suppressions.json so the gate only bites new code
and ratchets down as files are fixed

- local/filename-pascal-case: new local rule requiring PascalCase .tsx names,
  exempting Next.js reserved files (page, layout, route, ...) and test/spec files
  (239 grandfathered)
- max-lines: 800 lines over src/**, excluding tests, src/data, and generated
  schema.d.ts (20 grandfathered)
- local/no-complex-jsx-arrow: new local rule flagging inline JSX arrow handlers
  with block bodies over two statements; each failure is a small extract-to-named
  -handler refactor (65 grandfathered)
- prefer-const: flipped from off to error (103 grandfathered)
- no-restricted-imports: added antd to the phase-out ban alongside tremor, and
  pointed both messages at shadcn/ui primitives (405 antd import sites grandfathered)

Both new local rules ship with RuleTester coverage

* fix(ui): preserve secondary extensions in filename-pascal-case suggestion

The suggestion text built the rename from only the head segment, so a
multi-dot file like my-component.utils.tsx was told to become
MyComponent.tsx instead of MyComponent.utils.tsx. Rebuild it from the
PascalCased head plus the untouched remaining segments, and add tests
covering multi-dot filenames and the hyphenated Next.js reserved names
(global-error, apple-icon, opengraph-image, twitter-image)
2026-07-22 19:34:35 -07:00
mateo-berri
6a7454271b fix(bedrock): include type in tool_choice disable_parallel_tool_use config for Converse 2026-07-22 19:16:28 -07:00
shivam
3fb2d32f67 test(sagemaker_chat): drop redundant first-delta guard that could mask failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 01:50:15 +00:00
devin-ai-integration[bot]
0c2b86e167
test(e2e): induce spend failure row deterministically instead of skipping (#34282)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 01:32:44 +00:00
yuneng-jiang
7ceba78b46
ci: run UI unit tests on a 16-core runner (#34330)
The UI vitest suite is CPU-bound; move it to a 16-core larger runner and raise vitest fork concurrency from 4 to 14 (leaving headroom for the coordinator, jsdom, and the OS) so the full suite and PR-scoped runs finish faster.
2026-07-23 01:27:28 +00:00
tin-berri
578b8e188c
Merge pull request #33884 from BerriAI/litellm_lit4581_passthrough_authorize_dcr
fix(mcp): fall through to an ephemeral DCR mint when passthrough authorize has no client_id
2026-07-22 18:23:31 -07:00
devin-ai-integration[bot]
90f2670918
test(e2e): require a cache-hit row instead of skipping when absent (#34283)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-22 18:18:20 -07:00
Tin Chi Lo
050cef5f1f fix(mcp): stop leaking upstream server credentials in tool-call 403
Calling an MCP tool on a server the key is not scoped to raised a 403 whose
detail interpolated the caller's allowed List[MCPServer] config objects, so
pydantic's default repr printed authentication_token, client_secret, the AWS
keys, client_private_key, env and static_headers straight back to the caller.
The two sibling denial sites already returned a bare message, so this one was
the lone outlier

MCPServer now renders only server_id, name, transport and auth_type in repr
and str, so a future f-string or log line cannot re-leak a credential field.
Field types and model_dump serialization are unchanged
2026-07-22 18:16:54 -07:00
Tin Chi Lo
9e7b1c0b36 feat(ui): land keyless users on the connect page after login
A standalone /ui/connect route is only reachable if something points a user
at it. Post-login the dashboard always rendered the API-keys view, so a
keyless SSO user saw an empty dashboard and no path to connect.

Redirect to /ui/connect from the dashboard landing when the URL carries
?login=success, the user is not an admin, and their key list is empty.
Gating on the post-login marker keeps the dashboard reachable afterwards,
and an explicit stored return URL still wins. useKeys takes an optional
enabled flag so the lookup only runs on that landing.
2026-07-22 18:15:37 -07:00
shivam
0ff3ade224 fix(sagemaker_chat): forward stream events as they arrive to cut TTFT
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 01:13:28 +00:00
Mateo Wang
9ab3847c0b
fix(tests): remove importlib.reload of http_handler that breaks client injection in later tests (#34336)
The huggingface embedding test fixture reloaded
litellm.llms.custom_httpx.http_handler, creating a new HTTPHandler class
object. llm_http_handler keeps the class captured at import time, so any
test running later in the same process that injects a client built from
the reloaded class fails the isinstance check and the mock is silently
discarded, causing a real network call. Under pytest-xdist loadscope this
surfaced as a deterministic failure of
test_accept_header_in_completion_request_jwt whenever an unrelated PR
shifted worker distribution.

Also removes the same reload pattern from the vertex rerank integration
test (both were previously removed in a6df01caec and resurrected by a
merge conflict resolution) and hardens the agentcore victim test by
dropping the bare except that swallowed the real error
2026-07-23 01:11:35 +00:00
Tin Chi Lo
c6a0ad5424 fix(mcp): mint an ephemeral OAuth client when passthrough authorize has no client_id
Resolves LIT-4581

A true_passthrough MCP server created without the at-creation auth step
has no stored client_id, and the tools-page browser flow supplies none,
so GET /v1/mcp/server/oauth/{id}/authorize dead-ended on a 400
missing_client_id. The client-forwarded-token modes forbid the gateway
from persisting an OAuth client, so client acquisition moves into the one
chokepoint every caller crosses: the authorize endpoint.

resolve_ephemeral_dcr_client owns the whole mint policy (mode gate,
authorization-url precondition, required S256 PKCE, redirect trust, then
a TTL-deduped, per-server single-flighted RFC 7591 mint). The minted
client rides the encrypted OAuth state; /callback seals it with the
upstream code and server_id into an llm_ptcode_ gateway code, and
redeem_passthrough_authorization_code recovers it at the token endpoint
(server binding plus required code_verifier) to authenticate the upstream
exchange. Nothing is persisted; every value rides the encrypted blobs, so
it works across replicas.

Client acquisition is one predicate applied across the whole auth-mode
matrix: the gateway mints for a clientless authorize iff true_passthrough
(any dcr_bridge) or oauth_delegate-and-not-dcr_bridge, and the UI
gatewayMintsClientFor mirrors that set exactly so the browser pre-registers
a client through the dcr_bridge front door only for the cells the gateway
does not mint (the interactive oauth_delegate dcr_bridge sign-in and the
legacy oauth2 passthrough). A minted flow runs the bridge short-circuit
arm; the relay front door stays for external clients that present their
own client_id. Both sides are pinned against the same truth table
(test_resolve_ephemeral_dcr_client_mint_set_is_exact and the
gatewayMintsClientFor matrix test) so no mode can silently diverge. The
authorization_code hook and M2M/token-exchange modes are unchanged.
2026-07-22 18:03:58 -07:00
Tin Chi Lo
f6fc6d299a feat(ui): add standalone /connect route for MCP OAuth
The MCP connect surface only existed as the Integrations tab inside the
enable_chat_ui-gated /chat shell, so a keyless SSO user was bounced to the
dashboard and could never reach it unless an admin enabled Chat UI first.

Add a sibling /connect route with its own thin, auth-only layout that renders
the same MCPAppsPanel without the chat-ui gate or chat shell. The user OAuth
flow already returns to whatever URL started it, so no backend changes are
needed. The chat playground and its gate are left unchanged.
2026-07-22 17:49:14 -07:00
ryan-crabbe-berri
9f6b3d24f7
test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895)
* test(e2e): cover key max_budget blocks on personal, team, and team-member keys

* refactor(e2e): convert budget enforcement cases to the resources-fixture pattern

The E2ECase class pattern existed only in this file; every other suite uses
plain pytest tests with the resources fixture. Rewrites the nine cases as two
spec classes and removes the now-dead E2ECase protocol and run_case driver
from lifecycle.py
2026-07-22 17:48:18 -07:00
yuneng-jiang
301a02b7be
chore(ui): bump next to 16.2.11 (#34329)
Moves the dashboard's next pin from 16.2.6 to the latest 16.2.x patch and bumps eslint-config-next to match. Regenerating the lock also healed in explicit bundled-dependency records under @tailwindcss/oxide-wasm32-wasi
2026-07-22 17:29:29 -07:00
yuneng-jiang
169ba0e287
refactor(ui): migrate transform-request to shadcn (#34303)
* test(ui): characterise transform-request panel behaviour before migration

* refactor(ui): migrate transform-request to shadcn

* fix(ui): keep transform-request panels within the fixed-height content fold

* fix(ui): let transform-request flow naturally so the shell scrolls instead of clipping

* test(ui): select the copy button by its accessible name
2026-07-22 17:01:41 -07:00
yuneng-jiang
692e6d48e9
refactor(ui): migrate old-usage to shadcn (#34304)
* test(ui): characterise the old usage page before migrating it

Role- and text-based coverage of the route as it behaves on Tremor, so the
shadcn migration has a regression net it did not get to write. Pins the
DISABLE_EXPENSIVE_DB_QUERIES branch (warning copy, the docs link and its
target, and that every expensive query is skipped), the admin vs non-admin
tab set, the cost cards, and the provider and customer tables

* refactor(ui): migrate old-usage to shadcn

Replaces Tremor with the installed shadcn primitives and the shared recharts
wrappers on the only file the route owns. Tabs, cards, tables, the key select
and the tag multi-select come from src/components/ui; the bar, area and donut
charts come from src/components/shared/charts. Tremor BarList has no shared
equivalent, so Total Spend Per Team is composed from ui/meter, which also means
the per-team totals stay numbers in state instead of pre-formatted strings; a
team total of 1,000 or more used to make the bar widths NaN.

The Database Query Limit Reached warning moves with it: same copy, same docs
link, still short-circuiting every expensive query.

Drops the file's no-restricted-imports suppression and the dead customTooltip,
getTopKeys, DataDict and UserData symbols. The characterisation test from the
previous commit is unchanged and green on both sides
2026-07-22 17:01:13 -07:00
yuneng-jiang
d7bde69b99
refactor(ui): migrate prompts list page to shadcn (#34289)
* test(ui): pin prompts panel toolbar and delete behaviour before migration

* refactor(ui): migrate prompts list panel to shadcn

* fix(ui): resolve prompts environment label and hold the delete dialog while deleting
2026-07-22 17:00:24 -07:00
yuneng-jiang
0b7b5100d0
refactor(ui): migrate api-reference to shadcn (#34263)
* test(ui): characterise the API reference page before the shadcn migration

Pins the behaviour the migration must preserve: the three SDK tabs and their
accessible names, the default selection, that selecting a tab surfaces that
SDK's snippet wired to the resolved base url, and the title, blurb and docs
link. Written against the current Tremor markup with role and text queries so
it carries over unedited.

* refactor(ui): migrate the API reference page to shadcn

Replaces the Tremor Grid, Text and Tab primitives on the API reference route
with the shadcn Tabs primitive and token utilities, and prunes the file's now
stale no-restricted-imports suppression. Markup only; the characterisation
test added in the previous commit passes unedited.

The wrapper keeps an explicit grid-cols-1 because Tremor's Grid defaults to
numItems=1 and emitted it; the implicit auto column that replaces it sizes to
the widest child and made the code block overflow the viewport.

* test(ui): scope the API reference snippet assertion to the selected tab panel

Asserts against the rendered tabpanel instead of searching every mounted code
block, so the check keeps proving the selected tab drives the snippet even if
the panels are ever kept mounted.
2026-07-22 17:00:06 -07:00
devin-ai-integration[bot]
d25bac5a41
fix(mcp): use official Google Drive streamable HTTP MCP server (#34322)
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-22 23:41:51 +00:00
devin-ai-integration[bot]
1ae406953c
feat(ui): edit fallback chains from router settings (#32841)
* feat(ui): edit fallback chains from router settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): address review nits on edit fallbacks modal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): fetch models via react-query in edit fallbacks modal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-22 16:31:33 -07:00