Merging litellm_internal_staging dropped 2 console.log calls (the
currentUser logs removed in #32079), so the no-console budget max and
metric move from 486 to 484 to match the current count.
Introduce a gradual ratchet to remove raw console.* calls from the
dashboard, mirroring the existing no-explicit-any budget.
The no-console eslint rule is set to warn with allow: [warn, error] so
the 486 console.log/debug/info calls are tracked without force-deleting
the legitimate console.error/warn error reporting in catch blocks. The
count is grandfathered via eslint-budgets.json (max 486, target 0) and
eslint-metrics.json, so any newly added console.log fails the budget
check and follow-up PRs grind the max down toward zero.
Independently, next.config strips console output from production builds
via SWC removeConsole (exclude: [error]), gated on NODE_ENV=production so
dev keeps full console output. This gives an immediate prod-hygiene net
regardless of how long the source cleanup takes. Verified against a real
production build: app-code console.log dropped from 675 to 14 in the
bundle (remainder is node_modules, which the transform leaves alone),
console.warn app calls stripped, console.error preserved 906 to 906.
The ui_unit_tests CircleCI job logged ~45k lines for a single run, most of
it React act() warnings, antd deprecation notices and component stack traces
emitted as console output by passing tests, which buried real failures.
Set silent: "passed-only" (Vitest 3.2+) gated on process.env.CI so console
output from passing tests is suppressed while a failing test still prints its
logs and full stack trace. Also drop two stray console.log calls in
UsagePageView that dumped the whole currentUser object on every render in
production, not just tests.
Verified by running the suite the way CI does
(CI=true npm run test -- --run --pool forks --poolOptions.forks.maxForks=6):
45,075 lines before, 981 after, all 4075 tests still passing. A throwaway
failing test confirms its console.log and assertion diff remain visible.
The ui_unit_tests job runs vitest with maxForks=8 on an 8-vCPU xlarge
container, leaving no headroom for the main vitest process that services
worker RPCs. Under full CPU saturation the coordinator misses the
onTaskUpdate ack, vitest raises "Timeout calling onTaskUpdate" as an
unhandled error, and the job exits 1 even though every test passes.
Lower maxForks to 6 so the coordinator, jsdom, and OS keep two cores, and
raise teardownTimeout to 60s for extra slack on heavy runs.
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server
Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain
access to every MCP server on the proxy without listing each id. The
sentinel expands to the live registry at request time, so a server added
later is picked up with no change to the team's stored permission. The team
ceiling that validates a key's MCP scope expands the sentinel too, so a key
can be scoped to any server (including one registered after the team) and
still pass subset validation
Expose the option in the team create and edit forms via a new exclusive
"All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing
"No MCP Servers" sentinel
* Update litellm/proxy/management_helpers/object_permission_utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key
The sentinel was expanded inside the shared expand_permission_list, which
also feeds the key, org, end_user and agent resolvers. A key whose stored
object_permission ever held all-proxy-mcpservers (a stale write, a
configured default, or a bug) would silently resolve to every MCP server at
runtime, and a teamless key had nothing to cap it, so all servers got
injected. Only write-time validation stripping the value stood between that
value and a full grant
Move the expansion out of expand_permission_list and into
_get_allowed_mcp_servers_for_team so the sentinel is honored only where it is
settable (a team). Anywhere else it now passes through as an inert literal
that matches no registered server and is denied downstream. Reserved-id
protection already blocks a real server from taking that id
* fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel
Granting a team every MCP server on the proxy is a proxy-wide authorization
decision, but team create/update let any caller who can manage a team set
object_permission.mcp_servers, with no ceiling check. Org admins reach
/team/update by default (org_admin_allowed_routes) and _verify_team_access
also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers
and self-grant their team access to every MCP server on the proxy, including
servers never assigned to that team
Gate the grant in new_team and update_team: a non-proxy-admin cannot add the
all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a
team a proxy admin already scoped to all-proxy can still be edited by a team
admin without being forced to strip the sentinel. The UI only offers the
"All Proxy MCP Servers" option to proxy admins in the team create and edit
forms
* fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team
A team scoped to the all-proxy-mcpservers sentinel could be opened in the team
edit form by a team admin or org admin (canEditTeam admits them), but the
"All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the
proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored
sentinel was hydrated into the selected value with no matching Select.Option, so
antd showed the raw all-proxy-mcpservers literal as a chip, and adding another
server could persist a mixed [all-proxy-mcpservers, <id>] value.
Render the option whenever the sentinel is present in the value, not only when
the caller may grant it, and drive the real-option disabling off presence too so
the selection stays exclusive. A non-proxy-admin now sees the friendly label
read-only and cannot build a mixed state; only a proxy admin can newly add it,
which the backend already enforces.
Adds regression tests: the selector shows the friendly option (not the raw
literal) when the sentinel is stored but the grant flag is off, plus exclusive
emit and disabled-real-options coverage, and MCPServerPermissions renders the
green "All" state instead of the raw sentinel string.
* fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip
antd renders a Select option's children inside the selected tag, so the
all-proxy option showed both "All Proxy MCP Servers" and the green "All servers"
type-hint in the chip, which say the same thing. Collapse the option to a single
green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly
without the duplication.
* fix(ui): color the all-proxy MCP label blue to match server chips
Use the same blue (#1890ff) as regular MCP server entries for the
"All Proxy MCP Servers" option/chip instead of green.
* fix(ui): make the all-proxy MCP permissions display blue, not green
Match the blue used by the selector chip and regular server entries so the
"All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent
across the team/key/org detail views. The red "Blocked" state for
no-mcp-servers is unchanged.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Address Greptile P1 (stale state after reset): use key prop to force
BudgetFallbacksEditor remount when parent resets budgetFallbacks to {},
matching the existing routerSettingsKey pattern.
Address Greptile P2 (inconsistent empty payload): guard budget_fallbacks
in edit view to only include when non-empty, matching create form behavior.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): persist DCR client_id so interactive OAuth token refresh works
Interactive authorization_code MCP servers register an OAuth client via Dynamic
Client Registration (RFC 7591) during the authorize flow, but the minted
client_id and the discovered token_url were returned to the caller and never
written to the server row. The autonomous refresh_token grant reads client_id,
client_secret and token_url off the server, so an expired access token could not
be refreshed; the user was bounced back to re-authorize and tools/list returned
zero tools
Persist the DCR client_id (plus client_secret and token_endpoint_auth_method when
the registration returns them) and the discovered token_url onto the server row,
reusing the encrypt_credentials write that client_credentials and token exchange
already use, then refresh the in-memory registry so the value is live at refresh
time. Both the v1 refresher and the v2 AuthorizationCodeRefresher read those same
fields, so egress needs no change
* fix: reuse persisted MCP DCR clients
* fix(ui): persist DCR client_id from on-create MCP OAuth "Authorize & Fetch"
The interactive "Authorize & Fetch" flow on the create form registers an OAuth
client (RFC 7591) against a temporary server that has no DB row, then creates the
real server afterward. useMcpOAuthFlow captured the DCR client_id and client_secret
but passed only the token to onTokenReceived, so the create request dropped the
client identity and the created server could not refresh its access token; its row
had credentials={} and the refresh_token grant 401d at the upstream token endpoint
Forward the registered client to onTokenReceived and write client_id (and
client_secret when present) into the create form credentials, so the create request
carries them and the backend persists them through its existing encrypt_credentials
path. token_url is omitted because it is re-discovered on load (RFC 9728 then 8414);
token_endpoint_auth_method is unused because this flow only ever registers as
client_secret_post or none, never client_secret_basic
* fix(ui): prevent stale MCP OAuth credentials
* fix(ui): reset MCP OAuth authorization state
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
PR #31635 added a per-server token_endpoint_auth_method (client_secret_basic
or client_secret_post) for upstream OAuth token endpoints, but it could only be
set by editing the stored credentials JSON. This surfaces it in the dashboard as
an optional selector directly under the Token URL field, in both the create form
(OAuthFormFields, M2M and interactive flows) and the edit form. The field binds
to credentials.token_endpoint_auth_method, which the backend already reads; the
value is sent only when chosen, so leaving it blank keeps the existing setting
and preserves the client_secret_post default.
* feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix
Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the
official upgrade codemod renamed utilities across 151 files, and
tailwind.config.js (plus the dead tailwind.config.ts) is replaced by
@theme tokens, @source globs, and @plugin directives in globals.css. The
Tremor safelist becomes @source inline patterns and the legacy tremor
theme tokens carry over verbatim. ui_colors.json was build-time only and
fed the dying Tremor palette, so its brand values are inlined and the
file removed; runtime theming replaces that path next.
shadcn is initialized with a hand-authored components.json (rsc,
cssVariables, baseColor gray) pointing utils at the existing
lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead
of adding class-variance-authority as a second variant library. The two
ad-hoc cn helpers fold into it. Button lands as the canary primitive,
adapted to cva beta and React 18 forwardRef, with tests covering the
variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the
shadcn radius scale reproduces Tailwind defaults and legacy rounded-*
classes render unchanged.
antd v5 emits unlayered CSS-in-JS that would beat every layered v4
utility, so AntdGlobalProvider now wraps the app in StyleProvider layer
and ConfigProvider cssVar, and globals.css declares
@layer theme, base, antd, components, utilities. antd wins over
preflight but yields to utilities, which is what lets migrated shadcn
pages coexist with legacy antd pages. Preflight stays global with the
three v3 behaviors pinned (default border color, button cursor,
placeholder color).
* fix(ui): restore tremor opacity tints removed by tailwind v4
Tailwind v4 removed the *-opacity-* utilities, but the precompiled
@tremor/react dist still composes them with shade-500 palette classes
(bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout,
light Icon and Button, BarList, and ProgressBar lost their tints and
rendered solid 500-shade fills. Adversarial review caught it; the
original smoke pages only exercised antd Tags.
tremor-v3-compat.css restores exactly the pairs tremor emits: for each
of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover
bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade,
via color-mix into the utilities layer. Tremor's colorPalette maps both
background and iconRing to 500, so the -500 pairing covers every
composition in the dist; dark: variants are inert until dark mode ships.
The shim dies with @tremor/react at the end of the migration.
The upgrade codemod also missed two hand-rolled modal scrims using
bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and
bg-black/50. Removed the docker/build_admin_ui.sh copy of
enterprise_colors.json into the deleted ui_colors.json; that build-time
rebrand path is retired and its runtime replacement lands with the
theming phase.
* fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim
Tremor's colorPalette maps ring to shade 300, and the only consumer of
ring-opacity-40 (Icon variant outlined) composes it with that shade,
so the shade-500 rows were dead and outlined icon rings would render
at full opacity. Latent today (no dashboard usage of the outlined
variant); caught by adversarial review. ring-opacity-20 stays at 500
(iconRing), matching Badge and BadgeDelta.
A 401 while listing tools (a missing or expired per-user OAuth token, or an
upstream 401 for any auth_type) was swallowed to an empty tool list, so a
single-server client got a 200 with no tools and no WWW-Authenticate challenge
instead of a 401 it could re-authenticate against. Only oauth pass-through and
delegate-to-upstream oauth2 servers surfaced it; every other auth_type, and the
missing-token case for all of them, masked it.
The surface-vs-absorb decision now keys on the route, not the auth_type. An
upstream 401 in _fetch_tools_with_timeout becomes an MCPUpstreamAuthError
regardless of auth_type, and the per-user OAuth challenge raised during client
creation (a bare HTTPException 401 carrying a WWW-Authenticate header) is
converted to the same type in _get_tools_from_server. The challenge is scoped
to 401: a 403 (authenticated but forbidden, e.g. insufficient scope) is not a
re-auth signal and degrades to an empty list like any other non-auth error, and
the stdio-allowlist 403 (no challenge header) stays absorbed. The existing
routing then does the right thing: single-server routes turn the error into a
401 + WWW-Authenticate, while the multi-server aggregator absorbs it to an empty
list so one unauthenticated server does not fail the whole listing.
On the UI tools page, an OBO (per-user authorization_code) server now shows the
Authorize gate when the list call returns 401, not only when no credential row
exists. The backend already refreshes a still-refreshable token on the list
call, so a 401 means there is no valid token and none could be minted (expired
with no usable refresh token), which is exactly when the user must reauthorize.
* refactor(ui): drive cache settings form from a typed frontend schema
The Cache Settings form was dynamically generated from field metadata
shipped by the backend, and read its values back out of the DOM with
document.querySelector. That loses type safety and makes client-side
validation awkward, which is a poor fit for a form whose shape only
changes when a developer edits code.
Move the field definitions (name, label, type, default, help text, which
redis type they apply to, section, and validation rules) into a typed
frontend module and render them through antd Form with controlled state.
The GET /cache/settings endpoint is still used to populate current values,
and the save/test payload shape sent to POST /cache/settings and
/cache/settings/test is unchanged. Per-field validation now lives on each
field's antd rules, so an inline error can surface before and on submit;
this is where the upcoming Redis URL validation will slot in.
The backend's fields output in GET /cache/settings is no longer consumed
by the UI, but is left in place since removing it is a separate backend
change.
* refactor(ui): validate list-field JSON inline so bad input blocks save
sentinel_nodes and redis_startup_nodes had no validation rule, so
malformed JSON passed validateFields, was caught while building the save
payload, and the field was silently omitted; the user's cluster/sentinel
config was discarded with no feedback. Add a jsonListRule (same shape as
portRule) to both list fields so an invalid value surfaces inline and
blocks save.
* fix(ui): show valid-JSON examples for cache list fields and clarify the error
The Startup Nodes and Sentinel Nodes help text showed Python-style
single-quoted examples (e.g. [{'host': '127.0.0.1', 'port': '7001'}]),
which the JSON validator correctly rejects, so pasting the example we
display failed. Switch both examples to valid JSON with double quotes and
change the parse-error message to "Must be a valid JSON array (use double
quotes)" so the hint points at the fix. Also add a regression test
asserting a numeric field (Database Index) is included in the save payload.
* fix(ui): validate numeric cache fields as text so bad input blocks save
Numeric fields (Database Index, TTL, Max Connections, Similarity
Threshold) rendered as antd InputNumber, which silently coerces
non-numeric input to empty. Because the fields are optional, an invalid
entry like a full connection URL pasted into Database Index passed
validation and was silently dropped from the save payload.
Render numeric fields as text inputs with a validation rule (non-negative
integer for Database Index and Max Connections, number for TTL and
Similarity Threshold), mirroring how Port already works, so invalid input
is preserved, flagged inline, and blocks submit instead of vanishing. The
save payload still coerces these to real numbers. Adds a regression test
for a non-numeric value entered into a numeric field.
* fix(ui): show info message when MCP tool preview returns 403
Internal users submitting MCP servers hit an admin-only preview endpoint; replace the red connection error with a clear review notice while leaving other failures unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): let BYOM submitters see their approved servers
Approved user-submitted MCP servers defaulted to no access groups and allow_all_keys=false, so submitters could not see them after admin approval. Grant creator visibility for active submissions in get_allowed_mcp_servers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Improve dialogue box
* fix(security): restrict MCP semantic filter settings to proxy admins
Add an explicit PROXY_ADMIN check on PATCH /update/mcp_semantic_filter_settings
and hide Semantic Filter and Network Settings tabs from non-admin users in
the MCP Servers UI.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(lint): use list[str] instead of List[str] to satisfy UP006 budget
Co-authored-by: Cursor <cursoragent@cursor.com>
* perf(mcp): cache BYOM submitter server lookup with 60s TTL
Co-authored-by: Cursor <cursoragent@cursor.com>
* style: fix ruff format and prettier formatting
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: preserve approved BYOM server visibility
* fix(mcp): keep no-mcp-servers opt-out absolute and gate BYOM union by key scope
The autofix in 94fd2bf made the no-mcp-servers sentinel return the caller's
submitted BYOM servers, which weakened an explicit key-level opt-out into a
soft preference. Restore the absolute opt-out and additionally skip the BYOM
union for keys with an explicit object_permission.mcp_servers list and for
toolset-scoped requests, mirroring how allow_all_keys servers are handled.
Add unit tests for the sentinel, explicit scoping, toolset scope, the cache
invalidation helper, the cache-miss DB path, and the db.py query helper.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
Moves the user-management component tree (view_users plus BulkEditUsers, edit_user, DefaultUserSettings, user_edit_view, and the view_users table/columns/info-view) out of the shared src/components dump into the users route segment under _components, now that the app router owns the route. The page imports from a trimmed ./_components barrel
UserInfo moves into networking.tsx beside UserListResponse, its real owner: networking defines the user API response shapes that embed it, and previously reached up into a view folder (components/view_users/types) to import the type. Defining it in networking removes that backwards data-layer-to-view dependency and drains the view_users/ folder entirely. CreateUserButton and onboarding_link stay in components/ since the create-key flow also consumes them
Relative imports in the moved files are rewritten to @/components/* absolute paths, and the eight pre-existing eslint-suppressions entries are re-keyed to the new paths so the move stays behavior and lint neutral
Verified: the moved suites pass with the same 75 assertions as before the move, tsc and eslint are clean, and next build compiles the /users route
Knip flagged remark-gfm as unused and date-fns as imported-but-undeclared, so drop remark-gfm (which prunes its transitive markdown subtree from the lockfile) and declare date-fns, which keyExpiryUtils.ts imports but only received transitively. Also delete the dead memory/components/index.tsx barrel, since nothing imports it once the page pulls MemoryView from its module directly
Knip itself could not run: its Playwright plugin imports every config referenced by a --config flag in package.json scripts, and migration.serverRootPath.config.ts threw at import time when SERVER_ROOT_PATH was unset. Move that guard into a config-specific globalSetup so importing the config is side-effect-free; the check still fires loudly before any test runs when the prefix is missing
* feat(ui): disclaim that the Update API Key modal only rotates api_key
An adversarial review of the credential-rotation work noted the modal always
writes litellm_params.api_key, so models that authenticate with an Azure AD
token, AWS credentials, or a Vertex service-account JSON are not rotated by it.
Adds a warning Alert to the modal so users are not misled into thinking those
secrets were rotated; broadening the modal to those providers is a follow-up
* Update ui/litellm-dashboard/src/components/update_model_credentials_modal.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* style(ui): prettier-format the credential modal
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* feat(mcp): add tool search virtual tools for large catalogs
When mcp_tool_search_enabled is set on a key's object_permission,
tools/list returns only mcp_tool_search and mcp_tool_call instead of
the full catalog. The LLM searches by keyword then calls discovered
tools by name, avoiding context bloat with 100+ tool deployments.
* fix(mcp): persist mcp_tool_search_enabled and route tool_call by name
The mcp_tool_search_enabled flag existed on the Pydantic models but the
Prisma schema lacked the column, so keys generated with the flag never
persisted it and tools/list kept returning the full catalog. Add the
column across all three schema.prisma copies plus a migration.
handle_mcp_tool_call passed server_name="" into call_tool, which built a
malformed prefixed name ("-<tool>") and failed to resolve the server.
Resolve the caller's allowed servers and dispatch through execute_mcp_tool
instead, matching how the normal /tools/call path routes.
* fix(mcp): filter list_tools to virtual tools on the protocol path
The REST surface (/mcp-rest/tools/list) returned only the two virtual
tools when mcp_tool_search_enabled was set, but the MCP protocol handler
(handle_list_tools, used by real MCP clients over streamable-http/SSE)
still returned the full catalog. Apply the same early return there so an
actual MCP client sees mcp_tool_search and mcp_tool_call instead of every
tool. call_tool was already intercepted on this path.
* fix(mcp): enforce IP + server filtering on virtual tool search/call
Review flagged that the virtual mcp_tool_search/mcp_tool_call path skipped
access controls the normal MCP flow applies. mcp_tool_call resolved allowed
servers from key permissions only, never applying IP filtering, so a caller
on a public IP could invoke a tool on a server marked
available_on_public_internet: false. mcp_tool_search listed the raw catalog
via global_mcp_server_manager.list_tools, exposing tool names/schemas that
/tools/list would hide and ignoring per-key/per-server tool filters.
Route both virtual handlers through the same filtered paths used by the
normal MCP flow: search now calls _list_mcp_tools and call resolves servers
via _get_allowed_mcp_servers, both threaded with the request client IP so
filter_server_ids_by_ip applies. execute_mcp_tool then enforces the server
allowlist and per-key tool permissions. Thread client_ip through
_list_mcp_tools/_get_tools_from_mcp_servers and pass it from the REST and
SSE call sites.
* fix(ci): ruff format server.py and sync dashboard API types
ruff format normalizes the list_tools client_ip changes in server.py, and
schema.d.ts gains the mcp_tool_search_enabled object-permission field so the
generated dashboard types match the proxy OpenAPI spec.
* style(mcp): drop quoted annotations and sort imports
Clears UP037 on the virtual tool handler signatures (redundant with
from __future__ import annotations) and I001 on the list_tools import block.
* refactor(mcp): extract virtual-tool dispatch and host progress capture
Pulls the mcp_tool_search/mcp_tool_call interception and the host
progress-callback setup out of mcp_server_tool_call into helpers, keeping
that handler under the strict cyclomatic-complexity ceiling after the
client_ip threading. No behavior change.
* test(mcp): cover SSE virtual-tool dispatch and host progress helpers
Adds unit tests for _dispatch_virtual_mcp_tool (non-virtual passthrough,
flag-disabled rejection, search/call routing with client_ip),
_capture_host_progress_callback, and the protocol list_tools virtual
early-return, covering the new server.py paths.
* fix(mcp): forward per-request auth headers through virtual tool handlers
The virtual mcp_tool_search/mcp_tool_call path intercepted the request
before the normal header extraction ran, so client-supplied per-request
auth (Authorization for upstream pass-through, x-mcp-auth-<alias>) was
dropped and execute_mcp_tool/_list_mcp_tools received None. Thread
mcp_auth_header, mcp_server_auth_headers, oauth2_headers, and raw_headers
from both the REST and SSE call sites through the handlers so upstream MCP
servers that require pass-through auth can be listed and called.
* fix(mcp): preserve requested server scope in virtual tool calls
A scoped MCP session (/mcp/<server>/ or header-scoped) carries an
mcp_servers scope that the normal call path passes into routing so the
session can only reach that server. The virtual-tool branch dropped it and
resolved with mcp_servers=None, letting a scoped session call mcp_tool_call
for any server the key can access. Thread the context mcp_servers scope
through _dispatch_virtual_mcp_tool into both handlers so search and call
resolve against the same scoped server set.
* fix(mcp): convert virtual tool errors to isError on the protocol path
The virtual-tool dispatch ran before the protocol handler's HTTPException
and guardrail handling, so a rejected virtual call (e.g. an out-of-scope
403 from execute_mcp_tool) raised out of mcp_server_tool_call and broke the
MCP JSON-RPC stream instead of returning an isError CallToolResult. Move
the dispatch inside the same try that wraps call_mcp_tool so virtual-tool
errors get the same isError conversion as normal tool calls.
* fix(mcp): spend-log virtual tool calls on the REST path
The REST virtual-tool branch returned before common_processing_pre_call_logic,
so execute_mcp_tool ran without a litellm_logging_obj and virtual mcp_tool_call
invocations were not spend-logged or guardrail-checked like normal calls. Run
the same pre-call pipeline in the call branch and thread the resulting
litellm_logging_obj through handle_mcp_tool_call into execute_mcp_tool.
* fix(mcp): reject virtual tool call when key has no accessible servers
handle_mcp_tool_call passed an empty allowed_mcp_servers list into
execute_mcp_tool; an unprefixed local tool name then fell through to the
local registry, which has no server permission check, so a key with only
mcp_tool_search_enabled and no server grants could run operator-configured
local tools by name. Reject with 403 before dispatch when no servers are
accessible, matching call_mcp_tool.
* docs(mcp): document virtual tool_search module and parity rule in AGENTS.md
* style(mcp): apply ruff format at repo line-length (120)
* fix(mcp): add mcp_tool_search_enabled to ObjectPermissionDict and customer test fixture
* chore: trigger CI
* fix(mcp): mirror pre-call pipeline, guard imports, coerce top_k, honor include_disabled_tools
- SSE mcp_tool_call now runs common_processing_pre_call_logic so it spend-logs and runs guardrails like the REST path (P1)
- coerce_top_k avoids ValueError on non-integer top_k from clients (both REST and SSE)
- guard mcp.types import in tool_search behind runtime/TYPE_CHECKING per package convention
- admin list with include_disabled_tools returns the real catalog even when mcp_tool_search_enabled is set
* feat(ui): add provider auth editing to the model edit view
Provider API keys / auth could previously only be changed by hand-editing
the raw litellm_params JSON, so there was no first-class way to rotate a
model's key. Adds an Authentication section that renders the correct
provider-specific fields (reusing ProviderSpecificFields) keyed off the
model's custom_llm_provider; fields are blank ("leave blank to keep
current") so untouched secrets are preserved and only entered values are
PATCHed and encrypted at rest.
Resolves LIT-3169
* refactor(ui): simplify model auth editing; fix stale credential branch
Drop the onFieldsResolved/authFieldKeys round trip: the parent now resolves
provider auth field keys itself via the new useProviderAuthFieldKeys hook
(same metadata ProviderSpecificFields renders), removing the report-up effect
and its stable-reference footgun. ProviderSpecificFields keeps only
excludeKeys (real need: suppress duplicate visible inputs).
Fix the stale Authentication branch: derive it from the live
litellm_credential_name form value (Form.useWatch) instead of the server
snapshot, so clearing/adding a credential mid-edit shows the right UI. Also
skip inline auth updates entirely when a named credential is selected, so we
never submit a credential name and raw inline auth together.
* fix(ui): don't leak freshly-entered model auth secrets to display/console
The auth values a user types are still sent in the PATCH request, but:
- strip them from the locally-stored litellm_params after save so the
read-only LiteLLM Params JSON doesn't render the plaintext key
- remove the debug console.log in modelPatchUpdateCall that dumped the
full update payload (incl. api_key / vertex_credentials) to the browser
console on every model update
Backend stores these encrypted and returns them masked on refetch.
* fix(ui): don't require blank auth fields in model edit context
Auth fields render blank ('leave blank to keep'), but required metadata
(e.g. OpenAI api_key) added a required validation rule that blocked
onFinish entirely — making it impossible to save any unrelated edit
without re-entering the secret. Add a disableRequired prop to
ProviderSpecificFields and set it in the model edit Authentication section.
* fix(ui): rotate model credentials in a dedicated modal so a normal save can't overwrite secrets
The model edit form seeded the read-only LiteLLM Params textarea with the whole
litellm_params blob and re-sent all of it on every save. Because /model/info
redacts secrets by masking them ("azur****BBCC") rather than removing them, any
save re-encrypted the asterisk mask over the real value and silently destroyed
credentials such as azure_ad_token, aws_session_token, watsonx token/zen_api_key
and the OCI key fields. api_key, client_secret, vertex_credentials and the AWS
access/secret keys were safe only because the backend strips those entirely
Credential rotation now lives in a dedicated UpdateModelCredentialsModal that
PATCHes only the fields the user types, decoupled from the params blob; the
backend already merges partial litellm_params, so the rest of the deployment is
left untouched. The general edit form drops masked values from both the textarea
seed and the outbound payload, so a normal save can never carry a redacted secret
Also removes the now-unused inline auth section and its excludeKeys and
useProviderAuthFieldKeys plumbing, strips secret-leaking console.logs from the
provider upload handler and the model-update response, and fixes a
react-hooks/use-memo error that was failing the frontend-lint CI job
* chore(ui): ratchet no-explicit-any lint metric to 2013
Removing the credential-echoing console.log (and its info: any param) from the
provider upload handler dropped the tracked count by one; update the committed
baseline so the Check lint budgets CI step is not stale
* refactor(ui): scope the model credential modal to api-key rotation only
Narrows UpdateModelCredentialsModal to a single API Key field. On submit it
PATCHes only { api_key }, so the backend merge leaves every other deployment
param untouched; a model authed via azure_ad_token, AWS keys, or a Vertex JSON
won't have anything to rotate here yet, which is the intended scope for now.
Drops the multi-field provider rendering this added earlier, which also removes
the now-unused disableRequired prop from ProviderSpecificFields and reverts that
shared component to its prior shape. The "Update API Key" trigger button is now
an antd Button rather than a TremorButton, so the feature introduces no tremor.
* refactor(ui): convert the model detail toolbar buttons from tremor to antd
Switches Test Connection, Re-use Credentials and Delete Model to antd Button so
the toolbar matches the Update API Key button and no longer mixes libraries;
Delete Model uses antd's danger styling instead of hand-rolled red classes
* style(ui): make the api-key modal submit button primary and drop the Need Help link
The e2e runs against the real proxy, so a contract drift already fails the test at
runtime; tsc only checks the spec against schema.d.ts, a generated snapshot, so a
backend change with a stale snapshot would pass tsc while the live test still
catches it. The dedicated tsconfig + script + CI step were circular ceremony for
that. Keep the zero-runtime-cost type-only import, which still catches mistakes in
the editor, and make its comment honest about what enforces the contract.
Address an adversarial review of the Loadbalancing e2e:
- The "typed against the backend schema" claim was hollow: nothing type-checked
e2e_tests (the root tsconfig excludes it and no CI step runs tsc), so a
contract drift would compile and run unchanged. Add e2e_tests/tsconfig.json, a
typecheck:e2e script, and a CircleCI step so the schema typing actually gates.
- The two describe blocks both mutate the proxy's shared router_settings, and the
Loadbalancing save echoes the whole settings object, so they could clobber each
other under local fullyParallel. Run the file serially.
- patchRouterSettings swallowed a failed seed, which surfaced later as a
misleading UI timeout. Assert the write succeeded, and rely on the server-side
merge instead of echoing the whole settings object back (drops a cast and a GET).
- Empty routing_groups already reproduces the bug, so drop the non-empty seed and
its model coupling.
Drives the real save flow against a live proxy: seeds a present routing_groups
array (the LIT-4057 trigger) via the typed /config/update contract, changes
num_retries on the Loadbalancing tab, and asserts the POST returns 200 instead
of 422, the success toast appears, and the value still shows after a reload (the
ticket's "refresh shows old values" symptom). The round-trip is typed against the
OpenAPI-generated backend schema (ConfigYAML write, RouterSettingsResponse read)
through a type-only import, so a backend contract drift fails the type check.
Address Greptile P2: the routing_groups test read setCallbacksCall.mock.calls[0][1]
immediately after the now-async save handler, so any latency in the mock would throw
an opaque TypeError instead of a clean assertion failure. Assert through
toHaveBeenCalledWith inside waitFor with expect.not.objectContaining, dropping the
index access and the cast. Also drop the ticket id from the test names.
The Loadbalancing tab rendered routing_groups as a generic text input and
sent its array value back as the JSON string "[]", which fails Pydantic
list validation on POST /config/update and returns 422. routing_groups has
its own dedicated Routing Groups tab, so this tab must neither render nor
write it; exclude it the same way retry_policy and model_group_retry_policy
are excluded for the Model Retry Settings tab.
The save was also fire-and-forget: setCallbacksCall was not awaited, so the
rejected promise escaped the try/catch and the success toast fired
unconditionally, showing success even when the backend rejected the change.
Await the call, gate the success toast on resolution, and surface the error.
* fix(ui): allow any git host on the skills add form (LIT-4053)
The skills add form only accepted GitHub URLs: its URL parser bailed on
any host that did not start with github.com, so GitLab, Bitbucket, and
self-hosted repos (and any repo subfolder on them) were rejected before a
request was ever sent. The backend already accepts arbitrary git hosts
via its url and git-subdir sources, with no host allowlist, so this was a
client-side restriction only.
Generalize the parser into an exported, host-agnostic parseSkillSource:
GitHub URLs keep their github / git-subdir shorthand, every other host is
treated as a raw repo url, and an optional Subfolder path field turns any
repo into a git-subdir source (url + path). When a pasted GitHub
tree/blob URL already encodes a subfolder, the field is cleared and
disabled so a contradictory source can never be submitted.
The parser is hardened to match the backend contract: query strings and
fragments are stripped, the host match is case-insensitive and drops a
leading www., the extracted and field-entered subfolder paths are both
validated against the same regex the server uses, a real file-extension
allowlist (not "any dot") decides whether a trailing blob segment is a
file, a branch-only tree URL falls back to the repo, non-GitHub URLs
require at least an org/repo, and the suggested skill name is kebab-cased
so it satisfies the name field's own rule.
The git-subdir source is now handled in the display helpers
(getSourceDisplayText, getSourceLink, formatInstallCommand), which
previously showed it as "Unknown source" with no link. The submit path
is fully typed (RegisterPluginRequest plus an AddPluginFormValues
interface), removing the two prior any usages; as a result an
author with an email but no name is dropped rather than sent, since the
backend requires the author name.
No backend changes. Tests cover the full host/subfolder matrix at the
parser level plus form-submit assertions on the exact source payload.
* refactor(ui): sync skill register types to the generated OpenAPI schema, surface backend errors
Replace the hand-maintained, already-drifted API types for the skills add
flow with the generated ones from schema.d.ts: PluginAuthor now aliases
components["schemas"]["PluginAuthor"], the registration payload is a new
SkillRegisterRequest (the generated RegisterPluginRequest envelope with
source narrowed to our PluginSource union, since the backend types source
as a loose string map, and version kept optional since the backend
defaults it), and the dead, mismatched RegisterPluginResponse is deleted.
registerClaudeCodePlugin's inline payload type (which was missing the
git-subdir path field entirely) is replaced with SkillRegisterRequest, so
the networking layer and the form can no longer drift from the backend.
Error handling: the add-skill form swallowed the real failure and always
showed "Failed to register skill". registerClaudeCodePlugin already
derives the backend message and throws it, so the form now surfaces it
("Failed to register skill: <reason>"), and the networking helper falls
back to the raw body / status when the error response is not JSON instead
of throwing a JSON parse error. A regression test asserts the backend
message reaches the user.
* fix(ui): reject credentialed git URLs on the skills form
A repo URL with embedded user-info (user:token@host) passed the raw-host
parser and was stored verbatim as the skill source, which is served on
the unauthenticated /public/skill_hub and marketplace.json feeds, leaking
the credentials. Reject any host segment containing '@'.
* fix(ui): validate skill repo URLs through one WHATWG URL gate
Replace the ad-hoc string parsing (stripScheme / splitHost / manual
scheme, @, ?# checks) with a single parseRepoUrl gate built on the URL
parser, so every malformed/unsafe class is handled in one place and the
URL stored on the public skill feeds is always canonical. It enforces
https (rejecting http/ssh/git/file/javascript/data and protocol-relative
//host), rejects embedded credentials (user:token@host, including
userinfo-confusion like github.com@evil.com), rejects IP-literal hosts
(loopback/private/metadata and obfuscated/IPv6 forms), and rebuilds the
stored url from origin+pathname so query strings, fragments, and trailing
slashes can never be published. The GitHub org/repo shorthand is now
charset-validated like the other paths, so junk can't reach the stored
repo. Closes both Veria findings (credentialed and http sources) plus the
adversarial-review follow-ups, with regression tests for each class.
* feat(proxy): type Customer Management response_model for OpenAPI coverage
Add response_model to the five remaining untyped /customer operations
(block, unblock, new, update, delete) so the generated OpenAPI schema
documents a concrete response body. new/update reuse the canonical
LiteLLM_EndUserTable (matching info/list); block, unblock, and delete
get small dedicated models in
litellm/types/proxy/management_endpoints/customer_endpoints.py.
Together with the already-typed info/list/daily-activity routes this
brings the Customer Management group to full response_model coverage.
Regression tests assert each public /customer/* route declares the
expected response_model and that /customer/new surfaces a typed schema
in app.openapi(), so dropping a response_model fails CI.
* fix(proxy): keep budget_id in typed customer responses
Address review feedback on the Customer Management response_model typing.
Greptile flagged that response_model=LiteLLM_EndUserTable on /customer/new
and /customer/update silently drops fields the raw Prisma model_dump()
echoed. Checking the schema, budget_id is the only such scalar column that
was missing from the Pydantic model (created_at/updated_at/tpm_limit do not
exist on litellm_endusertable), so add budget_id to LiteLLM_EndUserTable.
This restores budget_id on new/update and also fixes the pre-existing gap
where /customer/info and /customer/list (already typed) dropped it, which
the UI Customer type expects. A regression test pins budget_id surviving the
response_model filter on /customer/update.
Also document UnblockUsersResponse.blocked_users via a Field description: it
holds the users that remain blocked after the call. The key name predates
this PR and is kept to avoid a backwards-incompatible rename on a beta route.
* fix(proxy): keep nested budget fields in customer responses
response_model=LiteLLM_EndUserTable nests the budget as the narrow write
allowlist LiteLLM_BudgetTable, which silently drops the server-managed
fields the customer endpoints used to return (budget_reset_at, created_at).
Introduce CustomerResponse, a thin response model that nests
LiteLLM_BudgetTableFull (the repo's budget response model), and apply it on
/customer/new, /customer/update, /customer/info and /customer/list. list
also builds CustomerResponse so its budget isn't narrowed at construction
time. created_by/updated_at/updated_by remain omitted, matching how budgets
are returned elsewhere.
The shared LiteLLM_EndUserTable is left untouched: it's constructed in many
places that pass narrow budget instances, and pydantic v2 won't coerce a
budget instance into a wider nested model. Typing only at the response
boundary (where the handler hands FastAPI a dict) sidesteps that. A
regression test pins budget_reset_at + created_at through the filter and
asserts the internal audit fields stay out.
* test(proxy): add golden-master characterization tests for customer responses
Lock the exact JSON body each customer-object endpoint (info/list/new/update)
and delete return today, so the upcoming type-safety refactor of the handlers
is only allowed to land if it reproduces these byte for byte. Pins null-field
inclusion, the nested budget shape (server fields kept, audit fields dropped),
and object_permission reverse-relation stripping. Green against current code.
* refactor(proxy): make the customer response flow type-safe
Replace the untyped dict + bolt-on response_model pattern on the customer
object endpoints with explicit typed construction. A single mapper,
_to_customer_response, validates a DB row into CustomerResponse at one
Any -> typed seam; new/update/info/list now return it (or a list of it) and
carry real -> CustomerResponse / -> List[CustomerResponse] return
annotations, and delete returns DeleteCustomersResponse. basedpyright now
verifies the handlers' return shapes instead of a runtime filter doing it
silently.
This also deletes the four copy-pasted object_permission reverse-relation
cleanup loops: pydantic's extra=ignore drops those undeclared fields during
validation, so the loops were dead code (proven by the golden-master tests,
which stay byte-for-byte green). basedpyright errors on the file drop from
140 to 116, all from removed dict plumbing.
CustomerResponse stays a thin subclass of LiteLLM_EndUserTable so it inherits
the existing validators/config unchanged (behavior preservation); only the
nested budget type is widened.
* refactor(proxy): annotate customer response mapper param as BaseModel
Address review nit: the mapper's untyped `record` added an ANN001 violation.
The incoming rows are pydantic v2 models, so type the param as BaseModel
rather than object (object has no model_dump, which would just move the
problem to basedpyright). This clears the ANN001 and also drops three
basedpyright unknown-type violations the untyped param was adding.
* style(test): ruff format customer endpoint tests
* test(proxy): give customer budget test update mocks a valid model_dump
The type-safe response refactor validates the update result via
_to_customer_response (CustomerResponse.model_validate(record.model_dump())).
These budget tests mocked the end-user update to return a bare MagicMock,
so model_dump() yielded a MagicMock that fails validation. Give each update
mock a minimal valid dict; the tests assert on the prisma calls, not the body.
* chore(ui): regenerate API types from proxy OpenAPI spec
* fix(ui): make generated API types stable across Python versions
Python 3.13 strips a docstring's common leading indentation at compile
time while 3.12 keeps it, so app.openapi() emits differently-indented
description strings depending on the interpreter. The dashboard type
generator ran locally on 3.13 and in CI on 3.12, so schema.d.ts drifted
and the "Verify schema.d.ts matches the proxy OpenAPI spec" check failed
Normalize every description through inspect.cleandoc in the spec dump so
the output is identical regardless of interpreter, then regenerate
* chore: remove _experimental/out
* fix(ci): recreate _experimental/out before copying UI build output
The build scripts cp the Next.js output into litellm/proxy/_experimental/out,
which was removed from git. cp failed because the target directory no longer
existed; mkdir -p recreates it before the copy.
* fix(proxy): make UI serving resilient to a missing _experimental/out
Removing the committed UI export means the source/test tree no longer
ships litellm/proxy/_experimental/out. Three things assumed it was always
present and broke once it was gone:
- get_favicon hard-coded the built favicon path and 404'd without it; it
now falls back to the bundled swagger/favicon.ico
- the /_next and /ui static mounts raised at construction when the export
was absent, so the whole UI-setup block was swallowed and no mounts
registered; they now use check_dir=False
- _restructure_ui_html_files was a nested function only exposed as a
module attribute when that block happened to succeed; it is now a real
module-level function
test_admin_ui_export_serves_nested_extensionless_routes validated the
committed artifact, whose premise this PR removes; it now drives the same
MCP OAuth callback restructure guarantee through a synthetic export.
* chore(greptile): ignore generated _experimental/out so review fits the file limit
* Revert "chore(greptile): ignore generated _experimental/out so review fits the file limit"
ignorePatterns is applied after Greptile counts the files changed, so it
does not bring the diff under the file limit; the config had no effect.
Drop the explicit size on Request ID so it falls back to the default
width like the other reverted columns. Narrow Session ID from 160px to
120px since its truncated value needs less room
The explicit 90px/80px sizes were too narrow for the Duration (s) and
TTFT (s) headers once the sort arrows were factored in, cramping the
header labels. Dropping the size lets these two columns fall back to the
default width like before
* fix(anthropic): drop unsignable thinking blocks and allow null signature in logging (LIT-4007)
Open-source reasoning models (DeepSeek-R1 and distills, Qwen3/QwQ, IBM
Granite 3.2 via vLLM/Ollama/OpenRouter/DeepSeek) return reasoning_content
with no Anthropic-style signature, which LiteLLM represents as a thinking
block with a null signature.
Two failures resulted. First, ChatCompletionThinkingBlock.signature was a
required str, so building the StandardLoggingObject raised a ValidationError
on signature=None and the success log record was silently dropped while the
request still returned 200; relaxing it to Optional[str] lets the log build.
Second, replaying such a turn to a real Anthropic model forwarded the
null-signature thinking block unchanged and Anthropic rejected it with
400 thinking.signature.str; since Anthropic verifies the signature
cryptographically, a null, empty, or missing signature cannot be repaired,
so anthropic_messages_pt now drops the unsignable thinking block while
preserving the assistant text and keeping genuinely signed blocks.
* style: use builtin generics for thinking-block filter helpers
* fix(ui): regenerate schema.d.ts for nullable thinking-block signature
* perf(ui): load virtual-keys team filter from the fast v2 endpoint
The virtual-keys table sourced all teams through fetchAllTeams, which
hits the unpaginated /team/list. On a proxy with 125 teams that call
takes ~9.5s, so the Team ID filter and the team-alias/budget columns sat
empty for that whole window. The key list itself does not carry
team_alias or team_max_budget, so the table genuinely needs a team
lookup and cannot just drop the fetch.
Add useAllTeams, which pages the fast /v2/team/list to completion (~0.6s
per 100-team page, so ~1.2s for 125 vs ~9.5s), and point VirtualKeysTable
at it instead of fetchAllTeams. The allTeams shape, the filter searchFn,
the column lookups, and the loading indicator are all unchanged; only the
source endpoint changes. fetchAllTeams stays for its other callers.
* test(ui): tighten team-filter test readability and robustness
Address adversarial review of the added tests. Rename the single-page
useAllTeams test to match what it asserts (one request for a one-page
result) rather than implying it guards the early-return, and drop the
unread, misleading total: 125 from the mock page response. Scope the
created_by alias-over-email assertion to the key's table row so it checks
the visible cell value; the hover popover that also holds the email is
portaled out of the row, so the previous document-wide negative assertion
was relying on antd's lazy popover mounting.
* fix(ui): scope useAllTeams cache by access token
The previous /team/list query keyed on accessToken, so a user switch in
the same SPA session produced a distinct cache entry. useAllTeams dropped
that, so team IDs and aliases could be briefly reused across users until
the staleTime expired. Put accessToken back in the query key to restore
per-identity isolation, and add a regression test that a token switch
triggers a refetch rather than serving the cached list.
* fix(ui): keep virtual-keys filters across delete and refresh (LIT-4080)
Filtering virtual keys by User ID and then deleting a key reset the filter to show all keys, and re-clicking Fetch did not re-apply it. The page ran two competing fetch paths: useKeys (React Query) fetched the page unfiltered while a separate useFilterLogic hook held its own filteredKeys list and, on any refresh, only re-applied Team and Organization client-side, silently dropping the User ID and Key Alias filters. Delete refreshed through the unfiltered useKeys path, so the filtered view collapsed back to everything
VirtualKeysTable now owns its filter state and feeds every filter (team, organization, key alias, user id, key hash) straight into the useKeys options, so the filters are part of the React Query key. Any refetch or invalidation re-runs the same filtered query, which makes the reset-on-delete bug structurally impossible. Free-text inputs are debounced with @tanstack/react-pacer, sorting and pagination are server-side, and changing a filter or sort resets to page 1
Delete now invalidates keyKeys.lists() from key_info_view, matching the create path, instead of prop-drilling a refetch; the window "storage" refetch effect is removed. The dual-path useFilterLogic hook (and its test) are deleted
Regression coverage: VirtualKeysTable threads an active User ID filter into the useKeys query and clears it on reset, useKeys encodes filter options in its query key so a filter change refetches, and key_info_view invalidates the keys list on delete
* refactor(ui): simplify virtual-keys table data flow
VirtualKeysTable now fetches its own teams and organizations via
useOrganizations and the existing all-teams query instead of taking them
as props, so the prop-drill through UserDashboard and the two page
callers (page.tsx, ApiKeysDashboard) is gone along with their redundant
organization state and fetch
Filter state collapses from a useState plus a useDebouncedState mirror
into a single source whose debounced copy is derived with
useDebouncedValue, and one typed toKeyListFilters adapter maps it to the
key/list query options. Behavior is unchanged; same 300ms debounce and
the same reset timing
The unused onSortChange/currentSort props and their sync effect are
removed since no caller passed them, leaving sorting fully internal
Adds a created_by_user alias-over-email regression test that fails if the
display precedence is swapped
* test(ui): add required last_active to useKeys mock fixtures
The KeyResponse type requires last_active, so the typed mockKeys
fixtures were missing it. Add it so the file type-checks cleanly.
* chore(ui): ratchet lint budgets after virtual-keys refactor
Deleting filter_logic.tsx and simplifying VirtualKeysTable lowered the
no-explicit-any (2026 to 2016) and complexity (128 to 127) counts, so the
eslint-metrics.json baseline was stale and failed the frontend-lint
budget gate. Regenerate it, and drop the now-dead filter_logic.tsx
suppression entry for the file this PR removed.
* fix(ui): show a loading state for data-backed filter dropdowns
The Team ID and Organization ID filters source their options from async
hooks (teams / organizations). While that data was still loading the
dropdowns rendered 'No results found', so they looked empty rather than
loading. Add an opt-in loading flag to FilterOption that the searchable
select surfaces as a spinner and a 'Loading...' empty state, and wire it
from the teams and organizations query loading states. While loading, the
filter no longer caches an empty initial-options list, so the real
options appear once the data arrives.
* feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents
Bump a2a-sdk to 1.x and wire send/stream through compat conversions so the proxy accepts A2A 1.0 JSON-RPC while preserving 0.3 wire clients.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add user controlled protocol version in agents
* Fix exeception mapping
* Fix a2a base url
* Add e2e test for a2a
* Fix lint
* Fix lint
* fix(a2a): harden card version detection and header isolation coverage
Use protocolVersion when inferring agent card wire format, assert distinct httpx cache keys in the header-isolation test, and suppress targeted basedpyright errors for optional SDK imports.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): suppress reportArgumentType for SDK compat types and fix streaming trace ID
- Add pyright: ignore[reportArgumentType] to SendMessageSuccessResponse id= and
result= args in _send_message, and SendStreamingMessageResponse root= in
_stream_messages, where a2a-sdk compat types diverge from basedpyright's
inferred signature, reducing the reportArgumentType count back within budget.
- Fix streaming trace ID in astream_a2a_message to use str(request.id) when
available instead of always generating a new uuid4(), restoring JSON-RPC
request-ID correlation for observability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style(a2a): expand SendStreamingMessageResponse for black formatting
Move pyright: ignore comment to the root= argument line so Black
accepts the expanded multi-line form.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(a2a): fix 2 reportArgumentType errors without suppression
- main.py: narrow logging_obj from object|None to Optional[Logging] via
isinstance check before A2AStreamingIterator call, fixing the
"Logging | object" argument type mismatch at line 699.
- a2a_endpoints.py: extract response_dict with explicit isinstance(dict)
guard before passing to normalize_jsonrpc_response, fixing the
"LLMResponseTypes | dict[str, Any]" type mismatch at line 835.
- Remove spurious pyright: ignore comments added in previous commits that
were not suppressing the actual errors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(a2a): rewrite upstream URL for 1.0 agent cards in getAuthenticatedExtendedCard
1.0 upstream agent cards store the endpoint URL in supportedInterfaces[0].url
rather than a top-level url field. The previous guard only rewrote url when
it existed at the top level, so after normalize_agent_card lowered a 1.0 card
to 0.3 the upstream internal address leaked into the url field of the 0.3
response.
Fix: rewrite both url and supportedInterfaces[0].url to the proxy address
before calling normalize_agent_card, ensuring the upstream address is never
visible to downstream clients regardless of the upstream card's wire format.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: extend _served_version to all PascalCase methods; add direct httpx-client isolation proof
- _served_version now checks `_PASCAL_TO_WIRE` membership instead of two
hardcoded names, so GetTask/CancelTask/etc. are promoted to 1.0 wire format
alongside SendMessage — prevents mixed wire formats mid-session
- test_create_a2a_client_uses_fresh_httpx_client now asserts
a2a_client_a._litellm_httpx_client is not a2a_client_b._litellm_httpx_client
(direct proof that header bleed cannot occur), in addition to the cache-key
inequality check
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: id:0 silently dropped in version_convert; explicit continue in stream retry
- version_convert.py: replace `request_id or ""` with
`str(request_id) if request_id is not None else ""` in both
_send_result_to and _stream_result_to; id=0 is valid JSON-RPC and
must not be coerced to "" which breaks response correlation
- main.py: add explicit `continue` after the A2ALocalhostURLError retry
in _execute_a2a_stream_with_retry so the control flow (retry → next
iteration → stream_succeeded guard) is unambiguous
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: preserve a2a retry and discovery card urls
* Fix black
* Fix test
* fix(a2a): avoid KeyError in discovery log after 0.3→1.0 card normalization
When a 0.3-style agent card is normalized to 1.0, the top-level url key is
replaced by supportedInterfaces; log the already-computed proxy_url instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): preserve taskId when lowering push notification config set params
Flatten 1.x create envelope fields before parsing into TaskPushNotificationConfig so 1.0 clients forwarding to 0.3 upstream keep taskId and config.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): ignore unknown fields in message/send proto fallback
ParseDict in _build_message_send_params now matches other inbound paths so 1.0 clients with extra proto fields are not rejected with -32602.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): normalize tasks/list params and response across protocol versions
Convert list task entries on the response path and lower ListTasksRequest params including status filters when forwarding 1.0 clients to 0.3 upstream.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): avoid reportArgumentType in _lower_list_tasks_params; use local var instead of _parse return
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(a2a): drop private SDK symbol in tasks/list status lowering
_lower_list_tasks_params imported _CORE_TO_COMPAT_TASK_STATE, a private
a2a-sdk symbol that could disappear on a patch release and silently break
status-filter lowering. Derive the 0.3 wire string from the public
protobuf enum name instead (TASK_STATE_<NAME> maps to the 0.3 value once
the prefix is dropped and underscores become dashes) and validate the
result against the 0.3 TaskState enum's own values via a fully-typed pure
helper. Behavior is unchanged for every state; unspecified or unrecognized
states still drop the filter. Adds parametrized regression tests covering
dashed wire values (input-required, auth-required) and the unspecified drop.
* fix(a2a): drop redundant push-notification envelope key; unify MessageToDict import
_flatten_create_push_notification_params used `config or pushNotificationConfig`,
which short-circuits so a co-present pushNotificationConfig key was never popped and
leaked into the flattened params. Pop both keys unconditionally and prefer config
when present. Adds a regression test on the helper that fails on the old leak.
Also import MessageToDict from a2a.compat.v0_3.conversions in _lower_list_tasks_params
to match every other conversion helper in the module instead of pulling it straight
from google.protobuf.json_format.
* fix(a2a): reject invalid message/stream params early with -32602
_handle_stream_message built MessageSendParams lazily inside the
stream_response() generator, so malformed 1.0 params surfaced as a generic
-32603 after the 200 status line was already committed. The non-streaming
path validates up front and returns -32602 (Invalid params). Validate
eagerly before returning the StreamingResponse and emit -32602 on failure
so both paths reject malformed params identically. Adds a regression test
asserting the streamed error code is -32602.
* fix(a2a): raise clear error when non-streaming send ends on an update event
_send_message fed the SDK iterator's last event straight into
SendMessageSuccessResponse, whose result only accepts Message or Task. A
non-standard upstream whose final event is a TaskStatusUpdateEvent or
TaskArtifactUpdateEvent made the response construction raise an opaque
pydantic ValidationError. Guard the converted result and raise a clear
RuntimeError instead, consistent with the no-response guard above it.
Adds regression tests for the Message happy path and the update-event
rejection via an injected fake client.
* test(a2a): lock in clean merged agent-card URL without PROXY_BASE_URL
Regression coverage proving _build_merged_agent_card produces no double
slash in supportedInterfaces[0].url when PROXY_BASE_URL is unset and
request.base_url carries a trailing slash. get_custom_url routes through
join_paths, which rstrips the base, so the f-string join stays clean.
* style(a2a): modernize type annotations to satisfy strict ruff budget
After merging the black->ruff-format migration from base, the A2A files
owned by this PR still used Optional[X]/quoted annotations that pushed
UP037/UP045 over their lowered ceilings. Convert to X | None, drop the
now-unnecessary quoted local annotation in _send_message, and remove the
imports left unused by the rewrite. Type semantics are unchanged.
* style(a2a): type a2a_endpoints dict params as dict[str, Any]
The merge with the formatter-migration baseline tightened the
reportUnknownArgumentType ceiling; bare dict annotations made every value
Unknown and pushed the codebase total over cap. Annotate the JSON-RPC
params, body, metadata, and litellm_params dicts as dict[str, Any] so
their values are typed, dropping the unknown-argument count back under the
ceiling. No behavior change.
* fix(a2a): guard localhost retry against a missing agent card
handle_a2a_localhost_retry rewrote the card URL and called create_client
with whatever agent_card it received. The caller resolves the card from
the SDK client (Optional), so a None card reached set_agent_card_url and
create_client, surfacing an opaque SDK error instead of a clear one. Add
an early RuntimeError guard mirroring the httpx-client check, drop the now
always-true card None-check on the stash line, and cover it with a
regression test.
* style(a2a): disable reportUnknownArgumentType in a2a-sdk boundary modules
The lint env type-checks without the optional a2a-sdk/protobuf installed, so
every call into the protobuf-generated compat conversions counts as an
Unknown-typed argument and the new A2A code pushed the codebase
reportUnknownArgumentType total over its ceiling. These three modules are
the A2A SDK boundary; turn the rule off file-wide with a documented reason
instead of scattering dozens of per-line ignores across every SDK call.
* fix(a2a): tolerate unknown fields when lowering 1.0->0.3; align streaming trace id
Two issues greptile flagged:
version_convert: the 1.0->0.3 lowering paths (_send_result_to, _task_to,
_stream_result_to) called ParseDict without ignore_unknown_fields=True, so a
1.0 upstream response carrying vendor extensions raised and best-effort fell
back to passing the un-lowered 1.0 shape to a 0.3 client. Set the flag to match
the agent-card path and every inbound path; unknown fields are now dropped and
the result is correctly lowered.
main.py: asend_message_streaming derived X-LiteLLM-Trace-Id from the JSON-RPC
request id, unlike asend_message which uses the logging object's
litellm_trace_id. Prefer the logging trace id (then request id, then a uuid) so
streamed and non-streamed calls correlate under the same trace.
Adds regression tests for both, including the stream-event lowering path.
* style(a2a): apply ruff format to a2a protocol and proxy modules
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): persist global retry_policy via /config/update (LIT-3152)
The Admin UI Model Retry Settings tab POSTs
{router_settings: {retry_policy: {...}}} to /config/update, but the
field was dropped on two write-side layers so it never reached the
router. UpdateRouterConfig did not declare retry_policy, so
dict(exclude_none=True) stripped it before the DB upsert. And even when
fed directly, Router.update_settings had no "retry_policy" entry in
_allowed_settings, so the assignment was a silent no-op. The DB row
stayed at {"model_group_alias": {}}, llm_router.retry_policy stayed
None, and the UI fell back to defaultRetry = num_retries = 2 on refresh.
Declare retry_policy on UpdateRouterConfig as a plain dict, and add a
retry_policy branch to update_settings that coerces dict payloads to
RetryPolicy before setattr, mirroring Router.__init__. get_settings
already lists retry_policy, so reads work once writes land.
* fix(router): guard retry_policy type in update_settings
Mirror Router.__init__ semantics in update_settings: only assign
retry_policy when it is None or a RetryPolicy (after dict coercion).
Previously a non-dict, non-RetryPolicy value (e.g. a YAML typo like
retry_policy: 5 flowing through /config/update) was stored verbatim,
deferring the failure to request time in get_num_retries_from_retry_policy
instead of being dropped at write time.
* refactor(ui): harden Model Retry Settings flow and validate retry_policy at the boundary
Types UpdateRouterConfig.retry_policy as RetryPolicy and model_group_retry_policy as Dict[str, RetryPolicy] so /config/update validates the payload and rejects malformed counts instead of silently persisting them; the apply path in update_settings keeps coercing the stored dict back to RetryPolicy
Makes the Model Retry Settings tab the single owner of retry_policy and model_group_retry_policy so the generic Router Settings page no longer renders or writes them, replaces the fire-and-forget save with a react-query mutation that only shows the success toast after the write resolves, surfaces real errors, disables Save while in flight, and re-reads authoritative state on success, and sends both the global and per-group policies atomically so edits in the inactive scope are no longer dropped
Decouples the retry-scope selector from the All Models filter and defaults it to Global, seeds the displayed default from num_retries (falling back to 2), and gives per-group rows real inherit semantics so an empty input shows the global value as a placeholder with a Reset control, keeping 0 ("no retries") distinct from inheriting the global value
* fix(keys): align router_settings examples with typed RetryPolicy and resync UI artifacts
model_group_retry_policy is now Dict[str, RetryPolicy], so the {"max_retries": 5} sample in the key-generate test and the /key/generate and /key/update docstrings no longer validate; they now use a valid {"gpt-4": {"RateLimitErrorRetries": 5}} shape.
Regenerated eslint-metrics.json (no-explicit-any drifted 2027 -> 2026) and schema.d.ts (new RetryPolicy schema, retry_policy field, model_group_retry_policy value type) so the UI build and api-types-sync checks pass
* test(router): pin retry_policy persistence end to end (LIT-3152)
The existing retry_policy tests exercise UpdateRouterConfig and Router.update_settings in isolation, so they would all still pass if a regression flipped ConfigYAML.router_settings back to a loose dict or stopped add_deployment from applying the stored row. This drives the real handler chain an Admin UI save triggers: update_config writes the LiteLLM_Config row, the apply path forwards it to the live router, and get_config serializes it back, pinning retry_policy across persist, apply, and read-back.
* fix(teams): use valid model_group_retry_policy example in router_settings docstring
Same stale {"max_retries": 5} example the key endpoints carried; model_group_retry_policy maps a model group to a RetryPolicy, so the team /team/new and /team/update docs now show {"gpt-4": {"RateLimitErrorRetries": 5}}. Regenerated schema.d.ts to match.
* fix(ui): load retry settings via deferred fetch to satisfy set-state-in-effect
The Model Retry Settings effect called loadRetrySettings synchronously; eslint-plugin-react-hooks (react-hooks/set-state-in-effect) traces into it and flags the setState calls, failing frontend-lint. Split the loader into fetchRouterSettings + applyRouterSettings and run the fetch in an inline async IIFE with a cancellation flag, so state is applied in the post-await callback rather than on the effect's synchronous path. Behavior is unchanged and onSuccess still refreshes via loadRetrySettings.
* fix(ui): match CI rendering of RateLimitError 429 docstring in generated schema
gen:api run on a dev env (python 3.13 / newer fastapi) rendered the RateLimitError response description with 4-space indentation, but CI regenerates it with 8-space under its frozen python 3.12 toolchain, which is the canonical committed form. The Check UI API Types Sync job regenerates and diffs, so restore that block to the CI rendering; verified byte-identical to the pre-existing committed version.
* fix(ui): pin RateLimitError 429 docstring to CI's frozen schema rendering
Base #29619 regenerated schema.d.ts on a newer FastAPI that renders the RateLimitError response description at 4-space indent, but the Check UI API Types Sync job regenerates under the frozen python 3.12 toolchain, which renders 8-space. Merging base pulled in the 4-space form; restore the 8-space rendering so the generated types match what CI produces (verified byte-identical to the pre-#29619 committed form), which also corrects the base drift once this PR merges.
* fix(agents): show an agent's attached virtual key in the UI
The A2A agent detail view never surfaced which virtual key was attached to
an agent, so after assigning a key during agent creation there was no way to
see it again. Surface the attached key(s) in the agent detail view, derived
from the key table's agent_id foreign key the same way spend is already
joined into the agent response.
Backend adds an agent_id filter to /key/list (mirrors team_id) and enriches
GET /v1/agents and GET /v1/agents/{id} with a non-secret key summary (alias,
masked key_name, hashed token id). The frontend renders a Virtual Keys
section in the agent detail view that lists the agent's keys and links
through to the key detail, and the list view drops its fetch-500-keys-and-
filter-client-side workaround in favor of the enriched response. The orphaned
AgentCard and AgentCardGrid components, left behind when the agent list
switched from a card grid to a table, are removed
* fix(agents): redact attached virtual keys for non-admins
_attach_keys_to_agents joins keys onto the agent response by agent_id with
no caller scoping, but _redact_sensitive_agent_fields never cleared the new
keys field. A non-admin able to view an agent therefore received the alias,
masked name, and hashed token of every key attached to it, including keys
owned by other users or teams; the old client-side path used the scoped
key list, so this was a visibility regression. Clear keys in the redaction
path so only admins see attached-key metadata.
Adds an endpoint-level regression test asserting keys is populated for admins
and null for non-admins, and a list-view test covering the Active vs Needs
Setup badge that lost coverage when the agent card tests were removed.
* fix(agents): satisfy strict lint and resync key/list types
- use builtin list/dict generics in the new agent key helpers to stay
under the UP006 strict-rule ceiling
- swap @tremor/react for antd Typography in agent_virtual_keys (tremor is
being phased out; the new component was the only unsuppressed import)
- regenerate schema.d.ts so the /key/list agent_id query param is typed
* style(agents): prettier-format key hook test and agent_info
Now that the page-overflow bug is fixed by letting the main pane shrink, bring back per-column sizing purely to control widths. Columns declare explicit pixel sizes and the table derives its min-width from getCenterTotalSize(), so it stretches to fill a wide card but scrolls once the columns no longer fit. The shared DataTable applies this only when columns declare sizes, leaving the other consumers on their existing fluid layout
Trim the columns that were eating horizontal space without earning it: Request ID and Key Hash drop ~30% (Key Hash now narrower than Key Alias, which is the more useful of the two), and Duration and TTFT shrink to fit their short numeric values
