Commit graph

6343 commits

Author SHA1 Message Date
yuneng-jiang
cd87fee9c5
feat(team): custom metadata validation hook for team create and update (#33353)
* feat(team): custom metadata validation hook for team create and update

Operators can point general_settings.custom_team_metadata_validate at an
async Python function that validates team metadata before /team/new,
POST /team/update, and PATCH /team/{team_id} commit their writes. The
hook receives the metadata that will actually be written (the merged
result on PATCH) plus the stored metadata and requester context, and
fails closed: a rejected value returns the function's own message as a
400 while any exception or timeout blocks the write with a configurable
generic message as a 503. Premium-gated like enforced_params.

* fix(team): validate metadata before model alias writes and strip system keys from validator input

Review follow-ups on the team metadata validation hook: run the validator
before the model_aliases table insert so a rejected create leaves no
orphaned model rows, strip system-managed keys from existing_metadata so
the validator sees symmetric input on both fields, and accept class
instances exposing an async __call__ as validators. Adds a three-way
validator implementation matrix (allowlist function, HTTP-service-backed
function, immutability-enforcing class instance) driven through the real
create, update, and patch endpoints, including an HTTP stub service and
outage coverage.

* test(team): run the metadata validation matrix against the DB-backed proxy in CI

Adds the validator matrix to the proxy_store_model_in_db_tests CircleCI
job so every scenario runs full e2e against a Postgres-backed proxy. The
proxy config registers a dispatching validator that routes each request
to one of the three implementations via a metadata key and accepts
anything that does not opt in, keeping the rest of the suite unaffected.
CI starts a stand-in cost center service on the host for the HTTP-backed
implementation, reached from the container via host.docker.internal, and
the outage path targets a closed port to prove the fail-closed 503
without stopping services.

* feat(ui): edit team metadata as key-value pairs in team create and edit forms

The team create and edit forms asked for metadata as a raw JSON blob in a
textarea buried under Additional Settings. Both forms now render a key-value
pair editor directly under the TPM/RPM limit fields, backed by a shared
MetadataKeyValueFields component. Values round-trip losslessly: non-string
values display as JSON and parse back to their typed form on save, and
JSON-ambiguous strings are quoted so their type survives the trip. The edit
form hides UI-managed keys (logging, guardrails, model rate limits, etc.)
that dedicated controls already own and re-add on save.

* fix(ui): explain typed JSON parsing in the team metadata help text

* feat(team): schema-driven metadata fields from team_metadata_schema config

* refactor(team): render schema metadata fields as locked key-value rows, drop allowed_values

* refactor(team): schema fields reduce to key and label, tag-rendered keys, clean rejection toasts

* refactor(ui): prepopulate declared metadata keys as ordinary key-value rows

* fix(team): let non-admin dashboard users read the team metadata schema

* test(proxy): pin timeout wiring, boundary, and error-message contracts for team metadata validation

* fix(proxy): use pooled async httpx client in the e2e team metadata validator example

* refactor(team): satisfy staging lint ratchets inherited by the merge
2026-08-03 18:37:45 -07:00
ryan-crabbe-berri
d4d0bf0acc
fix(ui): hide guardrail review buttons from non-admin users (#27535)
Some checks failed
Unit Tests: LLM Provider Transformations / All Other Providers (push) Has been cancelled
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Has been cancelled
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-server (push) Has been cancelled
Unit Tests: Proxy Infrastructure / proxy-infra (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / key-generation (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-config (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-server (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Has been cancelled
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Has been cancelled
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Has been cancelled
GitHub Actions Security Analysis / zizmor (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
* fix(ui): hide guardrail review buttons from non-admin users

The team guardrail submissions list rendered Approve/Reject buttons for
non-admin users even though the backend correctly rejected the calls.
Thread userRole from the page through GuardrailsPanel into
TeamGuardrailsTab and gate the row-card and detail-panel review buttons
on isAdmin so the UI matches the backend authorization.

Defense in depth only — the backend remains the source of truth and is
double-gated at both the route admin check and the explicit endpoint
role check.

Refs LIT-2494

* refactor(ui): read userRole from useAuthorized hook instead of prop drilling

Drop the userRole prop chain through GuardrailsPage → GuardrailsPanel →
TeamGuardrailsTab. Each component reads userRole directly from the
useAuthorized hook, matching the pattern used elsewhere in the dashboard.

Tests now mock useAuthorized per case (the same pattern as
top_key_view.test.tsx) instead of passing userRole as a prop.

Refs LIT-2494

* fix(ui): drop userRole prop on GuardrailsPanel call site in src/app/page.tsx

Missed in the earlier refactor — GuardrailsPanel no longer accepts
userRole as a prop (reads from useAuthorized hook), so callers must
not pass it. The build was failing in production type-check.

Refs LIT-2494

* fix(ui): gate guardrail forward-key toggle and header editors on proxy admin

* refactor(ui): remove dead app_admin case from user role formatting
2026-08-03 18:09:49 -07:00
Yuneng Jiang
60259d7986
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/spend-reports-implementation-25a080 2026-08-03 17:42:01 -07:00
yuneng-jiang
c6a796a84b
Merge pull request #35718 from BerriAI/litellm_/repo-fix-verify-925b5b
fix(ui): render Responses API request and response in the logs drawer
2026-08-03 17:26:22 -07:00
Yuneng Jiang
722d9ffa4f
feat(spend): add caller-scoped key/user/team/organization spend report endpoints 2026-08-03 17:22:11 -07:00
Yuneng Jiang
9c2c79f976
fix(ui): render Responses API request and response in the logs drawer
The Pretty view only parsed the Chat Completions shape (messages /
choices[0].message), so any spend log storing the Responses API shape
(input / output) rendered an empty Input card and the literal text
"No response data available" even though the row held the full request
and response. This also hit plain /v1/chat/completions callers, because
litellm may route those over the Responses bridge and then store the
upstream Responses-shaped body.

Parsing now branches on a tagged union covering both shapes, which also
replaces the any-typed key sniffing and the role guessing it relied on.
2026-08-03 16:26:01 -07:00
Yuneng Jiang
cd3b7ef427 test(ui): tier the MCP create tests into unit and integration
Adds 61 unit tests on the modules #35694 extracted: 46 on the payload
builder, 15 on the OAuth redirect snapshot. They run in 9ms against 240s
for the 77 full-render tests they partly replace. Nine of nine mutants
were killed when the extracted logic was deliberately broken, so the
speed does not come at the cost of signal.

Deletes six cases across four blocks that rendered the whole modal to
assert one payload key belonging to a field they never touched. Every
test that proves a form field reaches the right payload key stays; those
cover field to form value to payload, which a unit test cannot reach.

Replaces "should not render when user is not an admin", which asserted
the admin title was absent and so passed for the wrong reason: the modal
does render for a non-admin, retitled. registerMCPServer was mocked but
never asserted anywhere, leaving the whole non-admin submission path
uncovered. It now drives a real submit and asserts the call lands there
and never on createMCPServer.

Renames the slow file to CreateMCPServer.integration.test.tsx and
documents the three tiers in the dashboard CLAUDE.md. No production code
changes.
2026-08-03 16:22:54 -07:00
yuneng-jiang
47d2e225b7
Merge pull request #35694 from BerriAI/litellm_mcp_create_extract
refactor(ui): extract the MCP create form's logic and field groups
2026-08-03 16:22:50 -07:00
tin-berri
8ad5d144a1
feat(complexity_router): default session affinity off and expose it in the UI (#35714)
* feat(ui): expose an Auto-Router session affinity toggle

session_affinity on ComplexityRouterConfig defaults to True, and neither the
create form nor the edit modal ever emitted the key, so every auto-router built
in the UI silently pinned each session to its first turn's model for an hour
with no way to see or change that.

Adds an "Advanced: Session Affinity" switch to both surfaces, defaulted on to
match the backend field. Both paths now write the key explicitly instead of
falling through to the backend default, so a stored config states what the
router actually does. A stored config with the key absent hydrates as on, since
those routers are running with affinity enabled today; showing them as off would
report the opposite of reality and persist it on the next save.

* feat(complexity_router): default session affinity off and expose it in the UI

session_affinity defaulted to True and the Auto-Router UI never emitted the
key, so every router built there silently pinned each session to whatever model
its first turn classified into for an hour, refreshed on every hit. There was
no way to see that from the UI and no way to change it without hand-editing
config.yaml.

The default flips to False, so every turn is classified on its own merits and
lands on the cheapest adequate tier. Pinning is now opt-in.

The toggle added in the previous commit follows the field: it renders off, and
both the create tab and the edit modal keep writing the key explicitly, so a
stored config states what the router does instead of inheriting a default that
can move under it.

Behavior change for existing routers: those created before this have no
session_affinity key stored, so they pick up the new default and start
reclassifying every turn. That gives up the provider prompt cache the pin was
preserving, and a multi-turn session can now change model between turns. Set
session_affinity: true to keep the old behavior.
2026-08-03 15:18:27 -07:00
ryan-crabbe-berri
b7843193a0
chore(ui): note Google's Agent Platform rename in vector store setup (#28076)
Google Cloud has renamed Vertex AI RAG Engine to "RAG Engine" and
Vertex AI Search to "Agent Search" in its console. Users following our
setup instructions hit a naming mismatch when they cross-reference the
GCP console. Keep "Vertex AI" as the primary term (the generic new
names would make our provider UI ambiguous) and surface the new names
as secondary asides only where users leave the UI for the console.

Resolves LIT-3081
2026-08-03 14:27:40 -07:00
Yuneng Jiang
b03803b918 refactor(ui): extract the MCP create form's logic and field groups
Pulls four modules out of the 1398-line create component, which drops to
896 lines. No behavior changes: CreateMCPServer.test.tsx is untouched and
all 77 of its tests pass against the refactored component, which is the
review contract for this PR.

createServerPayload.ts is a pure form-values-to-payload function whose
failures are a tagged union instead of inline notification calls, so the
transformation is reachable without a DOM. createOAuthUiState.ts owns the
snapshot that survives the OAuth authorize redirect, keeping every
presence guard the inline version had. AwsSigV4Fields and
OpenApiByokFields are the two largest JSX blocks, moved verbatim so they
can be diffed as moves.

The create/edit setToken divergence, the mcpLogoImg export, and the
untyped form-values bag are left alone on purpose; each is a behavior or
cross-file change that does not belong in a move.
2026-08-03 13:39:10 -07:00
yuneng-jiang
9d5984b358
refactor(ui): rename the create MCP server component to PascalCase (#35686)
Pure rename, no behavior change. create_mcp_server.tsx and its test move
to CreateMCPServer, the two importers and one stale e2e comment follow,
and the local/filename-pascal-case suppression drops now that the file
passes the rule on its own.

The rename is scoped to this one component rather than the whole
directory because three PRs are currently open against its snake_case
siblings; the rest can follow once those land.
2026-08-03 13:39:07 -07:00
Yuneng Jiang
da648d44e7
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/npm-vulnerabilities-litellm-05f3f7 2026-08-03 13:19:05 -07:00
Yuneng Jiang
3bc4989ce4
chore(ui): update brace-expansion and postcss to current patch releases
The dashboard pins both packages exactly in `overrides`, so the lockfile
stays on whatever those pins say. Move brace-expansion from 5.0.8 to 5.0.9
and postcss from 8.5.22 to 8.5.23, both upstream patch releases, and
regenerate the lockfile.

`npm ci`, `next build`, and the 5888-test vitest suite all pass on the
updated lockfile.
2026-08-03 13:15:32 -07:00
ryan-crabbe-berri
b13e4eee5d
fix(ui): block Playground page for viewer roles on direct URL access (#35676) 2026-08-03 13:03:46 -07:00
ryan-crabbe-berri
46b6eae799
feat(teams): apply default organization to new teams from default team settings (#35540)
* feat(teams): apply default organization to new teams from default team settings

Adds organization_id to DefaultTeamSSOParams so proxy admins can pick a
default organization in Default Team Settings. new_team applies it before
org validation whenever a team is created without an explicit
organization_id, so API, Admin UI, SCIM, SSO, and team upsert creations
all inherit it and go through the same existence and org-limit checks.
Explicit organization selections win and existing teams are untouched.

The default is validated at save time (PATCH /update/default_team_settings
returns 400 for an unknown org) and at create time, where a missing org now
surfaces as a clean 400 instead of a 500 by routing OrganizationNotFoundError
into the previously dead org_table None guard.

The Admin UI Default Team Settings tab gets a Default Organization row
backed by the shared OrganizationDropdown.

* fix(teams): validate org limits against final team state including defaults

Applies default_team_params and the legacy max_budget fallback before the
organization validation block, so _check_org_team_limits sees the values the
team will actually be persisted with. Also loads the org's budget table in
the lookup; without include_budget_table every budget comparison in
_check_org_team_limits was skipped because litellm_budget_table was None.

* test(proxy_behavior): pin org team limits as enforced on /team/new

The dead-code pins existed to turn red when include_budget_table went
live; that happened, so the scenarios now assert the 400 rejections plus
within-cap acceptance, and the unknown-org pin asserts the handler's 400
instead of the surfaced 500.
2026-08-03 12:57:12 -07:00
ryan-crabbe-berri
41e4408906
feat(playground): add non-streaming response toggle (#35560)
Adds a Stream responses checkbox (default on) to the playground Model
Settings popover. When unchecked, chat completions and responses API
requests are sent with stream: false and the full reply renders at
once. The non-streamed result is replayed through the existing
streaming handlers as synthesized chunks/events so MCP events, vector
store results, usage and response ids behave identically in both
modes. TTFT is suppressed when not streaming; total latency now also
reported for the responses API. The toggle is scoped to the chat and
responses endpoints, persists via sessionStorage, and is isolated from
the simplified Agent Builder chat.

Resolves LIT-3251
2026-08-03 12:55:39 -07:00
yucheng-berri
7c8364c991
fix(team-callbacks): actually stop logging when disable_logging is called (#35520)
disable_team_logging cleared only metadata["callback_settings"], but callbacks
registered through POST /team/{team_id}/callback and the Admin UI live in
metadata["logging"], and request-time resolution stops at that slot without
ever reading callback_settings. The endpoint reported success while the team
kept sending request and response data to its third-party destination.

Empty the logging slot alongside the existing callback_settings reset, and
refresh the cached team object so the change applies to keys that are already
in flight rather than at the next cache expiry. The same refresh is added to
add_team_callbacks, which has the symmetric problem of a newly registered
callback staying dormant until the entry expires.

Resolves LIT-5101
2026-08-03 11:02:51 -07:00
yuneng-jiang
ceaf556b2e
Merge pull request #35523 from BerriAI/litellm_ui_login_no_mcp_landing
fix(ui): land general login on the keys dashboard, send MCP consent to /ui/connect
2026-08-01 17:17:18 -07:00
mateo-berri
23d26d5e64 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit4395_cursor_agent
# Conflicts:
#	litellm/completion_extras/litellm_responses_transformation/transformation.py
#	litellm/litellm_core_utils/llm_response_utils/convert_dict_to_response.py
#	litellm/litellm_core_utils/prompt_templates/common_utils.py
#	litellm/litellm_core_utils/streaming_chunk_builder_utils.py
#	litellm/llms/openai/chat/gpt_transformation.py
#	litellm/main.py
#	litellm/proxy/response_api_endpoints/endpoints.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-01 17:06:53 -07:00
yucheng-berri
7c3b578e77
fix(team-callbacks): report API-registered callbacks from GET /team/{team_id}/callback (#35512)
* fix(team-callbacks): report API-registered callbacks from GET /team/{team_id}/callback

POST /team/{team_id}/callback writes metadata["logging"] while the GET read
metadata["callback_settings"], so every team configured through the API or the
Admin UI got back an empty list. c620d76fe4 migrated the writer to the new key
and left this reader on the old one.

Resolve the read the same way request-time resolution does in
_get_dynamic_logging_metadata: a logging slot that is present wins outright and
callback_settings stays as the deprecated fallback, so the endpoint reports what
a request would really do rather than the union of both shapes. An empty logging
list therefore reports no callbacks, matching a request that fires none.

Decrypt callback_vars for the response and mask the credential keys. Ciphertext
would be unusable to the caller, and a value encrypted under a key that is no
longer classified as sensitive would otherwise come back as a raw blob.

Resolves LIT-5093

* Update litellm/proxy/management_endpoints/team_callback_endpoints.py

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(team-callbacks): mask callback vars that fail to decrypt

decrypt_callback_vars passes a value through untouched when it cannot be
decrypted, which happens to existing rows after a salt-key rotation. Under a
key that is not classified as sensitive that blob reached the caller as opaque
ciphertext it could not use or tell apart from a real value, so mask anything
still carrying the encrypted prefix.

Raised by Greptile on the first commit.

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-01 17:02:40 -07:00
mateo-berri
85f367b545 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_up035_abc_imports
# Conflicts:
#	litellm/llms/bedrock/base_aws_llm.py
#	litellm/proxy/proxy_cli.py
#	litellm/proxy/proxy_server.py
#	litellm/repositories/model_repository.py
#	litellm/router.py
2026-08-01 16:18:04 -07:00
Tin Chi Lo
7194cafbc1 fix(ui): land general login on the keys dashboard, send MCP consent to /ui/connect
A keyless internal user signing in to the Admin UI was redirected off the
post-login landing to /ui/connect, which renders nothing but the MCP apps panel,
so a plain gateway sign-in ended on an MCP OAuth surface the user never asked
for. The landing now renders the keys dashboard for every role. The key lookup
that existed only to make that routing decision goes with it, along with the
useKeys enabled flag it was the sole caller of and the role-hydration hold that
guarded its one-frame dashboard flash

The gateway DCR consent flow moves the other way. Its /authorize handed the
browser to /ui/chat/integrations, whose layout hard-blocks when enable_chat_ui
is off, which is the default, and client-side redirects to /ui/ without the
query string; that destroys the connect_flow handle and strands the MCP client
until the 600s flow cookie expires. It now lands on /ui/connect, which reads
connect_flow and connect_client, mounts the consent banner and puts the apps
panel in connect mode. /ui/chat/integrations keeps its connect-mode handling
this release so flows sealed before the deploy still finish

Resolves LIT-5104
Resolves LIT-4911
2026-08-01 16:09:30 -07:00
mateo-berri
b604e2b20c refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets
About 35,000 fixes ruff marks safe across 32 rules (UP006/UP045/UP007
modern annotations, UP032 f-strings, SIM114/SIM118, RET501, and
friends), removal of the 1,296 typing imports the rewrite orphaned, and
hand fixes for what the fixers could not see: five star-import
freeloaders of typing names, two F823 late-import annotations, the
/get/config/list introspection crash on types.UnionType, redundant
function-local RoleMappings imports in ui_sso.py that shadowed the
module-level name once the annotation lost its quotes, and one FURB168
tautology.

B009/B010/PIE804/RUF019 are excluded on purpose: their safe fixes
rewrite getattr/setattr/**-splat/key-in-dict escape hatches into forms
basedpyright then rejects (283 new errors measured), so their budgets
stay at base values.

ruff-strict-budget.json drops by 39,579 this commit (39,968 across the
branch) with 28 rules at an actual 0 and 9 more sharply down.
type-discipline-budget.json ratchets LIT002/LIT006/LIT009 down; LIT001
moves to the now-honest total: the checker matches the spelling `set`
but not the alias `Set`, so the 160 typing.Set annotations rewritten to
set[...] were always mutable-set annotations and only now count.
2026-08-01 15:43:29 -07:00
tin-berri
787a863022
feat(ui): expose the assistant-turn classifier context switch on Auto-Router screens (#35500)
PR #35471 added classifier_context_include_assistant_turns to ComplexityRouterConfig.
It worked through config.yaml and the model API but had no control on the Add Model or
Edit Auto-Router screens, so an operator working from the dashboard could not reach it.
Wires it into the create and edit forms, shown only when the LLM classifier is
selected, matching what #35315 did for the two context-window fields

The create and edit stacks share the rendered control but keep their own serializer,
their own hydration, and their own managed-key set, so the field is added in five
places rather than one. A field wired into only one stack fails in a way neither
serializer unit test can see, since those are handed a form value assembled by hand,
so the edit-modal test drives the real component through open, edit and save

The switch is emitted even when false, because there the operator turning it off is a
choice that has to overwrite a stored true rather than an absent value a truthiness
gate would drop
2026-08-01 15:38:32 -07:00
Tin Chi Lo
ebe48d67de fix(proxy): normalize each tool shape level independently on the Cursor messages arm
Live Cursor Ask-mode captures show the shape dialects mix PER LEVEL: the
tool envelope arrives chat-nested while the grammar format inside it is
still Responses-flat, so a normalizer that pattern-matches whole-tool
templates misses every hybrid. The cursor arm now normalizes the envelope
level and the format level independently and idempotently, making it
total over the envelope x format matrix; a parametrized 8-cell test pins
every combination. The reference BYOK bridge was checked and forwards
chat bodies verbatim, so there is no prior art for these hybrids
2026-08-01 11:26:09 -07:00
Tin Chi Lo
b45c99f6c5 fix(litellm): support OpenAI chat completions custom tool calls end to end
Cursor Ask mode sends chat bodies whose tools array mixes nested function
tools with flat Responses-style custom tools; the /cursor messages arm now
nests those before delegating, published via the request parsed-body cache.
Core chat parsing gains first-class custom tool call types mirroring the
openai SDK union: a single dict dispatch feeds the provider-dict sinks,
Delta dispatch stops both stream re-parse sites from silently swallowing
custom deltas, the chunk builder accumulates custom input for spend logs,
function-assuming consumers (json-mode gate, multi_tool_use repair,
helicone, lunary) skip custom entries, and the chat-to-responses bridge
flattens nested custom tools to the Responses flat shape
2026-08-01 11:26:09 -07:00
Tin Chi Lo
96916f29a6 feat(proxy): serve the OpenAI model list at /cursor/models for BYOK base URLs 2026-08-01 11:26:09 -07:00
Tin Chi Lo
14c97ba8db fix(proxy): make /cursor/chat/completions work with Cursor agent mode
- delegate messages-shaped bodies to the standard chat completions handler
- strip chat-only stream_options before the Responses pipeline
- fix cursor_data_generator signature (request kwarg) and duck-type the
  stream gate so router-wrapped streams convert instead of leaking raw
  Responses events
- convert custom_tool_call items and events to chat tool_calls in the
  streaming and non-streaming paths; remap streamed tool_call indices to
  0-based sequential; accumulate raw and pydantic tool calls into one choice
- normalize generic pydantic output items through the raw-dict handler
2026-08-01 11:26:08 -07:00
Yuneng Jiang
48b3d18893
feat(ui): note in the add-member modal that search covers existing users only
Both fields select from a server-side search over existing accounts, so a
typed-in address or id never becomes a value. Say so up front rather than
letting the form look like it accepts a new user and fail on submit.

Applies to the organization member modal too, which shares this component.
2026-08-01 09:28:02 -07:00
devin-ai-integration[bot]
e38df02d85
fix(ui): show pass through route selections and match team id substrings in team search (#35319)
* fix(ui): show pass through route selections in team/key forms and match team id substrings in team search

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(teams): keep team id search index-friendly with a prefix match

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(teams): keep /v2/team/list search id matching exact by default and add an opt-in prefix mode

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 18:18:48 -07:00
ryan-crabbe-berri
5466402274
fix(ui): keep the session view open when selecting a log inside it (#35399)
Opening a session from the logs table stored no ?session_id (row clicks
called openLog, which deletes it), so session mode was derived from the
clicked row's session_total_count. Rows fetched by the session drawer
come from /spend/logs/session/ui, which does not enrich that field, so
selecting any log inside the session view swapped in an unenriched row
and collapsed the drawer to a single-log Trace view

Row clicks on a multi-call session's row now call openSession, and
selectLog writes ?session_id when the session view is active, so session
mode is anchored in the URL instead of derived from row data
2026-07-31 16:36:32 -07:00
yuneng-jiang
5df98a8f47
Merge pull request #35309 from BerriAI/worktree-floofy-watching-dijkstra
feat(ui): add sorting, filtering and search to the budgets page
2026-07-31 16:36:05 -07:00
milan
7d5c5d8873 fix(ui): show the subfolder path for non-browsable skill sources
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 22:03:38 +00:00
milan
66e1ea6090 fix(ui): render non-browsable skill sources as text on the detail page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 21:58:27 +00:00
milan
1618617ea9 feat(ui): accept ssh clone urls when registering a skill
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 21:47:22 +00:00
devin-ai-integration[bot]
3083c55ffc
fix(ui): nest source object in Claude Code marketplace settings snippet (#35322)
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 14:34:42 -07:00
Yuneng Jiang
79b2a5e56e
feat(ui): put the budgets CTA in the tab bar and scroll the rows, not the page
The create button now sits in the tab bar beside the tabs, the way Teams
lays it out, with one divider between them and the rule running the full
width underneath.

Adds a fillHeight mode to DataTable that treats the parent's height as a
ceiling rather than a target, so the table still sizes to its rows and a
short one keeps its footer under the last row, while a long one scrolls
its rows under a sticky header instead of scrolling the page. This replaces
the hardcoded viewport-height caps those tables would otherwise need. Two
details the mode has to fix: the Table primitive's own overflow container
would capture the sticky header, and rows would show through the
semi-transparent header tint.
2026-07-31 12:55:16 -07:00
milan
c4ab243fa6 fix(ui): only label logs cost and duration a session total when the aggregate exists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 19:25:57 +00:00
Yuneng Jiang
2769fbe37b
feat(ui): give the budgets page a standard header and default column set
Matches the Virtual Keys layout: a page header with the wallet icon, the
create button directly beneath it, and the tab bar below that, on the same
page padding Teams and Access Groups use so the table no longer sits against
the window edge.

Reset and Created start hidden, so the table opens on the four columns it
has always shown and the two new ones are opt-in from the Columns menu.
2026-07-31 12:22:09 -07:00
Yuneng Jiang
e95f53b2d2
Merge remote-tracking branch 'origin/litellm_internal_staging' into worktree-floofy-watching-dijkstra 2026-07-31 11:54:35 -07:00
tin-berri
fa56283806
feat(ui): show which log rows are the auto-router's own classifier calls (#35304)
* feat(spend-logs): record when a spend log row is the auto-router's own classifier call

The complexity router's classifier sub-call copies the parent request's metadata
verbatim, so its spend log row carries the caller's key, team and user and is
indistinguishable from traffic the caller actually sent. Nothing on the row says
otherwise: call_type is "acompletion" either way, model_group is overwritten to the
classifier's own model group so the row never looks auto-routed, and routing_decision
is absent exactly as it is on an ordinary request.

Record the fact the system already knows at call time. internal_call_origin is
declared on SpendLogsMetadata, which is the allowlist _get_spend_logs_metadata
projects onto, and stamped in _classifier_call_metadata; both classifier paths
already route through that one function and it feeds the metadata and
litellm_metadata buckets alike, so every request surface is covered at one site.
The key is reserved rather than caller-supplied, so it joins routing_decision in the
untrusted-metadata strip and a caller cannot label their own traffic as router
overhead.

The classifier call also inherited no session identity, so the router minted a fresh
trace id and the row landed in a session of its own. Forwarding the parent's session
puts it in the trace of the request that triggered it, which is where an operator
looks for what the routing cost.

* feat(ui): show which log rows are the auto-router's own classifier calls

A classifier row now carries internal_call_origin and shares its parent's session,
so the session trace lists it beside the request that triggered it. Without a marker
in the sidebar it reads as another call the caller made, which is the confusion this
resolves.

The tag renders only for a recognized origin, so ordinary traffic and any future
origin this build does not know about stay unlabelled rather than being asserted as
classifier calls.
2026-07-31 11:51:36 -07:00
milan
11225b7f13 fix(spend): sum multi-round session duration in logs UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-31 18:48:31 +00:00
yuneng-jiang
fcec1488e2
feat(proxy): add GET /management/v1/budgets (#35310)
* feat(proxy): add a generic list contract for management/v1 entity lists

Paging, sorting, filtering and search for an entity collection, declared once
as a ListSpec and served by handle_list. The route injects a ListExecutor that
owns its table, so this module never imports Prisma.

The caller's scope is derived from the caller alone and ANDed with whatever
they filtered on, so a query parameter can only narrow what they may read.

This is the shared half of the budgets list; it lands here so the endpoint has
something to register against, and drops out when the framework arrives on its
own branch.

* feat(proxy): add GET /management/v1/budgets

The Budgets page reads /budget/list, which returns the whole table as a bare
array with no way to page, sort or filter it. A customer with enough budgets to
fill the page has no way to find one.

Registers LiteLLM_BudgetTable against the management/v1 list contract: sortable
on budget_id, max_budget, tpm_limit, rpm_limit and created_at, default order
newest-first with budget_id breaking ties, search on budget_id, and filters for
budget_duration, max_budget and created_at. budget_duration is deliberately not
sortable; the column holds "7d"/"30d" strings, so a lexicographic ORDER BY puts
"30d" ahead of "7d".

tpm_limit and rpm_limit are BigInt? in Prisma, so rows validate through a
pydantic model on the way out and serialize as JSON numbers.

A caller without admin view is refused 403 as a problem document rather than
served an empty page. /budget/list is untouched.

* fix(proxy): rework the budgets list onto the merged list contract

PR #35308 landed a different shape than this branch was written against: `where`
is a tuple of frozen predicates rather than a Prisma-shaped mapping, `ListSpec`
carries both the row and the wire type, and `where_sql` / `order_by_sql` render
for a raw-SQL executor. The budgets executor now queries through `query_raw` the
way the spend logs facet does, selecting only the columns it serves.

Also casts datetime binds in `where_sql`. They cross into the query engine as
JSON, so an uncast placeholder arrives as text and Postgres refuses
`timestamp >= text` outright; every `filter[created_at][gte|lte]` was answering
500. The cast reads the bind as an instant and drops it to naive UTC to match
Prisma's TIMESTAMP(3) column, the same one /spend/logs/ui applies.

* refactor(proxy): fold the predicate renderer instead of recursing

recursive_detector flags `_render_all`, and the flag is fair: it recursed once
per predicate, so the stack grew with the number of filters on the request for
no reason. Walking a predicate list is a running bind index, which is a fold.

`_render` still re-enters for `AnyOf`, but its clauses are plain comparisons
built by `?q=`, so that nesting is one level deep and no caller can drive it
deeper.
2026-07-31 11:47:05 -07:00
Yuneng Jiang
6a327aee65
refactor(ui): type the budgets list against the generated API schema
The management list route now exists, so budgetItem, the list envelope and
the response type come from schema.d.ts instead of being hand-written
against the contract. The optional fields widen accordingly, so the rate
limit and reset cells accept undefined alongside null.
2026-07-31 11:42:17 -07:00
Yuneng Jiang
8e60c2e725
Merge remote-tracking branch 'origin/litellm_/management-v1-budgets' into worktree-floofy-watching-dijkstra 2026-07-31 11:35:19 -07:00
Yuneng Jiang
d2307a0143
Merge remote-tracking branch 'origin/litellm_internal_staging' into worktree-floofy-watching-dijkstra 2026-07-31 11:35:01 -07:00
Yassin Kortam
0e9a624a97
feat(mcp): source the ID-JAG subject from the user's stored SSO assertion (#35147)
The ID-JAG egress arm could only assert a caller that presented its own IdP
identity token on the request, so an agent holding a brokered LiteLLM
credential got a 412 and never reached the upstream. The assertion captured at
SSO login was already persisted per user for exactly this purpose, but nothing
read it back.

The arm now falls back to that stored assertion, keyed on the authenticated
principal's user_id. The identity is always taken from the credential the
gateway authenticated, never from a caller-supplied field, so no caller can
select whose identity is asserted upstream. A missing, expired, or
unidentified subject stays a 412; ID-JAG exists to assert a specific user and a
missing subject has no safe substitute. A store outage is the one exception: it
is surfaced as a typed AssertionStoreUnavailable and mapped to 503, so a
database blip cannot 500 the egress or the upstream-401 retry, and does not
tell the user to sign in again over something they cannot fix.

Sourcing a subject from the store rather than the request changed what
invalidation can rely on, so the exchanged-token cache changed with it. The
entry is now addressed by a slot key derived from the principal, plus the
caller's own token when it presented one, with a fingerprint of the subject
token and config stored beside the bearer and compared on every read. A
mismatch reads as a miss and re-mints, so a rotated assertion or an edited
server config cannot be served a bearer authorized under the old inputs, and
two callers cannot receive each other's. Invalidation is a single delete of a
key it can always compute, needing no store lookup on the recovery path.

The upstream-401 invalidate-and-retry path was also gated on a truthy inbound
subject token, which skipped recovery entirely for store-sourced calls. The
gate is now mode-aware: token_exchange still requires an inbound token because
it has nothing else to mint from, id_jag does not.

oauth2_id_jag is also now selectable in the admin dashboard with its own field
set, instead of being reachable only from config.yaml or the REST API. The
auth-type selects drop antd list virtualization: at eleven options the last one
no longer mounts, which is a scroll in a browser but makes the option
unreachable to anything reading the rendered list.

Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
2026-07-31 10:25:38 -07:00
Yuneng Jiang
78c756dff9
fix(proxy): rework the budgets list onto the merged list contract
PR #35308 landed a different shape than this branch was written against: `where`
is a tuple of frozen predicates rather than a Prisma-shaped mapping, `ListSpec`
carries both the row and the wire type, and `where_sql` / `order_by_sql` render
for a raw-SQL executor. The budgets executor now queries through `query_raw` the
way the spend logs facet does, selecting only the columns it serves.

Also casts datetime binds in `where_sql`. They cross into the query engine as
JSON, so an uncast placeholder arrives as text and Postgres refuses
`timestamp >= text` outright; every `filter[created_at][gte|lte]` was answering
500. The cast reads the bind as an instant and drops it to naive UTC to match
Prisma's TIMESTAMP(3) column, the same one /spend/logs/ui applies.
2026-07-31 10:19:24 -07:00
Yuneng Jiang
f0d1bc3c5a
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/management-v1-budgets
# Conflicts:
#	litellm/proxy/management_endpoints/management_v1/list_framework.py
#	litellm/types/proxy/management_endpoints/management_v1.py
2026-07-31 09:46:22 -07:00