Commit graph

5109 commits

Author SHA1 Message Date
Mateo Wang
3a00e8020a
Merge pull request #39464 from BerriAI/litellm_containers_error_passthrough_pagination
fix(containers): pass upstream error status through and forward list pagination params
2026-09-03 14:36:44 -07:00
ryan-crabbe-berri
ed9865c945
Merge pull request #39604 from BerriAI/litellm_lit_4738_users_search_by_id
fix(ui): let the Internal Users search box match user_id as well as email
2026-09-03 14:14:22 -07:00
Mateo Wang
5a821b593c
Merge pull request #38440 from BerriAI/litellm_prompt_registry_env
fix(prompts): key the in-memory prompt registry by environment
2026-09-03 14:09:26 -07:00
ryan
792852b4f3 test(ui): name the mocked user list body to stay within the inline-object lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 21:04:55 +00:00
devin-ai-integration[bot]
aff6b7e212
fix(ui): clear agents when updating team permissions (#39600)
Always serialize object_permission.agents and agent_access_groups in
the team update payload so removing the last agent in the dashboard
sends an explicit empty array instead of omitting the key, which the
backend merge treats as no change

Resolves LIT-6861

Co-authored-by: yassin <yassin@berri.ai>
2026-09-03 13:51:04 -07:00
ryan-crabbe-berri
1c14ded0e4
Merge pull request #39598 from BerriAI/litellm_lit_4929_session_token_aggregation
fix(ui): aggregate session token usage in the logs table
2026-09-03 12:57:27 -07:00
ryan-crabbe-berri
f173a6baf1
Merge pull request #39215 from BerriAI/litellm_lit_6592_team_overview_inherited_mcp_agents
fix(ui): show MCP servers and agents inherited from access groups on team overview
2026-09-03 12:53:27 -07:00
devin-ai-integration[bot]
92122086ec
fix: stop a cleared Organization field from failing key creation (#39316)
* fix: stop a cleared Organization field from failing key creation

Clearing the Organization combobox in the Create Key modal left organization_id set to an empty string, so /key/generate looked up an organization named "" and failed with "Organization doesn't exist in db. Organization=".

OrganizationDropdown now emits null on clear, and GenerateKeyRequest normalizes an empty organization_id or project_id to None the same way it already does for team_id.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: drop customer-specific docstring from key request normalization test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-09-03 19:12:11 +00:00
devin-ai-integration[bot]
e046aee3d5
fix(spend_tracking): add missing_session_id: omit to leave SpendLogs.session_id null without a client session (#39458)
* fix(spend_tracking): leave SpendLogs.session_id null when no client session id was established

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(lint): ratchet basedpyright budget after session_id fix

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): ignore trace ids as session ids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): gate null SpendLogs.session_id behind missing_session_id: omit

Unset, generate and reject keep the legacy trace id fallback. omit records only
metadata.session_id, the key Langfuse reads, so a trace id copied into
litellm_session_id by get_litellm_params never becomes a session.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): stamp the omit decision on the request so a config reload cannot fabricate a session

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): keep omit covering requests the pre-call stamp never reaches

Router-model provider pass-through calls allm_passthrough_route directly and
skips add_litellm_data_to_request, so those requests never run the pre-call
helper and carry no omit stamp. Reading only the stamp made POST
/anthropic/v1/messages write a fabricated uuid into SpendLogs.session_id under
missing_session_id: omit while its Langfuse trace had no session, the exact
divergence the policy exists to remove.

The stamp now only pins omit on, and an unstamped request falls back to the
configured policy, so a config reload still cannot fabricate a session for a
request that was decided pre-call.

* fix(spend_tracking): make the session-omission marker proxy-owned so clients cannot forge it

* fix(spend_tracking): strip the client-sent omission marker from both metadata buckets before they merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): strip the session-omission marker from both metadata buckets

The pre-call policy ran before litellm_metadata is merged into metadata, so a
client that planted the marker in litellm_metadata had it copied back into the
route's own bucket after the strip and still got a null SpendLogs.session_id.

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-03 11:57:56 -07:00
ryan
4d2ffe2e8e test(ui): hoist inherited-grant fixture out of the inline createMockTeamData arg
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 18:38:22 +00:00
ryan
3b13a5fda9 test(ui): query the tokens cell by role instead of walking the DOM
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 18:31:43 +00:00
ryan-crabbe-berri
07c5908b18 fix(ui): let the Internal Users search box match user_id as well as email
GET /user/list gains an optional search query param that ORs a case-insensitive contains match over user_id and user_email. The Users page search box now sends that param and reads "Search by email or ID…", the way the Teams page already searches by name or ID. Every existing /user/list param keeps its meaning and the Filters drawer is untouched

Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
2026-09-03 11:27:56 -07:00
ryan-crabbe-berri
b3325750ae fix(ui): aggregate session token usage in the logs table
The logs table already rolled up cost per session but the Tokens column only showed the representative call's usage. The per-session aggregate query now also sums prompt, completion and total tokens, and the Tokens cell switches to those sums for multi-call sessions the same way the Cost cell does.

Claude-Session: https://claude.ai/code/session_01CNasFqyjnLN3Rqman25vde
2026-09-03 11:09:44 -07:00
devin-ai-integration[bot]
29c37141c3
feat(ui): keyset-paginate request logs by session trace (#38794)
* feat(ui): keyset-paginate request logs by session trace

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep session grouping within type discipline budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(proxy): ruff format session grouping helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): only group sessions when group_by_session is an explicit true

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): reset session cursor on custom range and live tail toggles

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): cover cursor reset on custom range toggle

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): ignore next page clicks while the grouped page is still fetching

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): only block next page while grouped placeholder data is shown

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-09-03 17:27:05 +00:00
ryan-crabbe-berri
2fd6e19051 fix(ui): name the granting access group on hover instead of an Inherited tag
`/team/info` access_group_details now carries mcp_server_ids and agent_ids per
group next to models, so the dashboard can say which group granted a server or
agent. The Object Permissions rows drop the Inherited badge and the row tooltip
reads "Granted via access group <name>. Full ID: <id>", listing every group
when more than one grants the same id and falling back to "an access group"
when the proxy did not say.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-03 10:20:10 -07:00
yuneng-jiang
342e4470c4
fix(ui): replace the key detail URL entry when a virtual key is rotated (#39471)
Regenerating a key repointed ?key= at the rotated hash with a pushed history
entry, so pressing the browser Back button landed on the hash that had just
been revoked. /key/info answers 404 for it and the page shows "Key not found
in database".

The rotated hash now replaces the current entry instead of pushing a new one,
so Back from a just-regenerated key returns to the key list. Opening a key
from the table still pushes, so Back from a normally opened key is unchanged.
2026-09-03 09:47:45 -07:00
moe-berri
4990f06acc
feat(auto-router): support classifier reasoning effort (#39372)
* feat(auto-router): support classifier reasoning effort

* fix(auto-router): harden classifier reasoning effort

* fix(ui): satisfy classifier config lint limits

* refactor(auto-router): simplify classifier effort support

* fix(auto-router): clear frontend-lint and type-discipline gates, trim LOC

---------

Co-authored-by: Tin Chi Lo <tin@berri.ai>
2026-09-03 08:59:04 -07:00
Mateo Wang
34d4f7f8ae
fix: 1.99.0-rc2 UI bug batch (empty org on key create, session pagination, access group rename/delete) (#39436)
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create

* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side

* fix(proxy): keep access groups in sync when a model is renamed or deleted

* fix(proxy): cap the Request Logs conversation total like the row total

* fix(proxy): judge access group backing by the database for db models

A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.

* fix(ui): keep the conversation badge when an MCP call represents a conversation

A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.

* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): type session spend aggregates

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): satisfy request logs lint budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): cap per-session model aggregation in request logs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: ratchet type-discipline budget after staging merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): send an explicit null when the key edit form clears the organization

Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 08:58:48 -07:00
Rakesh
1de960bce7
fix(docker): bump nginx runtime to 1.31.5-alpine3.24 and pin digest to resolve critical CVEs (#39561) 2026-09-03 08:46:07 -07:00
yuneng-jiang
658f50663d
fix(ui): keep Virtual Keys list state in the URL so it survives leaving the page (#39481)
* fix(ui): keep Virtual Keys list state in the URL so it survives leaving the page

The search term, sort, pagination and drawer filters lived in component
state, so navigating away from Virtual Keys and back reset the table to an
unfiltered first page. Move them into query state alongside the existing
?key= deep link, which also makes a filtered view shareable.

* fix(ui): namespace the Virtual Keys filter params and bound page inputs

The unprefixed team_id filter hijacked the /api-keys create-key deep link,
which already takes team_id as a prefill, so ?create=true&team_id=X silently
filtered the list underneath the modal. Prefix the four drawer filters.

Now that page and page_size come from the address bar, clamp them to what
/key/list accepts instead of forwarding 0, negatives or an int64-overflowing
page straight through, and trim filter values arriving from a URL the same
way the drawer already trims them.

* fix(ui): fall back to a sortable column when the URL names an unknown one

A hand-edited or stale sort_by reached /key/list, which 400s it, leaving the
Virtual Keys page on its loading skeleton with no error. Validate it against
the fields the table's own headers can produce, and clear sort_by rather than
blanking it when a sort is reset so the URL stays clean.

Also replaces a default-state URL assertion that ran before any query-state
write could land, so it could not fail for the regression it named.

* fix(ui): use TanStack's functionalUpdate instead of a hand-rolled updater resolver

The local helper narrowed typeof updater === "function" against an
unconstrained T, which TypeScript cannot do because T itself may be a function
type, so next build failed to type check. table-core already exports the same
helper.
2026-09-03 00:35:32 -07:00
yucheng-berri
ecabfbd5af
fix(guardrail): hide-secrets playground redaction and guardrail telemetry (#39398)
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry

The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.

- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
  so proxied traffic stays on async_pre_call_hook (per-key opt-out and
  data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
  via _process_response/_process_error; opted-out keys and legacy nameless
  callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
  and /guardrails/ui/provider_specific_params with a config model

Resolves LIT-3548

* Fix hide-secrets passthrough telemetry and JSON config input

* fix(guardrails): validate hide-secrets object config before submit

- apply_guardrail treats empty-string-only texts as no input, so no
  false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
  until it parses to a JSON object, instead of posting a string to an
  object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type

* fix(guardrails): record no hide-secrets telemetry when nothing was inspected

walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
2026-09-03 00:01:03 -07:00
Mateo Wang
62ed7e1942
Merge pull request #39478 from BerriAI/litellm_ui_presets_mock_build_ctx
fix(ui): read the preset catalog at runtime in the dashboard tests
2026-09-02 22:38:59 -07:00
yuneng-jiang
c841a56e9a
fix(ui): stop the create team form resetting organization and models (#39476)
* fix(ui): stop the create team form resetting organization and models

The organization preselect ran in an effect keyed on the organizations
query, so any refetch of that list while the Create Team modal was open
overwrote the user's organization pick, which in turn cleared their
models pick. The models field was also cleared whenever the available
models fetch resolved.

Preselect the organization when the modal opens instead, and clear the
models only when the user picks a different organization. An org admin
whose admin orgs narrow to one while the form is open can still pick,
rather than facing a locked empty field.

* fix(ui): block team create when the picked organization is no longer available

An organization picked in the Create Team form now survives a refetch of
the organization list, so it can outlive the admin's access to it. Refuse
the create with a message on the field rather than letting the request
fail authorization at the proxy.

* fix(ui): keep the team create organization field usable when the pick goes stale

Locking the field on a single admin organization also locked it while it
held a rejected organization, so an admin who lost access could not pick
the one organization left. Lock it only while it holds that organization.

* test(ui): hoist the created team fixture out of the mock call

The inline object pushed the repo past its no-large-inline-object-arg
lint budget, which has no headroom.
2026-09-02 22:34:15 -07:00
tin-berri
534003da03
feat(ui): add 1M context auto-router preset (#39490)
* feat(ui): add 1M context auto-router preset

* feat(ui): use heuristic v2 for 1M preset

* fix(ui): keep 1M preset test within lint budget
2026-09-02 22:33:14 -07:00
mateo-berri
f62e87d28c fix(ui): read the preset catalog through the shared mock in the lib test 2026-09-02 21:42:47 -07:00
mateo-berri
cd296814be Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_containers_error_passthrough_pagination 2026-09-02 21:26:40 -07:00
Tin Chi Lo
fcc9b813af fix(ui): resolve the preset catalog relative to the mock, not cwd
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HqEPCNLDrssxsuezhAaL4j
2026-09-02 20:01:35 -07:00
Tin Chi Lo
099d26204f fix(ui): read the preset catalog at runtime in the vitest mock
The autoRouterPresets mock imported litellm/proxy/public_endpoints/autorouter_presets.json
as a module. That path sits outside ui/litellm-dashboard, the only directory the UI
Dockerfile copies, so `next build` type-checking inside the image failed with
"Cannot find module" and the ui-image job went red on every PR that touched an
image-scan path. Read the file with fs at runtime instead; vitest still derives
expectations from the real bundled catalog.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HqEPCNLDrssxsuezhAaL4j
2026-09-02 19:54:09 -07:00
tin-berri
64e45a069d
feat(complexity_router): opt-in modality override of a kept session-affinity pin (#39454)
A session pinned to a text-only model failed every image turn with a provider 400, because the modality gate exempts a kept pin by cause. Add modality_pin_override so that exemption is conditional: the image turn is re-placed on a capable model for that request only, reported as cause modality_pin_override, and the stored pin is left untouched so the next text turn replays it.

The pin write on the replay path already happens upstream of the gate and stores the session's own model, so pin survival is structural rather than bookkeeping. The new cause joins the non-pinnable set. Default off at every layer.
2026-09-02 19:36:09 -07:00
mateo-berri
872e115295 fix(containers): pass upstream error status through and forward list pagination params
The container retrieve, list, delete, create and file routes validated the
provider's error body against the success model, so a deleted or unknown
container and a rejected API key surfaced as 500 pydantic errors instead of
the upstream 404 or 401. The handlers now raise the provider error class with
the upstream status and message before transforming the response.

GET /v1/containers dropped after, limit and order before calling the
provider, and GET /v1/containers/{id}/files dropped the same three, so
paginated list calls ignored their pagination arguments. Both routes now
forward their declared query params.
2026-09-02 18:46:00 -07:00
devin-ai-integration[bot]
8441dd6e8c
fix(proxy): keep SpendLogs and callback session ids in sync when the request has none (#39450)
* fix(proxy): keep SpendLogs and callback session ids in sync when the request has none

Add general_settings.missing_session_id (generate | reject). In generate mode one id is
stamped into litellm_session_id, litellm_trace_id and metadata.session_id before callbacks
run, so LiteLLM_SpendLogs.session_id and the Langfuse session id match. In reject mode such
requests get a 400. Unset keeps the legacy behavior. MCP routes are not affected

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(proxy): regenerate schema.d.ts and shorten mutable-ok comment for ruff format

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): mark generated session ids so affinity consumers do not pin on them

Fireworks x-session-affinity, the router session_affinity pre-call check and the
complexity router session pin all read metadata.session_id as a caller-chosen
stable key. A missing_session_id: generate id is fresh per request, so it now
carries metadata.litellm_session_id_generated and those consumers skip it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 18:28:06 -07:00
tin-berri
993766be0e
feat(proxy): serve the auto-router preset catalog at runtime (#39412)
The dashboard's template picker imported autorouter_presets.json at build time, so every
catalog change needed a dashboard rebuild and artifacts refresh. The catalog now lives in
litellm/proxy/public_endpoints/ and GET /public/autorouter_presets serves it, fetching
litellm.autorouter_presets_url (GitHub raw on main, 1h in-process cache, bundled fallback)
so a merged catalog change propagates to running proxies like the model cost map does.
The dashboard fetches it at runtime via useAutoRouterPresets and keeps no local copy.

Resolves LIT-6764
2026-09-02 18:03:22 -07:00
tin-berri
ff1f21aea9
fix(ui): paginate request logs by session groups server-side (#39257)
* fix(ui): paginate request logs by session groups server-side

The logs table server-paginated raw spend logs and then collapsed
multi-call sessions client-side, so a page could render 3 rows while
the footer claimed 25 and sessions straddled pages. Adds an opt-in
group_by_session param to /spend/logs/ui that pages and counts one
representative row per session (DISTINCT ON, newest non-MCP call),
keeps the bounded count contract, enriches whole-session llm/agent
composition counts, and deletes the client-side collapse pipeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve

* feat(ui): add a 10 rows-per-page option and default request logs to it

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve

* fix(ui): key session aggregates per api key in the logs enrichment

Grouped pagination splits a reused session id into one row per api key,
but the enrichment still aggregated by session_id alone, so both rows
showed combined spend and counts. The aggregate query now groups by
(session_id, api_key), the count folds into it (the separate group_by
query is deleted), and each row reads its own key's totals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve

* fix(ui): treat an empty api_key as a real session group value

The spend-log schema defaults api_key to an empty string; truthiness
guards in the enrichment treated it as missing, so keyless multi-call
sessions lost their count and spend. Only None means missing now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 17:54:17 -07:00
tin-berri
632898b2a6
feat(router): add a hybrid classifier that defers near tier boundaries (#39403) 2026-09-02 23:51:43 +00:00
tin-berri
9aeeca4ce3
feat(router): add heuristic v2 complexity routing (#39276)
* feat(router): add trained heuristic complexity routing

* feat(router): expose heuristic v2 classifier

* style(router): format heuristic v2 predictor
2026-09-02 23:33:04 +00:00
yujonglee
082bea851e
Merge pull request #39334 from BerriAI/litellm_rust_opt_in_configuration
feat(python): unify Rust opt-in and bridge policy
2026-09-02 16:26:36 -07:00
yucheng-berri
b4f5b6aa94
fix(logging): redact credential query params from the uvicorn access log (#39293)
* fix(logging): redact credential query params from the uvicorn access log

Raw virtual keys reached container stdout two ways:

- `GET /key/info?key=sk-...`, `/global/spend/report?api_key=sk-...`,
  `/key/spend/report`, `/spend/logs`, `/user/daily/activity` and the Gemini
  passthrough routes all put the credential in the request target, and
  `uvicorn.access` had no redaction filter (only `uvicorn.error` did).
- the key budget error interpolates `LiteLLM_VerificationToken.key_name`,
  a column with no enforced shape, into a message that is both logged and
  returned to the caller.

`SecretRedactionFilter` cannot be reused on an access logger: it collapses the
record into `record.msg` and clears `record.args`, and uvicorn's AccessFormatter
unpacks those args at emit time, so every access line would raise TypeError.
`AccessLogRedactionFilter` scrubs the positional args in place instead.

An access line is the one input to the secret regex an unauthenticated caller
controls end to end, so two bounds go with it. The request target is cut back to
a whole query parameter under 512 characters before it is scanned, since a half
parameter is too short to match its own pattern and would be logged raw, and the
dropped tail is not logged at all. The connection-string pattern is bounded too,
because its user half could previously re-scan the rest of the string from every
`://`: a 16 KB URL of `a://` pairs took 314s and now takes 0.12s, with the caps
set high enough that an RDS IAM auth token used as a DSN password still redacts.

Credential query params are terminated by `&` like the existing `key=` and
`sig=` patterns, so redacting one param no longer swallows the rest of the
request line, and a second credential in the same query string is now redacted
on its own instead of surviving once the first one stops the span. `key_name` is
echoed into the budget error only when it still has the masked `sk-...abcd`
shape `abbreviate_api_key` writes, so a value put there by a direct DB write or
a migration falls back to the key alias.

Also point the `/key/info` and spend-report examples at the sha256 hash both
endpoints already accept, so callers stop putting raw keys in URLs that
third-party access logs record.

Resolves LIT-5909

* test(logging): assert on emitted access lines instead of filter registration

The two registration tests checked that an AccessLogRedactionFilter instance
sits in uvicorn.access.filters, which is the shape of the code rather than its
behavior. Handing the logger a real access record and reading what a handler
wrote covers the same wiring and still fails when the registration is removed.

* fix(logging): redact percent-encoded credentials from access logs

?k%65y=sk%2D... is a working credential once the request parser decodes it,
but the redaction patterns match literal text and never see it. Decode the
request target as a detector and drop the query when decoding reveals a
secret. The decoded text is never logged back, so a %0A cannot forge a
following log line

Also accept any four non-space characters in the masked key_name check, since
abbreviate_api_key copies the last four characters of a custom key verbatim
and those can be punctuation or non-ASCII

* fix(auth): keep control codes out of the masked key label

/key/generate accepts a custom key ending in an escape sequence, and
abbreviate_api_key copies those four characters into key_name verbatim, so
the over-budget message carried them to a terminal and a log viewer. Bar
whitespace and C0/C1 control codes from the four, and keep everything else
2026-09-02 15:10:36 -07:00
devin-ai-integration[bot]
a76cb6feaf
feat(mcp): semantic tool search for the native MCP Gateway (#39404)
The mcp_tool_search virtual tool only did substring token matching, so a native MCP client asking for "FX" could not find a tool described as "foreign exchange rates" even though the same catalog is ranked by embeddings on /responses and /chat/completions.

Adds litellm_settings.mcp_tool_search (embedding_model, top_k, similarity_threshold, core_tools). With an embedding model the caller's authorized catalog from _list_mcp_tools is ranked by cosine similarity of name plus description; configured core tools the caller can reach come first and do not consume top_k. Without an embedding model the keyword fallback keeps the old behavior. Settings are hot-reloadable from the DB, exposed on /get and /update mcp_tool_search_settings, and editable from the Admin UI under MCP Servers > Tool Search. The embedding index is shared with agent_search via a new SemanticTextIndex.

Resolves LIT-6751

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 15:09:44 -07:00
Yassin Kortam
25991fe78a
feat(auth): enforce configurable password policy and SSO-only login (#39381)
Adds a configurable password-strength policy (default: min 12 chars,
upper/lower/number/special, all individually toggleable, floored at 8
so a misconfigured minimum cannot disable the length check, and
unicode-aware so an accented letter cannot satisfy the special-
character requirement) enforced on every path that sets a local
user's password: /user/update, /user/bulk_update, and the invitation
onboarding claim flow.

Adds general_settings.disable_password_login_when_sso_enabled, which
rejects username/password login on /login, /v2/login and /v3/login
(including the UI_USERNAME/UI_PASSWORD admin fallback) once ANY
configured SSO provider is FULLY ready: every companion secret/
endpoint an OAuth provider needs, checked independently per provider
so a stray leftover client id for an unused provider can't mask a
different, fully configured one; and for SAML, the optional
python3-saml runtime being importable, checked without letting a
fully-missing package's ModuleNotFoundError take down password login
itself. SSO becomes the enforced boundary for interactive UI access
without an incomplete, mixed, or half-installed SSO setup locking
every admin out or breaking login outright. Master-key API access is
untouched, and unsetting the setting plus a restart restores password
login as the documented recovery path.
2026-09-02 14:28:13 -07:00
Yassin Kortam
646f3404a5
fix(security): restrict and validate file uploads at /v1/files and /upload/logo (#39379)
* fix(security): restrict and validate file uploads at /v1/files and /upload/logo

Extends fast-fail upload validation to every purpose at POST /v1/files,
not just purpose=batch: a configurable max_file_size_mb size cap and a
blocked_file_extensions denylist, plus rejection of filenames carrying a
directory-traversal component before anything is read, stored, or
forwarded to a provider.

Also fixes two concrete gaps found while auditing every upload surface:
the Azure Blob Storage backend derived a blob path's extension with
filename.split(".")[-1], which does not parse path structure and let a
crafted filename embed a directory traversal sequence into the stored
blob path; and POST /upload/logo (the admin UI logo upload) had no
role check at all, so any authenticated API key, not just a proxy
admin, could write a file to the server's disk.

* fix(lint): drop cast()/mutation from settings coercion, sync blocked_file_extensions on reload

Replaces the TypeAdapter+cast() reads of max_file_size_mb and
blocked_file_extensions with small isinstance-based validators, since the
codebase's cast() budget (LIT006) had no headroom left. Also adds the
blocked_file_extensions reload block that was missing from
_update_general_settings: it was registered as an editable setting but
never re-synced into runtime state, so a value set through the DB-backed
settings editor would silently never take effect (Greptile finding).

* fix(security): declare max_file_size_mb and blocked_file_extensions on ConfigGeneralSettings

The DB-backed general-settings update endpoints validate every field
through ConfigGeneralSettings.model_fields before persisting it, so
without these declarations an operator could never actually set either
setting through that path even though both were registered for the
Admin UI's settings editor and reloaded on config refresh (Greptile
finding). blocked_file_extensions is typed as a tuple, not a list, to
stay out of the immutable-collections lint budget; the stored JSON
value is unaffected since the raw request payload, not the validated
model, is what gets persisted.

* chore: regenerate schema.d.ts for the new ConfigGeneralSettings fields

* fix(security): normalize configured blocked_file_extensions casing

check_blocked_extension lowercased the uploaded filename's extension
before comparing but compared it against blocked_extensions verbatim,
so an admin-configured blocked_file_extensions: ['.EXE'] would never
match an uploaded payload.exe (Greptile finding). Normalizes the
configured values the same way at comparison time, and adds the
missing case (mismatched-case config, lowercase upload) as a
regression test, mutation-checked against the unfixed comparison.

* fix(security): restore caller-owned stream position after size inspection

_file_size_bytes unconditionally seeked back to 0 after measuring a
BinaryIO's length, discarding wherever the caller had actually
positioned it (Greptile finding). Saves and restores the original
position instead. Rewrites the existing test that had encoded the
old "always resets to 0" behavior as its expectation, and adds a
sibling case for the under-cap path; both are mutation-checked
against the unfixed always-reset-to-0 behavior.
2026-09-02 14:27:46 -07:00
tin-berri
2ade3e16a9
feat(ui): update OpenAI preset model tiers (#39396) 2026-09-02 13:45:16 -07:00
Yassin Kortam
711430216e
fix(ui): preserve full AgentCore runtime ARN in agent edit form (#39382)
parseDynamicAgentForForm recovered a credential field's value from a
stored model string by splitting both the model_template and the model
on "/" and matching by array index. That breaks for any placeholder
value that itself contains "/", such as a Bedrock AgentCore runtime ARN
resource path (runtime/<runtime-id>), silently dropping everything
after the first slash when populating the edit form. Saving without
touching the field then persisted the truncated ARN.

Replace the index-matching split with a non-mutating template parse
(split on the placeholder pattern, escape and rejoin the literal
segments into a regex) so a placeholder captures everything it needs
regardless of embedded slashes. Also add a lightweight ARN-shape
validator for the AgentCore runtime ARN field, guarded against a
malformed pattern string, so a truncated value is rejected client-side
before it reaches the backend.

Resolves LIT-6737
2026-09-02 13:33:14 -07:00
Mateo Wang
b600f02fc2
Merge pull request #34788 from BerriAI/litellm_fix_s3_vectors_search
fix(vector_stores): s3 vectors search router bypass + rag query config drop + ui error swallow
2026-09-02 11:35:56 -07:00
mateo
0608f0a00f fix: reject unknown runtime router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:17:02 +00:00
devin-ai-integration[bot]
2b616fc479
feat(scim): add placeholder listing and merge so a shadowed account can be healed (#39231)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 01:04:53 +00:00
ryan-crabbe-berri
944d018c72 test(ui): query the screen in models page tests and drop restating comments
The merge of #38872 into staging kept the destructured render queries
in the models-and-endpoints page test, which pushes
testing-library/prefer-screen-queries to 21 against a budget of 18 and
fails frontend-lint for every PR on top of it. Two comments that only
restated the list merge below them are gone as well.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 16:25:15 -07:00
ryan-crabbe-berri
fc1a5fd7f9
Merge pull request #39206 from BerriAI/litellm_lit_3925_clear_team_key_create
fix: stop a cleared Team field from blocking personal key creation
2026-09-01 16:21:52 -07:00
ryan-crabbe-berri
1964d92fc6 test(ui): query the clear button and the models page tabs through accessible screen queries 2026-09-01 16:04:45 -07:00
ryan-crabbe-berri
4acc1d15fb fix(ui): map a cleared Team dropdown back to an empty string in the auto-router form 2026-09-01 15:50:19 -07:00
ryan-crabbe-berri
c493fc855c fix(ui): show MCP servers and agents inherited from access groups on the team overview
The team Overview and Settings tabs fed only object_permission into the Object
Permissions card, so a team whose access group grants MCP servers or agents read
"MCP Servers 0" and "Agents 0" while the Models card next to it already listed the
inherited models. /team/info has returned access_group_mcp_server_ids and
access_group_agent_ids for a while, nothing in the dashboard read them.

ObjectPermissionsView now accepts the inherited ids and MCPServerPermissions /
AgentPermissions merge them into their lists with an Inherited tag, deduped
against direct grants, so an admin can tell a group grant from a direct one.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 15:46:14 -07:00