Generalizes the heuristic_v2 ceiling from #39468 into a capability table whose
records own their in-process predicate, SQL spelling and refusal wording. The
existing heuristic_v2 capability keeps its own one-router ceiling. A single
customization capability combines operator-defined tier definitions with every
operator-written part of the classifier prompt. The prompt half only applies to
classifier types that call an LLM. The shipped default prompt, classification
rubric presets, tier-label renames and tier model choices remain ungated.
Scope every enforcement point to actual complexity routers. A model-less PATCH
or legacy update now decrypts the stored model before accepting strategy-router
settings, so a regular model cannot acquire a router config or spend a license
slot. Under the existing advisory lock, the cross-pod candidate query returns
only model scalars and the count decrypts and classifies them in process; old
non-router rows carrying a capability-shaped config no longer block a real
complexity router. The signed auto_router license feature makes both ceilings
unlimited.
A shadow eval job could only be scoped by identity, so "this user's traffic on model X
across every key they own" was not expressible and a models field on the start body was
silently dropped. The job now carries a models list that every target is narrowed to,
matched on the requested model group with model_group_alias resolved on both sides. An
unresolvable name is a 400 at start. Empty means every model, which is what every existing
row reads as. The dashboard start form gains an "Only on models" picker and the job
headline shows the scope.
The access group detail page rendered MCP servers, agents, attached teams and keys as bare ids, so an admin had to look each one up elsewhere to audit a group
Every access group response now also carries access_mcp_servers, access_agents, assigned_teams and assigned_keys as {id, name} pairs. Names come from the DB rows first and fall back to config-declared MCP servers and agents (including legacy agent ids), resolved with one query per table across all groups in a list call. The existing *_ids columns are unchanged
The UI renders the name with the id in a tooltip, links teams and keys to their detail pages, and shows the raw id only when nothing resolves
/key/regenerate carries the JWT-to-key mapping to the new token via FK
cascade, but the jwt_key_mapping cache entry kept resolving the old
(now invalid) token for up to virtual_key_mapping_cache_ttl. Snapshot
the key's mapping cache keys before the token update and evict them
with evict_and_broadcast so every worker drops the stale entry.
Also share the cache-key format through jwt_key_mapping_cache_key and
upgrade the /jwt/key/mapping CRUD endpoints from local-only deletes to
evict_and_broadcast, closing the same cross-worker staleness there.
* feat(team): report per-user spend within a team for JWT traffic
Add GET /team/spend/by_user, which groups raw spend logs by (team_id, user)
so JWT/SSO requests with no virtual key are attributed to the user inside
each selected team. Team admins see every member, plain members see only
their own row. The Team Usage page gets a Spend Per User Within Team card
with CSV export backed by the same endpoint.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(team): cover /team/spend/by_user in behavior suite, tf audit allowlist and EntityUsage unit test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(team): drop explanatory docstrings from /team/spend/by_user and regen schema.d.ts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Okta sends profile updates as full PUTs with no groups or groups: [], since SCIM User.groups is readOnly and membership is synced through /Groups. The PUT handler diffed that empty list against the stored teams, removed the user from every team (which also deletes their team keys) and recomputed the role from an empty group list. Treat an empty groups list on PUT as unspecified: keep the stored teams and leave the role alone. Explicit non-empty groups still replace memberships as before
Claude-Session: https://claude.ai/code/session_01CqwUV4Ywnu5aUjXx1UhJrM
A sent null for tpm_limit, rpm_limit, max_budget and the other budget fields
was dropped by a 'v is not None' filter, so update_budget was never called and
the request returned 200 without changing the budget row. Presence is now read
from model_fields_set (merge-patch semantics, matching /v2/organization) and
the nested litellm_budget_table payload no longer drops nulls either
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The search= param on /key/list, /audit, and /spend/logs/ui, plus key_hash= on /key/list, now compare the pasted value verbatim. Only a copied key ID (the hash) matches, so a raw virtual key never needs to travel in a GET query string
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
GET /key/list?search= matches the key hash (a raw sk- key is hashed
first) or a case-insensitive alias substring, and key_hash= now hashes a
raw sk- value too. GET /v1/memory?search= matches a key prefix or an
exact memory_id. GET /audit?search= matches id, object_id, changed_by,
or changed_by_api_key. GET /spend/logs/ui?search= matches request_id
across all time and api_key, team_id, user, end_user, session_id, or
model_id inside the date window; session grouping is skipped while a
search is active.
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
Without the auto_router feature in the signed enterprise license a proxy may hold
one complexity router with classifier_type heuristic_v2 across config.yaml and the
DB; with it the limit is lifted. The ceiling is derived once from LicenseCheck and
handed to the Router, which refuses the extra router at registration. config.yaml
over the limit refuses to start, and /model/new, /model/update and
PATCH /model/{id}/update refuse the write with a 403 before touching the DB.
Expiry follows the existing max_users/max_teams pattern: judged when the
license is verified, not on every call, and a verify that rejects the license
(expired or unreadable) leaves no signed payload behind. The rollback after a
failed upsert re-admits state that was already serving, so it is exempt from the
ceiling: an edit that fails, including one refused by a ceiling that has since
tightened, leaves the router serving its previous configuration.
A write that leaves a row on heuristic_v2 under a limited license runs in one
transaction that takes a Postgres advisory lock before counting the DB rows plus
this proxy's config.yaml routers, so concurrent writes on any pod cannot both
claim the sole slot and no surplus row is ever persisted.
Only the row insert runs under that lock: the team model bookkeeping, which
needs a second pool connection, runs after the transaction has committed.
PATCH /model/{id}/update follows the same order as create: the row is written
through the slot first and the team's model list is updated only afterwards, so
a refused write leaves the team as it was.
The slot transaction bypasses the repository's publish-on-write, so it
publishes the config change once after commit, as delete_team_models does.
* fix: stop a cleared Organization field from failing key creation
Clearing the Organization combobox in the Create Key modal left organization_id set to an empty string, so /key/generate looked up an organization named "" and failed with "Organization doesn't exist in db. Organization=".
OrganizationDropdown now emits null on clear, and GenerateKeyRequest normalizes an empty organization_id or project_id to None the same way it already does for team_id.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop customer-specific docstring from key request normalization test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* fix(proxy): return persisted team memberships from /user/new
new_user attached default teams after building its response from the
pre-membership snapshot, so NewUserResponse.teams was always empty for
users created with default_internal_user_params.teams. The CLI SSO flow
reads that response on a user's first login and minted a teamless JWT,
which skipped the default team's model allowlist.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): return team ids as a tuple to satisfy LIT001
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
GET /user/list gains an optional search query param that ORs a case-insensitive contains match over user_id and user_email. The Users page search box now sends that param and reads "Search by email or ID…", the way the Teams page already searches by name or ID. Every existing /user/list param keeps its meaning and the Filters drawer is untouched
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
`/team/info` access_group_details now carries mcp_server_ids and agent_ids per
group next to models, so the dashboard can say which group granted a server or
agent. The Object Permissions rows drop the Inherited badge and the row tooltip
reads "Granted via access group <name>. Full ID: <id>", listing every group
when more than one grants the same id and falling back to "an access group"
when the proxy did not say.
Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create
* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side
* fix(proxy): keep access groups in sync when a model is renamed or deleted
* fix(proxy): cap the Request Logs conversation total like the row total
* fix(proxy): judge access group backing by the database for db models
A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.
* fix(ui): keep the conversation badge when an MCP call represents a conversation
A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.
* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type session spend aggregates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): satisfy request logs lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap per-session model aggregation in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: ratchet type-discipline budget after staging merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send an explicit null when the key edit form clears the organization
Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Recovery already resolved the owner email for double-hashed spend keys,
then Key Activity dropped it. The Usage payload now carries user_email
and the key label falls back to that email before key-hash-...
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Recovery now resolves the key owner's email from UserTable via the
recovered token user_id, and SpendLogsMetadata keeps user_api_key_user_email
so new batch/export consumers see email without a separate user join.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Callback log replay also omitted user_api_key_hash, so it could double-hash
spend rows the same way batch costing did. On the read path, Usage key
metadata now reverse-hashes orphaned DailyUserSpend.api_key values against
VerificationToken and falls back to SpendLogs metadata so historical dirty
rows show their api_key_alias again instead of key-hash-...
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(sso): resolve multi-valued role claims to the highest privilege role
A role claim carrying several roles used to resolve to whichever one the IdP
listed first, so a user holding both proxy_admin_viewer and internal_user lost
org-level spend visibility depending on claim ordering alone.
get_litellm_user_role now picks the highest privilege role out of a list-valued
claim, and the Entra app_roles path shares that same resolution instead of
keeping its own copy of the hierarchy. SAML assertions carrying several role
values go through the same path rather than taking the first value.
* test(sso): lock ranked-over-unranked role resolution for mixed claims
org_admin, team and customer sit outside the privilege ladder. Pin the
resolution for a claim that mixes one of them with a ranked role so the
asymmetry is covered rather than implicit.
* fix(sso): label the claim-sequence cast for the type-discipline gate
* fix(sso): resolve claim entries without recursing
The repo's recursive-function gate rejects self-recursion here, and a role
claim is flat anyway. Pull the single-value lookup into its own helper so the
list branch maps over it instead of calling back into itself.
* fix(proxy): mark session/SSO/SAML cookies Secure behind a TLS-terminating reverse proxy
litellm only sees a plain-HTTP hop when TLS terminates at a reverse proxy,
so cookie Secure attributes previously derived from (or defaulted without
regard to) the literal request scheme could be dropped in production. The
token session cookie set by every login path never carried Secure/HttpOnly/
SameSite at all.
Adds IPAddressUtils.is_request_https, a single trust-aware resolver used by
every cookie-setting call site: PROXY_BASE_URL, then X-Forwarded-Proto only
from a configured trusted proxy (general_settings.use_x_forwarded_for +
mcp_trusted_proxy_ranges), then the literal scheme. An unconfigured or
untrusted caller cannot spoof the header to force Secure on.
Resolves LIT-6748
* fix(proxy): make the shared session-cookie helper public, type new test helpers
set_session_token_cookie is imported across modules (ui_sso.py ->
proxy_server.py), so the leading underscore was misleading and breached
basedpyright's reportPrivateUsage budget with zero headroom. Also adds
missing parameter/return type annotations to the new test helper functions
per repo convention.
Adds a configurable password-strength policy (default: min 12 chars,
upper/lower/number/special, all individually toggleable, floored at 8
so a misconfigured minimum cannot disable the length check, and
unicode-aware so an accented letter cannot satisfy the special-
character requirement) enforced on every path that sets a local
user's password: /user/update, /user/bulk_update, and the invitation
onboarding claim flow.
Adds general_settings.disable_password_login_when_sso_enabled, which
rejects username/password login on /login, /v2/login and /v3/login
(including the UI_USERNAME/UI_PASSWORD admin fallback) once ANY
configured SSO provider is FULLY ready: every companion secret/
endpoint an OAuth provider needs, checked independently per provider
so a stray leftover client id for an unused provider can't mask a
different, fully configured one; and for SAML, the optional
python3-saml runtime being importable, checked without letting a
fully-missing package's ModuleNotFoundError take down password login
itself. SSO becomes the enforced boundary for interactive UI access
without an incomplete, mixed, or half-installed SSO setup locking
every admin out or breaking login outright. Master-key API access is
untouched, and unsetting the setting plus a restart restores password
login as the documented recovery path.
