Commit graph

1125 commits

Author SHA1 Message Date
Joshua Valluru
664b1f16bb style(tests): wrap MCP health regression setup 2026-09-17 09:19:05 -07:00
Joshua Valluru
e21db01d67 fix(mcp): scope health discovery for route-restricted keys 2026-09-17 09:17:55 -07:00
Yuneng Jiang
a0869fe835
test(budgets): avoid mutable fixture state 2026-09-16 22:06:31 -07:00
Yuneng Jiang
5c41e0b8dc
test(budgets): cover management null handling 2026-09-16 21:42:55 -07:00
ryan-crabbe-berri
fc13cea479 fix(proxy): refuse a team admin's budget write when the budget changed mid-request
The keep-or-lower check compares against the budget update_team read, so the write now only lands while the stored max_budget still matches it and answers 409 otherwise. A concurrent proxy admin cut can no longer be overwritten with a higher value.
2026-09-16 18:11:53 -07:00
ryan-crabbe-berri
e3a82f2f66 fix(proxy): stop team admins raising an org team's max_budget under the org cap
The keep-or-lower budget rule only ran for standalone teams, so once max_budget is enabled a team admin on an org team could grow its own budget up to the organization's. It now applies to team admins on every team; org admins keep editing within the org cap.
2026-09-16 17:35:18 -07:00
ryan-crabbe-berri
81ae5caa7e fix(proxy): pass only a team admin's changed fields on to the team update 2026-09-16 16:11:52 -07:00
ryan-crabbe-berri
6e2ae19670 fix(proxy): enforce org budget ceilings on /team/update
update_team loaded the org without its budget row, so the org max_budget,
tpm_limit and rpm_limit checks silently passed. It now loads the budget the
same way /team/new does
2026-09-16 15:38:30 -07:00
ryan-crabbe-berri
a44a58e91a feat(proxy): let team admins edit tpm_limit when a proxy admin enables it
tpm_limit is the first field in the team-admin allow-list registry. The team
settings tab gives a team admin a form with only the enabled fields and sends
only those on save, and UI Settings labels the checkbox the same way
2026-09-16 15:38:30 -07:00
ryan-crabbe-berri
66519da9b6 fix(proxy): report the caller's team edit access on /team/info
The dashboard gated the team settings form on a role it guessed from the
is_* props, the members list and the org list. The org list is premium
gated and empty while loading, so a team admin who is also an org admin
was told team admins cannot edit, although /team/update accepts them as
an org admin

/team/info now returns caller_edit_access, resolved by the same helper
/team/update uses, and TeamInfo keys the form and the toast off that
field. The org list is only read for the organization dropdown now, and
general_settings is read through one validated accessor in both handlers
2026-09-16 15:38:30 -07:00
ryan-crabbe-berri
9b77b5c2cb feat(proxy): let proxy admins choose which team fields team admins may edit
Team admins could never reach POST /team/update: the route gate answered
401 before the handler's team-admin branch ran. This moves /team/update
into the self-managed routes, resolves proxy admin, org admin or team
admin inside the handler, and filters team admins through a new
proxy-wide UI setting, team_admin_editable_team_fields. The setting is
an allow-list of team fields. Empty means team admins cannot edit team
settings and get a 403 pointing at the proxy admin, and changing a field
outside the list fails 403 naming that field. Only values that differ
from what is stored count, since the dashboard resends the whole form.

The registry of fields the setting accepts ships empty on purpose. Each
field lands in its own follow-up PR with its value diff and dashboard
wiring. The Admin UI gains a "Team admin editable fields" section under
Settings > UI and a toast on the team page while editing is disabled.

Refs LIT-5722

Claude-Session: https://claude.ai/code/session_01A6SkwJdfZUmkzfUkrEkqX8
2026-09-16 15:38:06 -07:00
Yassin Kortam
f62f140f75
Merge pull request #41403 from BerriAI/litellm_key_total_spend
feat(proxy): expose lifetime total_spend on virtual keys
2026-09-16 14:49:02 -07:00
Yassin Kortam
2bfa10b6d9
Merge pull request #41311 from BerriAI/litellm_key_status_filter_deleted_lookup
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
2026-09-16 14:48:34 -07:00
Yassin Kortam
95abc9fb0b
Merge pull request #41330 from BerriAI/litellm_team_model_max_budget_v2
feat(team): team-level model_max_budget with key-level overrides
2026-09-16 14:48:29 -07:00
yassin
6cf35ed71b feat(proxy): expose lifetime total_spend on virtual keys
Adds a persistent total_spend column to LiteLLM_VerificationToken and LiteLLM_DeletedVerificationToken, incremented in the same write as spend and left alone by budget resets. Surfaces it on /key/info, /key/list and the Admin UI Virtual Keys table and key detail view

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 09:42:20 +00:00
Yassin Kortam
67cb0bc089
Merge pull request #41313 from BerriAI/litellm_model_activity_response_time
feat(ui): show average response time per model in usage model activity
2026-09-15 18:43:48 -07:00
yassin
d90e7b3aec fix(team): resolve model aliases in team admin model_max_budget authority check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 01:26:40 +00:00
yassin
3e8566d878 fix(keys): keep organization_id on archived key records and /key/info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 01:03:40 +00:00
yassin
260ff5f491 feat(team): team-level model_max_budget with key-level overrides
A team can now carry a per-model budget map that every key on the team
inherits. A key's own model_max_budget entry for the same model takes
precedence, so it is gated on and billed to the key alone.

Backend: NewTeamRequest/UpdateTeamRequest accept model_max_budget (validated
like the key-level field, enterprise gated); the value is hydrated onto
UserAPIKeyAuth via the token view, TeamGrants and the carried budget state;
_check_team_model_budget enforces it in the centralized common checks; the
limiter meters spend under team_model_spend:<team>:<model>:<duration> and
skips the team counter when the key overrides; /team/update lets only a
proxy admin raise, re-window or drop a cap; /team/info exposes usage.
The Anthropic context-management compaction summary subrequest runs the
same team gate. Both fallback token-view SQL definitions project the column.

UI: team create and edit forms reuse the key-level ModelMaxBudgetEditor,
premium gated, sending {} to clear and omitting unchanged fields.

A key entry overrides the team cap only when it spend-gates the model
(non-negative max_budget); a row that only carries tpm/rpm limits or a
negative cap leaves the team cap in force.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 00:40:58 +00:00
yassin
9541b0734b refactor(keys): drop status helper docstrings and test /key/list status through the endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 00:35:45 +00:00
Tin Chi Lo
39cf1f302d feat(router): apply entitlement limits to forecast classifiers 2026-09-15 16:17:00 -07:00
yassin
5b0fa89056 feat(ui): show average response time per model in usage model activity
Roll request_duration_ms of successful, non-internal requests into the
daily spend tables as total_response_time_ms plus timed_requests, expose
both through the daily activity endpoints, and derive the average in the
Usage -> Model Activity view of the Admin UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:16:12 +00:00
yassin
6c8b9a7b05 feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
Persist and expose the lifecycle of API keys so spend, audit and FinOps
workflows can still resolve a key after it is revoked, expires or is
deleted.

/key/list?status= now accepts active, expired and revoked next to the
existing deleted value. revoked means blocked=true, expired means not
blocked with a past expiry, active is the rest, so the three values
partition the live key table. deleted keeps reading the
LiteLLM_DeletedVerificationToken archive.

/key/info falls back to that archive when the key is no longer in the
live table, running the same owner/team/org authorization check, and
every response now carries a derived status field. The hashed token is
still stripped.

The Virtual Keys page gets a Status filter (URL-persisted) and a Deleted
badge that shows when and by whom the key was deleted.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:12:01 +00:00
Yassin Kortam
e54b93017b fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions 2026-09-15 21:02:17 +00:00
Yassin Kortam
e6af62aaac
Merge pull request #40997 from BerriAI/litellm_batch_tpd_limit
feat(proxy): add tpd_limit (tokens per day) for batch submissions
2026-09-15 13:39:45 -07:00
tin-berri
d07b2e87d2
Merge pull request #41270 from BerriAI/litellm_capability_classifier_pr
feat(router): add capability classifier as Fuse foundation
2026-09-15 13:04:21 -07:00
Yassin Kortam
a7b894f91b
Merge pull request #41271 from BerriAI/litellm_org_zero_max_budget
fix(proxy): enforce organization budgets when max_budget is 0
2026-09-15 12:47:18 -07:00
Yassin Kortam
501be3143d fix(proxy): enforce organization budgets when max_budget is 0
_organization_max_budget_check returned early whenever org_max_budget
was <= 0, so an organization with an explicit max_budget of 0 was
treated as unlimited instead of zero allowance. Key, team, and user
budget checks already skip only on None; align organization budgets
with that convention.

validate_team_org_change had the same defect in a different shape: it
used a truthy check on the org's max_budget when validating a team
move, so an explicit 0 there silently skipped the guard too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 11:49:05 -07:00
Tin Chi Lo
2da9bbfc0f chore: merge main into capability classifier copy 2026-09-15 11:24:01 -07:00
tin-berri
3ac79757f4
Merge pull request #41175 from BerriAI/litellm_team_member_auto_routers
feat(auto-router): allow opted-in team members to manage their routers
2026-09-15 11:19:58 -07:00
ryan-crabbe-berri
6dcca8c4ae
Merge pull request #41028 from BerriAI/litellm_bulk_new_user
feat(proxy): add POST /management/v1/users/bulk for batched user and team membership creation
2026-09-15 10:57:46 -07:00
mateo-berri
1f2d050386 Merge remote-tracking branch 'origin/main' into litellm_unified_key_policy_hook 2026-09-15 05:11:29 -07:00
Tin Chi Lo
109ca70f66 feat(auto-router): allow opted-in team members to manage their routers 2026-09-14 23:56:55 -07:00
Tin Chi Lo
398300c4e7 fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
yucheng-berri
91588221cd
Merge pull request #39050 from BerriAI/litellm_lit6314_guardrail_metadata_transfer
fix(guardrails): record not_run evaluation when scoping leaves nothing to scan
2026-09-14 22:31:16 -07:00
yucheng
9acdebf563 chore: merge origin/main into litellm_lit6314_guardrail_metadata_transfer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:03:31 +00:00
mateo-berri
8573241c49 fix(cost): resolve a missing 1h cache write rate after off-peak pricing
The one-hour cache write fallback now takes the applied cache write rate, so an off-peak write price carries into it instead of the input rate

The cost estimate test for a cost-map model without cache prices now expects writes at the input rate, which is what the proxy bills

The recording logger in the deferred guardrail test types its callback parameters
2026-09-14 21:46:33 -07:00
yucheng
0d0b96ed06 Revert "refactor(guardrails): rename scoped-out evaluation status from not_run to skipped"
This reverts commit b37ce94075.
2026-09-14 23:46:09 +00:00
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped"
This reverts commit f78dd921c9.
2026-09-14 23:46:09 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
ryan
8e83e91275 fix(proxy): keep bulk user row errors free of the tuple length message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:40:57 +00:00
yucheng
f78dd921c9 fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Usage tracking, compliance and the dashboard now treat both not_run (older spend logs) and skipped as unevaluated through a shared UNEVALUATED_GUARDRAIL_STATUSES set, so old records stop counting as passed. The skipped record is no longer written when the request carried images, since images without text were never dispatched to guardrails before this change and that gap is not a message-scoping skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:34:33 +00:00
ryan
96332f75e8 refactor(proxy): move bulk user creation to POST /management/v1/users/bulk
Follows the Management API modernization design: plural resource under
/management/v1, {data, meta} response envelope, unknown request fields
rejected with 422, and RFC 9457 problem+json for request-level errors.
Body validation failures under /management/v1 now answer 422 instead of
the 400 query-parameter problem. /user/bulk_new is removed.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:39 +00:00
yucheng
b37ce94075 refactor(guardrails): rename scoped-out evaluation status from not_run to skipped
The per-guardrail status a scoped-out evaluation records is now skipped, matching the
skip_*_in_guardrail settings that cause it. Request-level rollup still maps it to not_run
so the StandardLoggingPayload status contract is unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:15:53 +00:00
yassin
df5e193645 Merge remote-tracking branch 'origin/main' into pr-40997
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/litellm-dashboard/src/components/templates/key_edit_view.tsx
2026-09-14 22:07:48 +00:00
ryan-crabbe-berri
7b6d223105
Merge pull request #40196 from BerriAI/litellm_team_admin_all_proxy_models
fix(ui): let team admins grant a team all proxy models
2026-09-14 15:05:01 -07:00
Yassin Kortam
4123b4bc2b
Merge pull request #41086 from BerriAI/litellm_team_list_org_admin_membership_lit3723
fix(proxy): keep org admins' own team memberships in other orgs visible on team list
2026-09-14 13:57:08 -07:00
ryan
2d3b63fb5a fix(proxy): report parent org models on /team/info only to team managers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:01:47 +00:00
ryan
9cdfe311e9 test: assert only the org ceiling /team/info returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:39:18 +00:00
ryan
5ddb0fe691 fix(ui): keep the team model select loading until /team/info reports the org ceiling
useTeam seeds its cache from the team list, which has no organization_models, so the select briefly rendered unfiltered. Also drop the Prisma include assertion from the backend test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:05 +00:00