Commit graph

5474 commits

Author SHA1 Message Date
Yassin Kortam
e6af62aaac
Merge pull request #40997 from BerriAI/litellm_batch_tpd_limit
feat(proxy): add tpd_limit (tokens per day) for batch submissions
2026-09-15 13:39:45 -07:00
tin-berri
d07b2e87d2
Merge pull request #41270 from BerriAI/litellm_capability_classifier_pr
feat(router): add capability classifier as Fuse foundation
2026-09-15 13:04:21 -07:00
yassin
ea6314492f refactor(ui): move key rate limit fields into KeyRateLimitFields
key_edit_view.tsx crossed the 800 line eslint max-lines ceiling once the
tpd_limit field landed. Move the tpm/rpm/tpd fields into a shared
KeyRateLimitFields control so the edit view stays under the limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 19:51:43 +00:00
Tin Chi Lo
914ae9b248 test(ui): use the current deployment affinity label 2026-09-15 11:38:28 -07:00
Yassin Kortam
837423237a
Merge pull request #36775 from MvdB/litellm_presidio_new_entities
feat(guardrails): add new upstream presidio pii entities including german set
2026-09-15 11:26:21 -07:00
Tin Chi Lo
2da9bbfc0f chore: merge main into capability classifier copy 2026-09-15 11:24:01 -07:00
tin-berri
3ac79757f4
Merge pull request #41175 from BerriAI/litellm_team_member_auto_routers
feat(auto-router): allow opted-in team members to manage their routers
2026-09-15 11:19:58 -07:00
ryan-crabbe-berri
8b6c398b92
Merge pull request #41039 from BerriAI/litellm_bulk_user_delete
feat(proxy): add POST /user/bulk_delete and POST /team/bulk_member_delete
2026-09-15 11:12:40 -07:00
ryan
6ea1085bc3 Merge branch 'main' into litellm_bulk_user_delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:03:15 +00:00
ryan-crabbe-berri
08b433267e
Merge pull request #40917 from BerriAI/litellm_credential_conflict_409
fix(credentials): answer 409 on a credential name collision, make Terraform adoption opt-in
2026-09-15 10:58:26 -07:00
ryan-crabbe-berri
6dcca8c4ae
Merge pull request #41028 from BerriAI/litellm_bulk_new_user
feat(proxy): add POST /management/v1/users/bulk for batched user and team membership creation
2026-09-15 10:57:46 -07:00
ryan-crabbe-berri
81806f33cf fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id
POST /credentials let a duplicate name hit the unique index and handed back
Prisma's "Unique constraint failed" as a 500, so callers string-matched that
message to tell a caller mistake from a server fault. The unique violation now
maps to a 409 whose message names the PATCH route, two concurrent creates of
one name agree on it, and the detection lives in a repository helper the five
hand-rolled copies can move onto later

PATCH /credentials/{name} took a CredentialItem body, so the model_id the
Terraform adopt path sent was dropped. It now accepts UpdateCredentialItem and
shares the deployment lookup with create. Both handlers take the router as a
FastAPI dependency instead of reading the proxy global, which is what the
tests override
2026-09-15 10:41:43 -07:00
tin-berri
3ad9a7f336
Merge pull request #41174 from BerriAI/litellm_tier_model_affinity
fix(router): preserve session model choice within each complexity tier
2026-09-15 09:54:53 -07:00
Tin Chi Lo
81340439fc fix(router): preserve session model choice within each complexity tier 2026-09-15 00:09:17 -07:00
Tin Chi Lo
109ca70f66 feat(auto-router): allow opted-in team members to manage their routers 2026-09-14 23:56:55 -07:00
Tin Chi Lo
398300c4e7 fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
yucheng
9acdebf563 chore: merge origin/main into litellm_lit6314_guardrail_metadata_transfer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:03:31 +00:00
kerry-berri
03a4c806c4
Merge pull request #41157 from BerriAI/litellm_gemini_embedding_2_per_token_billing
fix(cost): bill gemini-embedding-2 per token and stop double charging audio
2026-09-14 19:49:11 -07:00
kerry
c0c5044c45 fix(batches): keep modality token details in raw vertex batch usage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:57:50 +00:00
yucheng-berri
8481bc27f9
Merge pull request #41132 from BerriAI/litellm_bedrock_grounding_from_plain_messages
fix(bedrock guardrails): derive contextual grounding source and query from plain messages
2026-09-14 17:56:58 -07:00
yucheng
0d0b96ed06 Revert "refactor(guardrails): rename scoped-out evaluation status from not_run to skipped"
This reverts commit b37ce94075.
2026-09-14 23:46:09 +00:00
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped"
This reverts commit f78dd921c9.
2026-09-14 23:46:09 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
yucheng
f78dd921c9 fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Usage tracking, compliance and the dashboard now treat both not_run (older spend logs) and skipped as unevaluated through a shared UNEVALUATED_GUARDRAIL_STATUSES set, so old records stop counting as passed. The skipped record is no longer written when the request carried images, since images without text were never dispatched to guardrails before this change and that gap is not a message-scoping skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:34:33 +00:00
ryan
ca04b03c2c feat(proxy): move bulk user delete and team member delete under /management/v1
Replaces POST /user/bulk_delete and POST /team/bulk_member_delete with
POST /management/v1/users/bulk_delete and
POST /management/v1/teams/{team_id}/members/bulk_delete per the Management
API modernization one-pager: {data} envelopes, application/problem+json
errors with urn:litellm:error:* types, 422 on unknown body fields, 400 on
unknown query params, 403 on authorization failures, 404 on unknown team.

Route checks now match parametrized management/v1 paths so team-scoped
callers reach the endpoint's own authorization and get a 403 problem
instead of the generic 401.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:31:07 +00:00
ryan
96332f75e8 refactor(proxy): move bulk user creation to POST /management/v1/users/bulk
Follows the Management API modernization design: plural resource under
/management/v1, {data, meta} response envelope, unknown request fields
rejected with 422, and RFC 9457 problem+json for request-level errors.
Body validation failures under /management/v1 now answer 422 instead of
the 400 query-parameter problem. /user/bulk_new is removed.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:39 +00:00
yucheng
b37ce94075 refactor(guardrails): rename scoped-out evaluation status from not_run to skipped
The per-guardrail status a scoped-out evaluation records is now skipped, matching the
skip_*_in_guardrail settings that cause it. Request-level rollup still maps it to not_run
so the StandardLoggingPayload status contract is unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:15:53 +00:00
yassin
df5e193645 Merge remote-tracking branch 'origin/main' into pr-40997
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/litellm-dashboard/src/components/templates/key_edit_view.tsx
2026-09-14 22:07:48 +00:00
ryan-crabbe-berri
7b6d223105
Merge pull request #40196 from BerriAI/litellm_team_admin_all_proxy_models
fix(ui): let team admins grant a team all proxy models
2026-09-14 15:05:01 -07:00
yucheng
442af3aab6 fix(bedrock guardrails): gate plain-message grounding behind contextual_grounding_from_messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:04:42 +00:00
ryan-crabbe-berri
51926c2e79
Merge pull request #41023 from BerriAI/litellm_ui_move_metadata_tags_to_tags_field
fix(ui): move tags typed into key metadata JSON into the Tags field
2026-09-14 15:03:54 -07:00
ryan
ef71222349 Merge remote-tracking branch 'origin/main' into litellm_bulk_user_delete 2026-09-14 22:00:10 +00:00
ryan
52a3b0ede7 fix(ui): trim and drop blank tags moved out of key metadata JSON
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:50:25 +00:00
ryan
df87a49f8b fix(ui): offer no models when an org team's ceiling never arrives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:35:55 +00:00
yassin
39f6ac4788 perf(proxy): serialize /model/info listing once with orjson
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:47:51 +00:00
Yassin Kortam
687cb6cb7a
Merge pull request #40992 from BerriAI/litellm_model_info_team_alias
fix(ui): show the team alias on the model info page and in its raw JSON
2026-09-14 12:40:02 -07:00
ryan
5ddb0fe691 fix(ui): keep the team model select loading until /team/info reports the org ceiling
useTeam seeds its cache from the team list, which has no organization_models, so the select briefly rendered unfiltered. Also drop the Prisma include assertion from the backend test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:05 +00:00
ryan-crabbe-berri
b6a5563d05 fix(ui): let team admins grant a team all proxy models
The team edit form only offered "All Proxy Models" when the dashboard could
read the parent organization, and /organization/info 403s for anyone who is
not a proxy admin or an admin of that org. A team admin with the internal_user
proxy role therefore saw only "No Default Models", which is the opposite of
what they wanted, and had no way to grant their team everything on the proxy.

/team/info now reports the parent org's model ceiling as organization_models,
which the same authorization already admits, and ModelSelect reads the ceiling
from the team it is editing before falling back to the organization. That also
makes the individual model list respect the org's allow-list instead of
listing every proxy model to a caller whose save would be rejected.

useTeam was typed as Team while returning the /team/info envelope, so its one
other caller unwrapped it behind a cast. It now returns the team itself.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
yucheng
b1a006ea66 test(ui): hoist not_run guardrail fixtures out of inline call args
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:54:24 +00:00
yassin
82ef6ea6ab feat(proxy): add general_settings.allowed_file_extensions for /v1/files
Opt-in allowlist for upload filename extensions, checked before the existing blocked_file_extensions blocklist and mapped through the same upload validation failure path. None keeps today's behaviour, [] rejects every upload, matching is case-insensitive on both sides, and a filename with no extension is rejected when the allowlist is set.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:53:29 +00:00
yucheng
e41faf54a8 fix(ui): anchor guardrail lifecycle on timed entries and show not_run skip reason
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:39:10 +00:00
yucheng
4be84a859d Merge remote-tracking branch 'origin/main' into litellm_lit6314_guardrail_metadata_transfer 2026-09-14 18:01:01 +00:00
yucheng
b4e3d7c533 chore: merge main into litellm_lit6314_guardrail_metadata_transfer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 17:33:04 +00:00
ryan
450f7deff8 docs(proxy): shorten bulk delete endpoint descriptions to one sentence
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 05:46:02 +00:00
ryan
595aba3cb0 fix(proxy): evict deleted keys from the auth cache and make bulk user deletion transactional
/user/bulk_delete now deletes the users' keys, invitation links, org and team
memberships and user rows in one transaction and reports a rolled-back batch
per row instead of leaving partial deletes behind. Both bulk endpoints evict
the deleted keys (and deleted user objects) from the auth cache, so a deleted
key stops authenticating immediately rather than at TTL expiry.

/team/bulk_member_delete rejects member rows that carry both user_id and
user_email, reports repeated rows as duplicates, and only cleans up keys and
memberships of members it actually matched.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 05:12:34 +00:00
ryan
cd9c39921b feat(proxy): add POST /user/bulk_delete and POST /team/bulk_member_delete
Batch user deletion that also removes each user from every team they belong to, and batch removal of many members from one team. Each touched team is rewritten once under the team advisory lock from a roster re-read under that lock

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 04:55:39 +00:00
ryan
24a1d772b6 fix(proxy): keep key policy fields and reconcile committed rows in /user/bulk_new
Rows opting into auto_create_key lost blocked, permissions, aliases, config,
agent_id, budget_fallbacks and budget_limits before reaching the key helper.
When create_many commits but the response is lost, re-read which ids landed
and retry only the rest so committed rows report success and get their teams.
Regenerate schema.d.ts and allowlist the endpoint in the Terraform audit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 01:48:44 +00:00
ryan
0201ca60e7 fix(ui): move tags typed into key metadata JSON into the Tags field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 23:09:27 +00:00
yassin
a41b719920 fix(proxy): refund batch TPD reservation on failure and report active window reset time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 10:56:17 +00:00
yassin
c47120cbf7 fix(proxy): add tpd_limit to deleted token table and fix CI fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 10:31:35 +00:00