Commit graph

5617 commits

Author SHA1 Message Date
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped"
This reverts commit f78dd921c9.
2026-09-14 23:46:09 +00:00
yucheng-berri
99245f9323
Merge pull request #41131 from BerriAI/litellm_prompt_security_created_status
fix(prompt_security): keep polling file sanitization through non-terminal statuses
2026-09-14 16:43:37 -07:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
yassin
c6a36fb8b6 fix(health): skip background health check DB writes when the latest-row read fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 6d2c4899b0)
2026-09-14 23:03:30 +00:00
yassin
ece2bbfed7 fix(proxy): resolve x-litellm-call-id from response metadata when routes omit call_id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:03:27 +00:00
ryan
8e83e91275 fix(proxy): keep bulk user row errors free of the tuple length message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:40:57 +00:00
yucheng
f78dd921c9 fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Usage tracking, compliance and the dashboard now treat both not_run (older spend logs) and skipped as unevaluated through a shared UNEVALUATED_GUARDRAIL_STATUSES set, so old records stop counting as passed. The skipped record is no longer written when the request carried images, since images without text were never dispatched to guardrails before this change and that gap is not a message-scoping skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:34:33 +00:00
ryan
ca04b03c2c feat(proxy): move bulk user delete and team member delete under /management/v1
Replaces POST /user/bulk_delete and POST /team/bulk_member_delete with
POST /management/v1/users/bulk_delete and
POST /management/v1/teams/{team_id}/members/bulk_delete per the Management
API modernization one-pager: {data} envelopes, application/problem+json
errors with urn:litellm:error:* types, 422 on unknown body fields, 400 on
unknown query params, 403 on authorization failures, 404 on unknown team.

Route checks now match parametrized management/v1 paths so team-scoped
callers reach the endpoint's own authorization and get a 403 problem
instead of the generic 401.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:31:07 +00:00
yucheng
d8d7210b57 test(bedrock guardrails): cover tagged messages on the apply_guardrail response path with the flag off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:58 +00:00
ryan
96332f75e8 refactor(proxy): move bulk user creation to POST /management/v1/users/bulk
Follows the Management API modernization design: plural resource under
/management/v1, {data, meta} response envelope, unknown request fields
rejected with 422, and RFC 9457 problem+json for request-level errors.
Body validation failures under /management/v1 now answer 422 instead of
the 400 query-parameter problem. /user/bulk_new is removed.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:39 +00:00
yucheng
b37ce94075 refactor(guardrails): rename scoped-out evaluation status from not_run to skipped
The per-guardrail status a scoped-out evaluation records is now skipped, matching the
skip_*_in_guardrail settings that cause it. Request-level rollup still maps it to not_run
so the StandardLoggingPayload status contract is unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:15:53 +00:00
yucheng
2e11f7bc7b test(bedrock guardrails): shorten grounding test docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:09:59 +00:00
ryan-crabbe-berri
7b6d223105
Merge pull request #40196 from BerriAI/litellm_team_admin_all_proxy_models
fix(ui): let team admins grant a team all proxy models
2026-09-14 15:05:01 -07:00
yucheng
442af3aab6 fix(bedrock guardrails): gate plain-message grounding behind contextual_grounding_from_messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:04:42 +00:00
yucheng
3a3075b8a2 fix(prompt_security): poll only on queued statuses, keep 500 for terminal or missing status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:01:39 +00:00
ryan
ef71222349 Merge remote-tracking branch 'origin/main' into litellm_bulk_user_delete 2026-09-14 22:00:10 +00:00
Yassin Kortam
3e417fa6e0
Merge pull request #41113 from BerriAI/litellm_realtime_release_max_parallel_slot
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
2026-09-14 14:56:11 -07:00
Yassin Kortam
24bfd5fba1
Merge pull request #41102 from BerriAI/litellm_team_membership_once_main
fix(auth): load team membership once per request and skip prisma on an L1 hit
2026-09-14 14:46:06 -07:00
yassin
b4d0f4ad26 refactor(realtime): move session ownership marker keys into constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:33:48 +00:00
yucheng
26e2e208ad fix(bedrock guardrails): derive contextual grounding source and query from plain messages
Bedrock only runs a contextualGroundingPolicy when the ApplyGuardrail payload
carries grounding_source and query qualifiers. Callers sending ordinary system
and user messages never got those, so a configured grounding threshold was
silently skipped on /v1/chat/completions and /guardrails/apply_guardrail.

When no explicit grounding_source or query tags are present, system and
developer text is sent as grounding_source and the latest user message as
query. The apply_guardrail response branch now forwards the request messages,
which it previously dropped.

Resolves LIT-4224

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:18:16 +00:00
yucheng
33fb6625ad test(guardrails): cover nameless evaluated entries in the malformed usage batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:18:00 +00:00
yassin
db8dfe93a5 fix(auth): wait for the in-flight membership load before evicting its cache key on invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:10:43 +00:00
yassin
2441e8a2a9 Merge remote-tracking branch 'origin/main' into litellm_realtime_release_max_parallel_slot 2026-09-14 21:03:28 +00:00
yucheng
fb60f80c80 fix(prompt_security): keep polling file sanitization through non-terminal statuses
Prompt Security reports a queued sanitization job as status "created" before
it moves to "in progress" and "done". The poller treated anything other than
those two known strings as an error and returned HTTP 500 on the first poll, so
every image or file request through the guardrail failed while the vendor job was
still queued.

Only "done" is terminal now. Every other status is logged and polled again until
max_poll_attempts or the outer file_sanitization_timeout, after which the existing
fail-open or fail-closed (408) policy applies.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:03:02 +00:00
Yassin Kortam
4123b4bc2b
Merge pull request #41086 from BerriAI/litellm_team_list_org_admin_membership_lit3723
fix(proxy): keep org admins' own team memberships in other orgs visible on team list
2026-09-14 13:57:08 -07:00
yassin
ffeea30f23 test(auth): cover a stale membership write landing after a fresh reload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:55:22 +00:00
yucheng
937179bde1 fix(guardrails): never map empty guardrail names onto a sibling id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:53:11 +00:00
yucheng
586d51f15e fix(guardrails): skip malformed guardrail entries instead of failing the usage batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:50:41 +00:00
tin-berri
2cad7a49af
Merge pull request #41116 from BerriAI/litellm_auto_router_session_api_access
fix(proxy): allow LLM API keys to read auto-router sessions
2026-09-14 13:49:03 -07:00
yassin
abc85ba607 fix(proxy): leave the realtime max_parallel_requests slot to the limiter failure callback when a refusal was logged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:47:37 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:46:09 +00:00
yucheng
7ebb169a4d fix(guardrails): coalesce usage index rows per request and guardrail, keeping policy linkage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:45:47 +00:00
Yassin Kortam
161ae2f3b4
Merge pull request #41107 from BerriAI/litellm_hide_credentials_hint_when_ui_password_set
fix(proxy): hide default credentials login hint when UI_PASSWORD is set
2026-09-14 13:44:09 -07:00
yucheng
c3f52fe0d5 fix(guardrails): key not_run index rows by the sibling evaluation's guardrail_id
A not_run entry from the base guardrail only carries guardrail_name, while the
content filter's evaluated entry carries guardrail_id. Keyed apart, one request
listed twice in the monitor for a logging_only guardrail (Not run and Passed).
Resolve the id from a same-name sibling in the payload so the severity pick applies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:37:09 +00:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
Yassin Kortam
856aedd14a
Merge pull request #40817 from BerriAI/litellm_redis_timeout_log_throttle
fix(redis): log a timeout streak once per interval instead of one line per cache call
2026-09-14 13:34:45 -07:00
yassin
1b31be1a9c fix(proxy): leave the realtime max_parallel slot to the success callback when one is enqueued
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:23:23 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:13:39 +00:00
ryan
2d3b63fb5a fix(proxy): report parent org models on /team/info only to team managers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:01:47 +00:00
yassin
81ffc3125f fix(proxy): release realtime max_parallel_requests slot when the task is cancelled during pre-call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:50:01 +00:00
yassin
39f6ac4788 perf(proxy): serialize /model/info listing once with orjson
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:47:51 +00:00
Yassin Kortam
f67cc62032
Merge pull request #40995 from BerriAI/litellm_all_deployments_cooldown_error
fix(router): name the all-deployments-in-cooldown error on 429 responses
2026-09-14 12:39:42 -07:00
ryan
9cdfe311e9 test: assert only the org ceiling /team/info returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:39:18 +00:00
ryan
5ddb0fe691 fix(ui): keep the team model select loading until /team/info reports the org ceiling
useTeam seeds its cache from the team list, which has no organization_models, so the select briefly rendered unfiltered. Also drop the Prisma include assertion from the backend test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:05 +00:00
Yassin Kortam
7c55c4ba7d
Merge pull request #40925 from BerriAI/litellm_proxy_error_headers_from_litellm_response_headers
fix(proxy): forward provider request id headers on mapped error responses
2026-09-14 12:36:59 -07:00
ryan-crabbe-berri
3f42a4fc98 test: suppress TQ008 on the two module-state patches
team_info resolves prisma_client and get_all_team_memberships from module
state, so there is no collaborator to inject; the two sibling team_info tests
patch the same way.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
ryan-crabbe-berri
b6a5563d05 fix(ui): let team admins grant a team all proxy models
The team edit form only offered "All Proxy Models" when the dashboard could
read the parent organization, and /organization/info 403s for anyone who is
not a proxy admin or an admin of that org. A team admin with the internal_user
proxy role therefore saw only "No Default Models", which is the opposite of
what they wanted, and had no way to grant their team everything on the proxy.

/team/info now reports the parent org's model ceiling as organization_models,
which the same authorization already admits, and ModelSelect reads the ceiling
from the team it is editing before falling back to the organization. That also
makes the individual model list respect the org's allow-list instead of
listing every proxy model to a caller whose save would be rejected.

useTeam was typed as Team while returning the /team/info envelope, so its one
other caller unwrapped it behind a cast. It now returns the team itself.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
yassin
f9d0cd6409 chore(proxy): drop explanatory docstrings from credentials hint helper and tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:30 +00:00
yassin
ebcd9bcb18 fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:11 +00:00
yassin
ff02390685 test(auth): drop docstrings from team membership tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:26:14 +00:00