Commit graph

5502 commits

Author SHA1 Message Date
yassin
e91c6ca789 fix(proxy): fall back to direct spend increments once the early reconcile has finalized the reservation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:22:01 +00:00
yassin
d08e43c6af fix(proxy): invalidate reserved counters when the early reconcile fails
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:01:30 +00:00
yassin
a3636acd0d fix(proxy): reconcile budget reservation before enqueuing spend to the DB
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:01:30 +00:00
ryan-crabbe-berri
7b6d223105
Merge pull request #40196 from BerriAI/litellm_team_admin_all_proxy_models
fix(ui): let team admins grant a team all proxy models
2026-09-14 15:05:01 -07:00
Yassin Kortam
3e417fa6e0
Merge pull request #41113 from BerriAI/litellm_realtime_release_max_parallel_slot
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
2026-09-14 14:56:11 -07:00
Yassin Kortam
24bfd5fba1
Merge pull request #41102 from BerriAI/litellm_team_membership_once_main
fix(auth): load team membership once per request and skip prisma on an L1 hit
2026-09-14 14:46:06 -07:00
yassin
b4d0f4ad26 refactor(realtime): move session ownership marker keys into constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:33:48 +00:00
yassin
db8dfe93a5 fix(auth): wait for the in-flight membership load before evicting its cache key on invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:10:43 +00:00
yassin
2441e8a2a9 Merge remote-tracking branch 'origin/main' into litellm_realtime_release_max_parallel_slot 2026-09-14 21:03:28 +00:00
Yassin Kortam
4123b4bc2b
Merge pull request #41086 from BerriAI/litellm_team_list_org_admin_membership_lit3723
fix(proxy): keep org admins' own team memberships in other orgs visible on team list
2026-09-14 13:57:08 -07:00
yassin
ffeea30f23 test(auth): cover a stale membership write landing after a fresh reload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:55:22 +00:00
tin-berri
2cad7a49af
Merge pull request #41116 from BerriAI/litellm_auto_router_session_api_access
fix(proxy): allow LLM API keys to read auto-router sessions
2026-09-14 13:49:03 -07:00
yassin
abc85ba607 fix(proxy): leave the realtime max_parallel_requests slot to the limiter failure callback when a refusal was logged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:47:37 +00:00
yassin
91c964a338 fix(auth): evict the membership cache entry when invalidation lands during the cache write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:46:09 +00:00
Yassin Kortam
161ae2f3b4
Merge pull request #41107 from BerriAI/litellm_hide_credentials_hint_when_ui_password_set
fix(proxy): hide default credentials login hint when UI_PASSWORD is set
2026-09-14 13:44:09 -07:00
Tin Chi Lo
3177c37e22 fix(cli): show routed models and session stats for LLM API keys 2026-09-14 13:36:00 -07:00
Yassin Kortam
856aedd14a
Merge pull request #40817 from BerriAI/litellm_redis_timeout_log_throttle
fix(redis): log a timeout streak once per interval instead of one line per cache call
2026-09-14 13:34:45 -07:00
yassin
1b31be1a9c fix(proxy): leave the realtime max_parallel slot to the success callback when one is enqueued
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:23:23 +00:00
yassin
4ac168b3fc fix(auth): drop in-flight membership load on invalidation so it cannot repopulate the cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:13:39 +00:00
ryan
2d3b63fb5a fix(proxy): report parent org models on /team/info only to team managers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 20:01:47 +00:00
yassin
81ffc3125f fix(proxy): release realtime max_parallel_requests slot when the task is cancelled during pre-call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:50:01 +00:00
yassin
39f6ac4788 perf(proxy): serialize /model/info listing once with orjson
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:47:51 +00:00
Yassin Kortam
f67cc62032
Merge pull request #40995 from BerriAI/litellm_all_deployments_cooldown_error
fix(router): name the all-deployments-in-cooldown error on 429 responses
2026-09-14 12:39:42 -07:00
ryan
9cdfe311e9 test: assert only the org ceiling /team/info returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:39:18 +00:00
ryan
5ddb0fe691 fix(ui): keep the team model select loading until /team/info reports the org ceiling
useTeam seeds its cache from the team list, which has no organization_models, so the select briefly rendered unfiltered. Also drop the Prisma include assertion from the backend test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:38:05 +00:00
Yassin Kortam
7c55c4ba7d
Merge pull request #40925 from BerriAI/litellm_proxy_error_headers_from_litellm_response_headers
fix(proxy): forward provider request id headers on mapped error responses
2026-09-14 12:36:59 -07:00
ryan-crabbe-berri
3f42a4fc98 test: suppress TQ008 on the two module-state patches
team_info resolves prisma_client and get_all_team_memberships from module
state, so there is no collaborator to inject; the two sibling team_info tests
patch the same way.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
ryan-crabbe-berri
b6a5563d05 fix(ui): let team admins grant a team all proxy models
The team edit form only offered "All Proxy Models" when the dashboard could
read the parent organization, and /organization/info 403s for anyone who is
not a proxy admin or an admin of that org. A team admin with the internal_user
proxy role therefore saw only "No Default Models", which is the opposite of
what they wanted, and had no way to grant their team everything on the proxy.

/team/info now reports the parent org's model ceiling as organization_models,
which the same authorization already admits, and ModelSelect reads the ceiling
from the team it is editing before falling back to the organization. That also
makes the individual model list respect the org's allow-list instead of
listing every proxy model to a caller whose save would be rejected.

useTeam was typed as Team while returning the /team/info envelope, so its one
other caller unwrapped it behind a cast. It now returns the team itself.

Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
2026-09-14 19:30:44 +00:00
yassin
f9d0cd6409 chore(proxy): drop explanatory docstrings from credentials hint helper and tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:30 +00:00
yassin
ebcd9bcb18 fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:29:11 +00:00
yassin
ff02390685 test(auth): drop docstrings from team membership tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:26:14 +00:00
yassin
53c8a82a31 Merge remote-tracking branch 'origin/main' into pr-40817
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/caching/redis_cache.py
#	litellm/proxy/proxy_server.py
#	tests/test_litellm/caching/test_redis_cache.py
2026-09-14 19:11:03 +00:00
yassin
b00bb35563 fix(auth): drop the membership write-epoch and background Redis replicate, load membership lazily
Move the cache-miss marker out of litellm.constants into auth_checks (CodeQL cyclic import) and stop
logging user_id/team_id in the lookup failure (CodeQL log injection).

Write the membership row through DualCache synchronously again instead of a background Redis task
guarded by a bounded write-epoch map: the epoch was sampled after the Prisma read, so an invalidate
that raced the read could be cached as current, and eviction of the epoch entry could let an old
Redis write land. The synchronous write keeps invalidate_team_member_spend_state authoritative.

Lookup failures return None again (fail-open like main) instead of 503, and the load is skipped on
routes that neither resolve a model nor run budget checks.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:06:29 +00:00
yassin
ba171d9eb4 fix(proxy): hide default credentials login hint when UI_PASSWORD is set
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:06:04 +00:00
yassin
ce0301c23f refactor(proxy): drop redundant docstrings from upload allowlist helpers and tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 19:00:49 +00:00
yassin
82ef6ea6ab feat(proxy): add general_settings.allowed_file_extensions for /v1/files
Opt-in allowlist for upload filename extensions, checked before the existing blocked_file_extensions blocklist and mapped through the same upload validation failure path. None keeps today's behaviour, [] rejects every upload, matching is case-insensitive on both sides, and a filename with no extension is rejected when the allowlist is set.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 18:53:29 +00:00
Shivi Jain
75b16df6fa fix(auth): keep prefetched org entries on the 5s getter TTL
Organization mutations do not evict those cache keys, so stretching prefetch to the management TTL would leave stale org grants in L1.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:13:49 +00:00
Shivi Jain
abaa2f8b81 fix(auth): put TQ008 suppressions on the patch call lines
The test-quality gate attributes the comment to the `patch(` line, so reasons on the closing paren did not count and lint failed after format started passing.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 18:13:49 +00:00
Shivi Jain
53ba8b9866 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
Shivi Jain
70ddc7e492 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
Shivi Jain
5457f48290 fix(auth): load team membership once per request and skip prisma on an L1 hit
common_checks was querying get_team_membership twice, and DualCache awaited Redis SET on the auth path, so LRU eviction plus a hung Redis write showed up as two postgres spans
2026-09-14 18:13:49 +00:00
yassin
4b39344d13 fix(proxy): only treat a missing user as no memberships on team list, surface db errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 16:49:54 +00:00
yassin
dc30be006b refactor(proxy): read org admin's own team ids via get_user_object(check_db_only=True)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 16:27:31 +00:00
yassin
d643909789 fix(proxy): read org admin's own team ids from db and assert v2 list results
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 16:03:23 +00:00
yassin
8d9edfc03c fix(proxy): keep /team/list self query for org admins membership-only across orgs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 15:40:41 +00:00
yassin
58c3ccf668 fix(proxy): keep org admins' own team memberships in other orgs visible on team list
An org admin listing their own teams on GET /team/list and GET /v2/team/list
only saw teams in the orgs they administer. Teams they belong to in other
orgs were dropped because the org scope and the membership scope were ANDed.
A self query now unions the two, while a query for another user keeps the
org boundary intersection.

Resolves LIT-3723

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 15:27:06 +00:00
yassin
10f411e60d fix(router): name the all-deployments-in-cooldown error on 429 responses
RouterRateLimitError now carries the model group's deployment ids so it
can tell when every deployment is cooled down, and exposes that as
type=all_deployments_in_cooldown with an explicit message. A partial
cooldown keeps type=rate_limit_error. Either way the proxy no longer
reports type=internal_server_error next to code 429

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:34:16 +00:00
yassin
e0cf63c5fa Merge remote-tracking branch 'origin/main' into litellm_per_issuer_virtual_key_claim_field 2026-09-13 05:00:54 +00:00
yassin
268b9b167f Merge remote-tracking branch 'origin/main' into litellm_proxy_error_headers_from_litellm_response_headers 2026-09-13 04:24:01 +00:00
ryan-crabbe-berri
760119681c
Merge pull request #40814 from BerriAI/litellm_gate_health_services_alert_tests
fix(proxy): gate the webhook test alert on proxy admins
2026-09-12 21:13:30 -07:00