Commit graph

74 commits

Author SHA1 Message Date
Moe Khalil
e0545a787b Merge remote-tracking branch 'origin/main' into litellm_jev_autorouter_launch_1789767495 2026-09-19 17:35:42 +00:00
ryan
b4c5f6fa44 Merge remote-tracking branch 'origin/main' into litellm_team_member_budget_link_default
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/proxy/management_helpers/utils.py
#	tests/test_litellm/proxy/management_helpers/test_management_helpers_utils.py
2026-09-18 23:05:03 +00:00
Moe Khalil
86e079d7a8 feat(auto-router): integrate JEV context and usage accounting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:38:15 +00:00
ryan
2d61fa66b1 fix(proxy): lock teams in sorted team id order and keep existing member budgets on re-add
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
de4b520153 fix(proxy): track team member spend when the member has no budget
add_new_member only wrote a LiteLLM_TeamMembership row when a budget id
resolved, and the spend writer used update_many so a missing row failed
silently. Members without a budget therefore never accrued per-member spend.

The membership row is now always upserted (budget_id NULL when no budget
applies) and the spend write is an upsert so members added before this fix
start accruing on their next request.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
ryan-crabbe-berri
6b54238083
Merge pull request #40807 from BerriAI/litellm_service_account_team_key_mgmt
feat(keys): let team service account keys use key management endpoints for their own team
2026-09-18 12:58:41 -07:00
ryan-crabbe-berri
4f70b88a1f
Merge pull request #41707 from BerriAI/litellm_jwt_mapping_cache_evict_on_bulk_key_delete
fix(proxy): evict jwt key mapping cache on user, team, org, and bulk key deletion
2026-09-18 08:41:31 -07:00
shivam
a052da6974 feat(keys): let team service account keys use key management endpoints for their own team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
ryan
e79d03e604 fix(proxy): evict jwt key mapping cache on bulk user and team member deletion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:00:17 +00:00
mateo-berri
1e7e5b695f fix(router): reject a blank Jev api_key so it cannot pair with a caller-chosen api_base 2026-09-17 16:38:54 -07:00
mateo-berri
7f581f6bc7 fix(router): keep the TypeSafe key off caller-chosen Jev endpoints 2026-09-17 16:09:31 -07:00
ryan
ab99be9dad test(team): cover team_member_budget propagation and per-member isolation end to end
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 01:43:21 +00:00
ryan
5b5bbac769 fix(team): link new members to the shared team member budget so /team/update applies to them
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 01:16:21 +00:00
yassin
264305de23 fix(proxy): keep access-group raw SQL writes on the writer while writer_unavailable is stale
A stale RoutingPrismaWrapper.writer_unavailable flag made WriterPinnedClient hand back the routed wrapper, where query_raw is classified as a read, so the access-group UPDATE statements behind /key/regenerate, /key/generate with access_group_ids and model rename/delete went to the read replica and failed with SQLSTATE 25006. Route those raw statements through the underlying writer regardless of the flag; a raw SQL write has no replica fallback. WriterPinnedClient keeps yielding to the replica for degraded reads.

The model sync's backing-row count stays on the writer too: it runs right after the row delete/update on the writer and a lagging replica could still report the removed row, which would leave the group naming a model nobody serves.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 19:55:07 +00:00
tin-berri
3ac79757f4
Merge pull request #41175 from BerriAI/litellm_team_member_auto_routers
feat(auto-router): allow opted-in team members to manage their routers
2026-09-15 11:19:58 -07:00
ryan
6ea1085bc3 Merge branch 'main' into litellm_bulk_user_delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:03:15 +00:00
Tin Chi Lo
109ca70f66 feat(auto-router): allow opted-in team members to manage their routers 2026-09-14 23:56:55 -07:00
ryan
98ea14758f fix(proxy): only widen an id-only bulk member delete to its email when the user lists the team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:09:57 +00:00
ryan
5ace6fa731 fix(proxy): match an id-only bulk member delete against a legacy email-only roster entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:51:15 +00:00
ryan
ca04b03c2c feat(proxy): move bulk user delete and team member delete under /management/v1
Replaces POST /user/bulk_delete and POST /team/bulk_member_delete with
POST /management/v1/users/bulk_delete and
POST /management/v1/teams/{team_id}/members/bulk_delete per the Management
API modernization one-pager: {data} envelopes, application/problem+json
errors with urn:litellm:error:* types, 422 on unknown body fields, 400 on
unknown query params, 403 on authorization failures, 404 on unknown team.

Route checks now match parametrized management/v1 paths so team-scoped
callers reach the endpoint's own authorization and get a 403 problem
instead of the generic 401.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:31:07 +00:00
ryan
96332f75e8 refactor(proxy): move bulk user creation to POST /management/v1/users/bulk
Follows the Management API modernization design: plural resource under
/management/v1, {data, meta} response envelope, unknown request fields
rejected with 422, and RFC 9457 problem+json for request-level errors.
Body validation failures under /management/v1 now answer 422 instead of
the 400 query-parameter problem. /user/bulk_new is removed.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:39 +00:00
ryan
c25498b66d fix(proxy): remove a bulk-deleted user's email-only roster entries without touching same-email teammates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 06:08:42 +00:00
ryan
dbb4de7bc2 fix(proxy): match bulk-deleted users on team rosters by user_id only and give /team/bulk_member_delete the 60s batch timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 05:59:19 +00:00
ryan
82872c9627 fix(proxy): run /user/bulk_delete team rewrites and user deletes in one transaction
Lock affected teams in sorted order inside a single 60s transaction so a
failure on any team rolls back every rewrite and every user row delete.
PrismaClient.tx() gains an optional timeout for the larger batch.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 05:35:55 +00:00
ryan
595aba3cb0 fix(proxy): evict deleted keys from the auth cache and make bulk user deletion transactional
/user/bulk_delete now deletes the users' keys, invitation links, org and team
memberships and user rows in one transaction and reports a rolled-back batch
per row instead of leaving partial deletes behind. Both bulk endpoints evict
the deleted keys (and deleted user objects) from the auth cache, so a deleted
key stops authenticating immediately rather than at TTL expiry.

/team/bulk_member_delete rejects member rows that carry both user_id and
user_email, reports repeated rows as duplicates, and only cleans up keys and
memberships of members it actually matched.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 05:12:34 +00:00
ryan
cd9c39921b feat(proxy): add POST /user/bulk_delete and POST /team/bulk_member_delete
Batch user deletion that also removes each user from every team they belong to, and batch removal of many members from one team. Each touched team is rewritten once under the team advisory lock from a roster re-read under that lock

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 04:55:39 +00:00
ryan
0c9e0c407e fix(proxy): keep a team on a /user/bulk_new row when the roster already lists that user id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 03:37:20 +00:00
ryan
4285f1dfb0 fix(proxy): do not claim rows a concurrent request inserted when /user/bulk_new create_many fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 02:39:47 +00:00
ryan
24a1d772b6 fix(proxy): keep key policy fields and reconcile committed rows in /user/bulk_new
Rows opting into auto_create_key lost blocked, permissions, aliases, config,
agent_id, budget_fallbacks and budget_limits before reaching the key helper.
When create_many commits but the response is lost, re-read which ids landed
and retry only the rest so committed rows report success and get their teams.
Regenerate schema.d.ts and allowlist the endpoint in the Terraform audit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 01:48:44 +00:00
ryan
ff5b59b173 feat(proxy): add POST /user/bulk_new for batched user and team membership creation
Creates up to 500 internal users in one request with set-based validation,
a single create_many for user rows, and one locked write per referenced team.
Rows fail independently, keys are opt-in per row via auto_create_key, and
send_invite_email is rejected for the batch.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 01:17:19 +00:00
devin-ai-integration[bot]
6cc13e07a6
feat(proxy): granular key/team access control for Claude Code marketplace plugins (#40518)
Adds object_permission.skills to keys and teams, enforces it on
/claude-code/marketplace.json?key=, /claude-code/plugins and
/claude-code/plugins/{name}, and exposes an Allowed Skills selector in
the key and team create/edit forms of the Admin UI

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-10 14:26:09 -07:00
devin-ai-integration[bot]
a46a076b2a
fix(proxy): reject ambiguous name or alias keys in mcp_tool_permissions on write (#39947)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-05 14:22:26 -07:00
ryan-crabbe-berri
c5c10bc91f feat(access-groups): resolve resource names on access group responses
The access group detail page rendered MCP servers, agents, attached teams and keys as bare ids, so an admin had to look each one up elsewhere to audit a group

Every access group response now also carries access_mcp_servers, access_agents, assigned_teams and assigned_keys as {id, name} pairs. Names come from the DB rows first and fall back to config-declared MCP servers and agents (including legacy agent ids), resolved with one query per table across all groups in a list call. The existing *_ids columns are unchanged

The UI renders the name with the id in a tooltip, links teams and keys to their detail pages, and shows the raw id only when nothing resolves
2026-09-04 16:08:50 -07:00
Mateo Wang
34d4f7f8ae
fix: 1.99.0-rc2 UI bug batch (empty org on key create, session pagination, access group rename/delete) (#39436)
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create

* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side

* fix(proxy): keep access groups in sync when a model is renamed or deleted

* fix(proxy): cap the Request Logs conversation total like the row total

* fix(proxy): judge access group backing by the database for db models

A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.

* fix(ui): keep the conversation badge when an MCP call represents a conversation

A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.

* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): type session spend aggregates

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): satisfy request logs lint budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): cap per-session model aggregation in request logs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: ratchet type-discipline budget after staging merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): send an explicit null when the key edit form clears the organization

Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-03 08:58:48 -07:00
devin-ai-integration[bot]
ffc0a8e428
fix: run access group key sync UPDATEs on the writer, not the read replica (#39128)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 10:51:17 -07:00
ryan-crabbe-berri
4118db5a50 test: drop litellm-internal patches from the cross-region alias test (TQ008)
The pure helpers express the same regression: the save-side drop must
leave the alias in place, and two registries must expand it to their
own ids. The full validate path is already covered by the
persists-verbatim test and the live e2e test.
2026-09-01 08:23:26 -07:00
ryan-crabbe-berri
0d7035989c fix(mcp): persist alias MCP grants verbatim instead of rewriting to local server ids
Since PR #29128, key create/update/regenerate resolved every
object_permission.mcp_servers entry against the saving instance's
DB + config registry and persisted the resolved server ids. For
config-loaded servers the id is derived from a hash of the regional
URL, so in a shared-database multi-region deployment the rewrite baked
one region's ids into the row and every other region denied the key.
Grants written before v1.88.0 kept the raw alias and kept working,
which is why only newly provisioned keys broke.

Keep the validation and the stale-entry drop (the LIT-3278 fix), but
persist the caller's original identifiers for everything that resolves.
Read-time expand_permission_list already maps a name to each region's
local server id.
2026-09-01 08:10:36 -07:00
devin-ai-integration[bot]
6b1844442c
fix(key_management): allow /key/update to keep or shrink MCP server grants the key already holds (#38463)
* fix(key_management): allow /key/update to keep or shrink MCP server grants the key already holds

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(key_management): reuse key row's included object_permission instead of a second lookup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-27 12:51:17 -07:00
Yassin Kortam
6c0c91c5ad
fix(team): serialize member_add, member_delete, and delete under the team's advisory lock (#37969)
* fix(proxy): make /team/member_delete's four cleanups atomic

The team roster update, the user.teams update, the team membership
delete, and the team-scoped verification token delete ran as four
sequential writes with no transaction around them, so a failure
between any two left the removal half applied. Thread a single
prisma transaction through all four writes, following the same
tx.<table> pattern /team/member_add and /team/member_update already
use, so either all four land or none do.

* fix(team): serialize member_add, member_delete, and delete under the team's advisory lock

/team/member_add validated a team exists and then wrote the user's teams array and
a membership row without holding anything across that gap, so a /team/delete could
commit its reference sweeps in between and leave a member pointing at a team id that
no longer exists. The write path already re-read members_with_roles under a row lock
before this change, but SELECT ... FOR UPDATE can deadlock with the access-group
endpoints, which lock an access group and then a team.

member_add now takes pg_advisory_xact_lock(hashtext(team_id)) before re-reading the
team and only writes if it is still there, so a delete that already committed is
visible before any write happens. delete_team takes the same lock around its own
row delete and reference sweep, so the two requests can never interleave: whichever
acquires the lock first runs to completion before the other's read can proceed.

Dropping the row lock from member_add's read also dropped the incidental protection
it gave against a concurrent member_delete, which still wrote from the snapshot it
validated against, unlocked, and could silently overwrite whatever member_add had
just committed. member_delete now takes the same advisory lock and re-reads the
roster under it before computing its own write, so it can never resurrect a member
by overwriting from stale data.

Resolves LIT-5544

* fix(team): run member writes on the advisory lock's transaction

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(team): keep member writes on the lock holder's connection after merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(team): keep the transactional member create an upsert on user_id

The transaction path was creating the email-identified user row outright, where the
regular client path upserts on user_id. Share one upsert helper between both member
paths so the create stays idempotent on the lock holder's connection.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(team): read member_delete's user and key rows on the lock-holding transaction

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-25 21:55:11 +00:00
yuneng-jiang
6a0d03914c
test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)
* test: drop the cwd-relative sys.path.insert calls from the test suite

TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.

Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.

Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.

* test: drop the duplicate imports the sys.path sweep exposed to F811

* test(pre-call-utils): restore the os import the new bedrock tests need
2026-08-22 09:25:58 -07:00
yuneng-jiang
49da936efb
test(audit-logs): let monkeypatch own the audit log and s3 callback globals (#37842)
Fifteen tests assigned litellm.audit_log_callbacks, s3_callback_params or
s3_audit_callback_params directly and leaned on two autouse fixtures to put
them back. monkeypatch.setattr does that at the point of use, so each test now
says what it sets, including the one that swaps the value mid-test to prove the
cache does not serve the stale params.

The fixtures keep only the work monkeypatch cannot do: the per-test empty
callback list, and clearing the logger and audit caches around each test.
2026-08-21 20:16:10 -07:00
ryan-crabbe-berri
b76def0e5d
test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.

PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.

PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
2026-08-20 20:24:49 -07:00
ryan-crabbe-berri
a112ba5f63
test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748)
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions

`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.

Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.

The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.

Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.

* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
2026-08-20 19:36:26 -07:00
ryan-crabbe-berri
680bcfd8aa
test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731)
* test(lint): ban blind pytest.raises(Exception) with ruff B017

A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.

All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.

Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.

* fix(test): widen the openai batch-dispatch assertion to OpenAIError

The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
2026-08-20 18:09:42 -07:00
devin-ai-integration[bot]
3a04860122
feat(proxy)!: default audit logs on for enterprise licenses (#37518)
* feat(proxy): enable audit logs by premium license

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): support premium audit logging mocks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): disable audit logging for key rotation mocks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 18:49:21 -07:00
Yassin Kortam
6704a105ee
fix(access groups): sync assigned_team_ids from the team write paths (#36825) 2026-08-14 04:45:36 +00:00
yuneng-jiang
cd87fee9c5
feat(team): custom metadata validation hook for team create and update (#33353)
* feat(team): custom metadata validation hook for team create and update

Operators can point general_settings.custom_team_metadata_validate at an
async Python function that validates team metadata before /team/new,
POST /team/update, and PATCH /team/{team_id} commit their writes. The
hook receives the metadata that will actually be written (the merged
result on PATCH) plus the stored metadata and requester context, and
fails closed: a rejected value returns the function's own message as a
400 while any exception or timeout blocks the write with a configurable
generic message as a 503. Premium-gated like enforced_params.

* fix(team): validate metadata before model alias writes and strip system keys from validator input

Review follow-ups on the team metadata validation hook: run the validator
before the model_aliases table insert so a rejected create leaves no
orphaned model rows, strip system-managed keys from existing_metadata so
the validator sees symmetric input on both fields, and accept class
instances exposing an async __call__ as validators. Adds a three-way
validator implementation matrix (allowlist function, HTTP-service-backed
function, immutability-enforcing class instance) driven through the real
create, update, and patch endpoints, including an HTTP stub service and
outage coverage.

* test(team): run the metadata validation matrix against the DB-backed proxy in CI

Adds the validator matrix to the proxy_store_model_in_db_tests CircleCI
job so every scenario runs full e2e against a Postgres-backed proxy. The
proxy config registers a dispatching validator that routes each request
to one of the three implementations via a metadata key and accepts
anything that does not opt in, keeping the rest of the suite unaffected.
CI starts a stand-in cost center service on the host for the HTTP-backed
implementation, reached from the container via host.docker.internal, and
the outage path targets a closed port to prove the fail-closed 503
without stopping services.

* feat(ui): edit team metadata as key-value pairs in team create and edit forms

The team create and edit forms asked for metadata as a raw JSON blob in a
textarea buried under Additional Settings. Both forms now render a key-value
pair editor directly under the TPM/RPM limit fields, backed by a shared
MetadataKeyValueFields component. Values round-trip losslessly: non-string
values display as JSON and parse back to their typed form on save, and
JSON-ambiguous strings are quoted so their type survives the trip. The edit
form hides UI-managed keys (logging, guardrails, model rate limits, etc.)
that dedicated controls already own and re-add on save.

* fix(ui): explain typed JSON parsing in the team metadata help text

* feat(team): schema-driven metadata fields from team_metadata_schema config

* refactor(team): render schema metadata fields as locked key-value rows, drop allowed_values

* refactor(team): schema fields reduce to key and label, tag-rendered keys, clean rejection toasts

* refactor(ui): prepopulate declared metadata keys as ordinary key-value rows

* fix(team): let non-admin dashboard users read the team metadata schema

* test(proxy): pin timeout wiring, boundary, and error-message contracts for team metadata validation

* fix(proxy): use pooled async httpx client in the e2e team metadata validator example

* refactor(team): satisfy staging lint ratchets inherited by the merge
2026-08-03 18:37:45 -07:00
yuneng-jiang
64aad5877a
fix(proxy): restore atomic user upsert when adding team members (#34457)
* fix(proxy): restore atomic user upsert when adding team members

Parallel /team/new calls naming the same not-yet-existing member were
returning 500 "Unique constraint failed on the fields: (`user_id`)".

The upsert in add_new_member passed an empty update branch. Prisma only
compiles an upsert down to a single INSERT ... ON CONFLICT when that branch
writes something; with an empty one it emits SELECT-then-INSERT instead, so
concurrent requests all read "no such user" and all insert. Postgres
statement logs confirm it: the empty form logs BEGIN/SELECT/INSERT/COMMIT,
the non-empty form logs INSERT ... ON CONFLICT ("user_id") DO UPDATE SET.

Re-state user_id in the update branch as a no-op so the native upsert path
comes back. The teams append stays in the filtered update below it, so an
already-existing member still cannot pick up a duplicate team id.

tests/test_team.py::test_team_new failed 9 of 15 runs against a live proxy
before this and 0 of 15 after. The existing unit test asserted only that
upsert had been called on a mock, so it passed either way; it now pins the
shape of both branches and fails when the update branch goes back to empty.

* test: point the live codex tests at gpt-5.3-codex

OpenAI deprecated gpt-5.2-codex, so test_openai_codex and
test_openai_codex_stream started failing against the live API with
model_not_found. gpt-5.3-codex is the current codex model; both tests pass
on it. The remaining gpt-5.2-codex references in the suite are mocked
transformation tests and are unaffected.

* test(e2e): update models page specs for the shared DataTable

The DataTable migration in #34363 changed three things the models page
specs were pinned to, and five tests went red.

Row click no longer opens the detail view; the Model ID cell owns that
now, so both specs click its `model-id-<id>` test id instead of the row.
The search box placeholder switched from an ASCII "..." to a real
ellipsis, so the specs use getByPlaceholder with a substring instead of
an exact attribute match that punctuation can break again. The results
count moved from `models-results-count` ("Showing 1 - 50 of 137 results")
to the shared pagination's `pagination-range` ("Showing 1-50 of 137").

The Team-BYOK test also filtered rows on the team alias, which the Team
ID column has never rendered in either the old or the new table; it
filters on the team id now, which is what the column actually shows and
what the assertion's own comment intends.

Verified against a local proxy serving a fresh build with the seeded
e2e postgres and mock upstream: all five failing tests pass, and the
full suite is 82 passed / 4 skipped at CI parity (workers=1).
2026-07-24 02:22:34 +00:00
Yassin Kortam
38467631b6
fix(scim): use members_with_roles as the source of truth for group membership (#34162)
* fix(scim): use members_with_roles as the source of truth for group membership

SCIM group provisioning tracked membership inconsistently. Team creation and
the real team endpoints persist membership in members_with_roles (and each
member's user.teams), but the SCIM group PATCH handler and the GET /Groups
listing read the legacy team.members String[] column, which team creation never
populates. Seeding a PATCH result from that empty column made an Okta "add
member" operation recompute the member set from scratch and silently drop
everyone already in the team, so users ended up missing from the groups they
were provisioned into. Reading the same empty column on GET /Groups reported an
empty member list back to the IdP, which drove repeated re-provisioning.

Separately, add_new_member appended the team id to user.teams with an
unconditional array push. Under the concurrent group PATCHes an IdP sends during
a reconcile, each request passed the members_with_roles duplicate check and
pushed, so user.teams accumulated duplicate ids for the same team. A duplicate
also breaks auth logic that keys off the number of teams a user belongs to.

Read current membership from members_with_roles in the SCIM group PATCH seed and
the GET /Groups listing, and make the user.teams append idempotent via a
filtered update that no-ops once the team is present.

Resolves LIT-4283

* fix(scim): address review; atomic user-creation and stop writing legacy members

Keep the concurrent-safe team append but create the user via an atomic upsert
(create-or-update) instead of a check-then-create, so provisioning the same new
user concurrently cannot race into a duplicate-key failure; the team is still
appended idempotently by a filtered update so an existing user gets no duplicate
team id. Stop writing the legacy team.members column in the group PATCH apply so
the only membership record is the source of truth (members_with_roles plus each
member's user.teams), reconciled by team_member_add/team_member_delete.

Tests: existing add_new_member and team-creation mocks updated to the upsert
plus filtered-append shape, and new tests cover atomic creation and that the
PATCH apply does not write the legacy members column.
2026-07-22 13:58:15 -07:00
ryan-crabbe-berri
57ca48a863
feat(mcp): add all-proxy-mcpservers sentinel to grant teams every MCP server (#32012)
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server

Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain
access to every MCP server on the proxy without listing each id. The
sentinel expands to the live registry at request time, so a server added
later is picked up with no change to the team's stored permission. The team
ceiling that validates a key's MCP scope expands the sentinel too, so a key
can be scoped to any server (including one registered after the team) and
still pass subset validation

Expose the option in the team create and edit forms via a new exclusive
"All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing
"No MCP Servers" sentinel

* Update litellm/proxy/management_helpers/object_permission_utils.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key

The sentinel was expanded inside the shared expand_permission_list, which
also feeds the key, org, end_user and agent resolvers. A key whose stored
object_permission ever held all-proxy-mcpservers (a stale write, a
configured default, or a bug) would silently resolve to every MCP server at
runtime, and a teamless key had nothing to cap it, so all servers got
injected. Only write-time validation stripping the value stood between that
value and a full grant

Move the expansion out of expand_permission_list and into
_get_allowed_mcp_servers_for_team so the sentinel is honored only where it is
settable (a team). Anywhere else it now passes through as an inert literal
that matches no registered server and is denied downstream. Reserved-id
protection already blocks a real server from taking that id

* fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel

Granting a team every MCP server on the proxy is a proxy-wide authorization
decision, but team create/update let any caller who can manage a team set
object_permission.mcp_servers, with no ceiling check. Org admins reach
/team/update by default (org_admin_allowed_routes) and _verify_team_access
also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers
and self-grant their team access to every MCP server on the proxy, including
servers never assigned to that team

Gate the grant in new_team and update_team: a non-proxy-admin cannot add the
all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a
team a proxy admin already scoped to all-proxy can still be edited by a team
admin without being forced to strip the sentinel. The UI only offers the
"All Proxy MCP Servers" option to proxy admins in the team create and edit
forms

* fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team

A team scoped to the all-proxy-mcpservers sentinel could be opened in the team
edit form by a team admin or org admin (canEditTeam admits them), but the
"All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the
proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored
sentinel was hydrated into the selected value with no matching Select.Option, so
antd showed the raw all-proxy-mcpservers literal as a chip, and adding another
server could persist a mixed [all-proxy-mcpservers, <id>] value.

Render the option whenever the sentinel is present in the value, not only when
the caller may grant it, and drive the real-option disabling off presence too so
the selection stays exclusive. A non-proxy-admin now sees the friendly label
read-only and cannot build a mixed state; only a proxy admin can newly add it,
which the backend already enforces.

Adds regression tests: the selector shows the friendly option (not the raw
literal) when the sentinel is stored but the grant flag is off, plus exclusive
emit and disabled-real-options coverage, and MCPServerPermissions renders the
green "All" state instead of the raw sentinel string.

* fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip

antd renders a Select option's children inside the selected tag, so the
all-proxy option showed both "All Proxy MCP Servers" and the green "All servers"
type-hint in the chip, which say the same thing. Collapse the option to a single
green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly
without the duplication.

* fix(ui): color the all-proxy MCP label blue to match server chips

Use the same blue (#1890ff) as regular MCP server entries for the
"All Proxy MCP Servers" option/chip instead of green.

* fix(ui): make the all-proxy MCP permissions display blue, not green

Match the blue used by the selector chip and regular server entries so the
"All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent
across the team/key/org detail views. The red "Blocked" state for
no-mcp-servers is unchanged.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-07-03 13:59:28 -07:00