The access group detail page rendered MCP servers, agents, attached teams and keys as bare ids, so an admin had to look each one up elsewhere to audit a group
Every access group response now also carries access_mcp_servers, access_agents, assigned_teams and assigned_keys as {id, name} pairs. Names come from the DB rows first and fall back to config-declared MCP servers and agents (including legacy agent ids), resolved with one query per table across all groups in a list call. The existing *_ids columns are unchanged
The UI renders the name with the id in a tooltip, links teams and keys to their detail pages, and shows the raw id only when nothing resolves
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create
* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side
* fix(proxy): keep access groups in sync when a model is renamed or deleted
* fix(proxy): cap the Request Logs conversation total like the row total
* fix(proxy): judge access group backing by the database for db models
A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.
* fix(ui): keep the conversation badge when an MCP call represents a conversation
A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.
* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type session spend aggregates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): satisfy request logs lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap per-session model aggregation in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: ratchet type-discipline budget after staging merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send an explicit null when the key edit form clears the organization
Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The pure helpers express the same regression: the save-side drop must
leave the alias in place, and two registries must expand it to their
own ids. The full validate path is already covered by the
persists-verbatim test and the live e2e test.
Since PR #29128, key create/update/regenerate resolved every
object_permission.mcp_servers entry against the saving instance's
DB + config registry and persisted the resolved server ids. For
config-loaded servers the id is derived from a hash of the regional
URL, so in a shared-database multi-region deployment the rewrite baked
one region's ids into the row and every other region denied the key.
Grants written before v1.88.0 kept the raw alias and kept working,
which is why only newly provisioned keys broke.
Keep the validation and the stale-entry drop (the LIT-3278 fix), but
persist the caller's original identifiers for everything that resolves.
Read-time expand_permission_list already maps a name to each region's
local server id.
* fix(key_management): allow /key/update to keep or shrink MCP server grants the key already holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(key_management): reuse key row's included object_permission instead of a second lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make /team/member_delete's four cleanups atomic
The team roster update, the user.teams update, the team membership
delete, and the team-scoped verification token delete ran as four
sequential writes with no transaction around them, so a failure
between any two left the removal half applied. Thread a single
prisma transaction through all four writes, following the same
tx.<table> pattern /team/member_add and /team/member_update already
use, so either all four land or none do.
* fix(team): serialize member_add, member_delete, and delete under the team's advisory lock
/team/member_add validated a team exists and then wrote the user's teams array and
a membership row without holding anything across that gap, so a /team/delete could
commit its reference sweeps in between and leave a member pointing at a team id that
no longer exists. The write path already re-read members_with_roles under a row lock
before this change, but SELECT ... FOR UPDATE can deadlock with the access-group
endpoints, which lock an access group and then a team.
member_add now takes pg_advisory_xact_lock(hashtext(team_id)) before re-reading the
team and only writes if it is still there, so a delete that already committed is
visible before any write happens. delete_team takes the same lock around its own
row delete and reference sweep, so the two requests can never interleave: whichever
acquires the lock first runs to completion before the other's read can proceed.
Dropping the row lock from member_add's read also dropped the incidental protection
it gave against a concurrent member_delete, which still wrote from the snapshot it
validated against, unlocked, and could silently overwrite whatever member_add had
just committed. member_delete now takes the same advisory lock and re-reads the
roster under it before computing its own write, so it can never resurrect a member
by overwriting from stale data.
Resolves LIT-5544
* fix(team): run member writes on the advisory lock's transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep member writes on the lock holder's connection after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep the transactional member create an upsert on user_id
The transaction path was creating the email-identified user row outright, where the
regular client path upserts on user_id. Share one upsert helper between both member
paths so the create stays idempotent on the lock holder's connection.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): read member_delete's user and key rows on the lock-holding transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
Fifteen tests assigned litellm.audit_log_callbacks, s3_callback_params or
s3_audit_callback_params directly and leaned on two autouse fixtures to put
them back. monkeypatch.setattr does that at the point of use, so each test now
says what it sets, including the one that swaps the value mid-test to prove the
cache does not serve the stale params.
The fixtures keep only the work monkeypatch cannot do: the per-test empty
callback list, and clearing the logger and audit caches around each test.
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions
`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.
Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.
The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.
Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.
* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
* test(lint): ban blind pytest.raises(Exception) with ruff B017
A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.
All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.
Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.
* fix(test): widen the openai batch-dispatch assertion to OpenAIError
The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
* feat(team): custom metadata validation hook for team create and update
Operators can point general_settings.custom_team_metadata_validate at an
async Python function that validates team metadata before /team/new,
POST /team/update, and PATCH /team/{team_id} commit their writes. The
hook receives the metadata that will actually be written (the merged
result on PATCH) plus the stored metadata and requester context, and
fails closed: a rejected value returns the function's own message as a
400 while any exception or timeout blocks the write with a configurable
generic message as a 503. Premium-gated like enforced_params.
* fix(team): validate metadata before model alias writes and strip system keys from validator input
Review follow-ups on the team metadata validation hook: run the validator
before the model_aliases table insert so a rejected create leaves no
orphaned model rows, strip system-managed keys from existing_metadata so
the validator sees symmetric input on both fields, and accept class
instances exposing an async __call__ as validators. Adds a three-way
validator implementation matrix (allowlist function, HTTP-service-backed
function, immutability-enforcing class instance) driven through the real
create, update, and patch endpoints, including an HTTP stub service and
outage coverage.
* test(team): run the metadata validation matrix against the DB-backed proxy in CI
Adds the validator matrix to the proxy_store_model_in_db_tests CircleCI
job so every scenario runs full e2e against a Postgres-backed proxy. The
proxy config registers a dispatching validator that routes each request
to one of the three implementations via a metadata key and accepts
anything that does not opt in, keeping the rest of the suite unaffected.
CI starts a stand-in cost center service on the host for the HTTP-backed
implementation, reached from the container via host.docker.internal, and
the outage path targets a closed port to prove the fail-closed 503
without stopping services.
* feat(ui): edit team metadata as key-value pairs in team create and edit forms
The team create and edit forms asked for metadata as a raw JSON blob in a
textarea buried under Additional Settings. Both forms now render a key-value
pair editor directly under the TPM/RPM limit fields, backed by a shared
MetadataKeyValueFields component. Values round-trip losslessly: non-string
values display as JSON and parse back to their typed form on save, and
JSON-ambiguous strings are quoted so their type survives the trip. The edit
form hides UI-managed keys (logging, guardrails, model rate limits, etc.)
that dedicated controls already own and re-add on save.
* fix(ui): explain typed JSON parsing in the team metadata help text
* feat(team): schema-driven metadata fields from team_metadata_schema config
* refactor(team): render schema metadata fields as locked key-value rows, drop allowed_values
* refactor(team): schema fields reduce to key and label, tag-rendered keys, clean rejection toasts
* refactor(ui): prepopulate declared metadata keys as ordinary key-value rows
* fix(team): let non-admin dashboard users read the team metadata schema
* test(proxy): pin timeout wiring, boundary, and error-message contracts for team metadata validation
* fix(proxy): use pooled async httpx client in the e2e team metadata validator example
* refactor(team): satisfy staging lint ratchets inherited by the merge
* fix(proxy): restore atomic user upsert when adding team members
Parallel /team/new calls naming the same not-yet-existing member were
returning 500 "Unique constraint failed on the fields: (`user_id`)".
The upsert in add_new_member passed an empty update branch. Prisma only
compiles an upsert down to a single INSERT ... ON CONFLICT when that branch
writes something; with an empty one it emits SELECT-then-INSERT instead, so
concurrent requests all read "no such user" and all insert. Postgres
statement logs confirm it: the empty form logs BEGIN/SELECT/INSERT/COMMIT,
the non-empty form logs INSERT ... ON CONFLICT ("user_id") DO UPDATE SET.
Re-state user_id in the update branch as a no-op so the native upsert path
comes back. The teams append stays in the filtered update below it, so an
already-existing member still cannot pick up a duplicate team id.
tests/test_team.py::test_team_new failed 9 of 15 runs against a live proxy
before this and 0 of 15 after. The existing unit test asserted only that
upsert had been called on a mock, so it passed either way; it now pins the
shape of both branches and fails when the update branch goes back to empty.
* test: point the live codex tests at gpt-5.3-codex
OpenAI deprecated gpt-5.2-codex, so test_openai_codex and
test_openai_codex_stream started failing against the live API with
model_not_found. gpt-5.3-codex is the current codex model; both tests pass
on it. The remaining gpt-5.2-codex references in the suite are mocked
transformation tests and are unaffected.
* test(e2e): update models page specs for the shared DataTable
The DataTable migration in #34363 changed three things the models page
specs were pinned to, and five tests went red.
Row click no longer opens the detail view; the Model ID cell owns that
now, so both specs click its `model-id-<id>` test id instead of the row.
The search box placeholder switched from an ASCII "..." to a real
ellipsis, so the specs use getByPlaceholder with a substring instead of
an exact attribute match that punctuation can break again. The results
count moved from `models-results-count` ("Showing 1 - 50 of 137 results")
to the shared pagination's `pagination-range` ("Showing 1-50 of 137").
The Team-BYOK test also filtered rows on the team alias, which the Team
ID column has never rendered in either the old or the new table; it
filters on the team id now, which is what the column actually shows and
what the assertion's own comment intends.
Verified against a local proxy serving a fresh build with the seeded
e2e postgres and mock upstream: all five failing tests pass, and the
full suite is 82 passed / 4 skipped at CI parity (workers=1).
* fix(scim): use members_with_roles as the source of truth for group membership
SCIM group provisioning tracked membership inconsistently. Team creation and
the real team endpoints persist membership in members_with_roles (and each
member's user.teams), but the SCIM group PATCH handler and the GET /Groups
listing read the legacy team.members String[] column, which team creation never
populates. Seeding a PATCH result from that empty column made an Okta "add
member" operation recompute the member set from scratch and silently drop
everyone already in the team, so users ended up missing from the groups they
were provisioned into. Reading the same empty column on GET /Groups reported an
empty member list back to the IdP, which drove repeated re-provisioning.
Separately, add_new_member appended the team id to user.teams with an
unconditional array push. Under the concurrent group PATCHes an IdP sends during
a reconcile, each request passed the members_with_roles duplicate check and
pushed, so user.teams accumulated duplicate ids for the same team. A duplicate
also breaks auth logic that keys off the number of teams a user belongs to.
Read current membership from members_with_roles in the SCIM group PATCH seed and
the GET /Groups listing, and make the user.teams append idempotent via a
filtered update that no-ops once the team is present.
Resolves LIT-4283
* fix(scim): address review; atomic user-creation and stop writing legacy members
Keep the concurrent-safe team append but create the user via an atomic upsert
(create-or-update) instead of a check-then-create, so provisioning the same new
user concurrently cannot race into a duplicate-key failure; the team is still
appended idempotently by a filtered update so an existing user gets no duplicate
team id. Stop writing the legacy team.members column in the group PATCH apply so
the only membership record is the source of truth (members_with_roles plus each
member's user.teams), reconciled by team_member_add/team_member_delete.
Tests: existing add_new_member and team-creation mocks updated to the upsert
plus filtered-append shape, and new tests cover atomic creation and that the
PATCH apply does not write the legacy members column.
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server
Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain
access to every MCP server on the proxy without listing each id. The
sentinel expands to the live registry at request time, so a server added
later is picked up with no change to the team's stored permission. The team
ceiling that validates a key's MCP scope expands the sentinel too, so a key
can be scoped to any server (including one registered after the team) and
still pass subset validation
Expose the option in the team create and edit forms via a new exclusive
"All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing
"No MCP Servers" sentinel
* Update litellm/proxy/management_helpers/object_permission_utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key
The sentinel was expanded inside the shared expand_permission_list, which
also feeds the key, org, end_user and agent resolvers. A key whose stored
object_permission ever held all-proxy-mcpservers (a stale write, a
configured default, or a bug) would silently resolve to every MCP server at
runtime, and a teamless key had nothing to cap it, so all servers got
injected. Only write-time validation stripping the value stood between that
value and a full grant
Move the expansion out of expand_permission_list and into
_get_allowed_mcp_servers_for_team so the sentinel is honored only where it is
settable (a team). Anywhere else it now passes through as an inert literal
that matches no registered server and is denied downstream. Reserved-id
protection already blocks a real server from taking that id
* fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel
Granting a team every MCP server on the proxy is a proxy-wide authorization
decision, but team create/update let any caller who can manage a team set
object_permission.mcp_servers, with no ceiling check. Org admins reach
/team/update by default (org_admin_allowed_routes) and _verify_team_access
also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers
and self-grant their team access to every MCP server on the proxy, including
servers never assigned to that team
Gate the grant in new_team and update_team: a non-proxy-admin cannot add the
all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a
team a proxy admin already scoped to all-proxy can still be edited by a team
admin without being forced to strip the sentinel. The UI only offers the
"All Proxy MCP Servers" option to proxy admins in the team create and edit
forms
* fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team
A team scoped to the all-proxy-mcpservers sentinel could be opened in the team
edit form by a team admin or org admin (canEditTeam admits them), but the
"All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the
proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored
sentinel was hydrated into the selected value with no matching Select.Option, so
antd showed the raw all-proxy-mcpservers literal as a chip, and adding another
server could persist a mixed [all-proxy-mcpservers, <id>] value.
Render the option whenever the sentinel is present in the value, not only when
the caller may grant it, and drive the real-option disabling off presence too so
the selection stays exclusive. A non-proxy-admin now sees the friendly label
read-only and cannot build a mixed state; only a proxy admin can newly add it,
which the backend already enforces.
Adds regression tests: the selector shows the friendly option (not the raw
literal) when the sentinel is stored but the grant flag is off, plus exclusive
emit and disabled-real-options coverage, and MCPServerPermissions renders the
green "All" state instead of the raw sentinel string.
* fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip
antd renders a Select option's children inside the selected tag, so the
all-proxy option showed both "All Proxy MCP Servers" and the green "All servers"
type-hint in the chip, which say the same thing. Collapse the option to a single
green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly
without the duplication.
* fix(ui): color the all-proxy MCP label blue to match server chips
Use the same blue (#1890ff) as regular MCP server entries for the
"All Proxy MCP Servers" option/chip instead of green.
* fix(ui): make the all-proxy MCP permissions display blue, not green
Match the blue used by the selector chip and regular server entries so the
"All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent
across the team/key/org detail views. The red "Blocked" state for
no-mcp-servers is unchanged.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* feat(mcp): add tool search virtual tools for large catalogs
When mcp_tool_search_enabled is set on a key's object_permission,
tools/list returns only mcp_tool_search and mcp_tool_call instead of
the full catalog. The LLM searches by keyword then calls discovered
tools by name, avoiding context bloat with 100+ tool deployments.
* fix(mcp): persist mcp_tool_search_enabled and route tool_call by name
The mcp_tool_search_enabled flag existed on the Pydantic models but the
Prisma schema lacked the column, so keys generated with the flag never
persisted it and tools/list kept returning the full catalog. Add the
column across all three schema.prisma copies plus a migration.
handle_mcp_tool_call passed server_name="" into call_tool, which built a
malformed prefixed name ("-<tool>") and failed to resolve the server.
Resolve the caller's allowed servers and dispatch through execute_mcp_tool
instead, matching how the normal /tools/call path routes.
* fix(mcp): filter list_tools to virtual tools on the protocol path
The REST surface (/mcp-rest/tools/list) returned only the two virtual
tools when mcp_tool_search_enabled was set, but the MCP protocol handler
(handle_list_tools, used by real MCP clients over streamable-http/SSE)
still returned the full catalog. Apply the same early return there so an
actual MCP client sees mcp_tool_search and mcp_tool_call instead of every
tool. call_tool was already intercepted on this path.
* fix(mcp): enforce IP + server filtering on virtual tool search/call
Review flagged that the virtual mcp_tool_search/mcp_tool_call path skipped
access controls the normal MCP flow applies. mcp_tool_call resolved allowed
servers from key permissions only, never applying IP filtering, so a caller
on a public IP could invoke a tool on a server marked
available_on_public_internet: false. mcp_tool_search listed the raw catalog
via global_mcp_server_manager.list_tools, exposing tool names/schemas that
/tools/list would hide and ignoring per-key/per-server tool filters.
Route both virtual handlers through the same filtered paths used by the
normal MCP flow: search now calls _list_mcp_tools and call resolves servers
via _get_allowed_mcp_servers, both threaded with the request client IP so
filter_server_ids_by_ip applies. execute_mcp_tool then enforces the server
allowlist and per-key tool permissions. Thread client_ip through
_list_mcp_tools/_get_tools_from_mcp_servers and pass it from the REST and
SSE call sites.
* fix(ci): ruff format server.py and sync dashboard API types
ruff format normalizes the list_tools client_ip changes in server.py, and
schema.d.ts gains the mcp_tool_search_enabled object-permission field so the
generated dashboard types match the proxy OpenAPI spec.
* style(mcp): drop quoted annotations and sort imports
Clears UP037 on the virtual tool handler signatures (redundant with
from __future__ import annotations) and I001 on the list_tools import block.
* refactor(mcp): extract virtual-tool dispatch and host progress capture
Pulls the mcp_tool_search/mcp_tool_call interception and the host
progress-callback setup out of mcp_server_tool_call into helpers, keeping
that handler under the strict cyclomatic-complexity ceiling after the
client_ip threading. No behavior change.
* test(mcp): cover SSE virtual-tool dispatch and host progress helpers
Adds unit tests for _dispatch_virtual_mcp_tool (non-virtual passthrough,
flag-disabled rejection, search/call routing with client_ip),
_capture_host_progress_callback, and the protocol list_tools virtual
early-return, covering the new server.py paths.
* fix(mcp): forward per-request auth headers through virtual tool handlers
The virtual mcp_tool_search/mcp_tool_call path intercepted the request
before the normal header extraction ran, so client-supplied per-request
auth (Authorization for upstream pass-through, x-mcp-auth-<alias>) was
dropped and execute_mcp_tool/_list_mcp_tools received None. Thread
mcp_auth_header, mcp_server_auth_headers, oauth2_headers, and raw_headers
from both the REST and SSE call sites through the handlers so upstream MCP
servers that require pass-through auth can be listed and called.
* fix(mcp): preserve requested server scope in virtual tool calls
A scoped MCP session (/mcp/<server>/ or header-scoped) carries an
mcp_servers scope that the normal call path passes into routing so the
session can only reach that server. The virtual-tool branch dropped it and
resolved with mcp_servers=None, letting a scoped session call mcp_tool_call
for any server the key can access. Thread the context mcp_servers scope
through _dispatch_virtual_mcp_tool into both handlers so search and call
resolve against the same scoped server set.
* fix(mcp): convert virtual tool errors to isError on the protocol path
The virtual-tool dispatch ran before the protocol handler's HTTPException
and guardrail handling, so a rejected virtual call (e.g. an out-of-scope
403 from execute_mcp_tool) raised out of mcp_server_tool_call and broke the
MCP JSON-RPC stream instead of returning an isError CallToolResult. Move
the dispatch inside the same try that wraps call_mcp_tool so virtual-tool
errors get the same isError conversion as normal tool calls.
* fix(mcp): spend-log virtual tool calls on the REST path
The REST virtual-tool branch returned before common_processing_pre_call_logic,
so execute_mcp_tool ran without a litellm_logging_obj and virtual mcp_tool_call
invocations were not spend-logged or guardrail-checked like normal calls. Run
the same pre-call pipeline in the call branch and thread the resulting
litellm_logging_obj through handle_mcp_tool_call into execute_mcp_tool.
* fix(mcp): reject virtual tool call when key has no accessible servers
handle_mcp_tool_call passed an empty allowed_mcp_servers list into
execute_mcp_tool; an unprefixed local tool name then fell through to the
local registry, which has no server permission check, so a key with only
mcp_tool_search_enabled and no server grants could run operator-configured
local tools by name. Reject with 403 before dispatch when no servers are
accessible, matching call_mcp_tool.
* docs(mcp): document virtual tool_search module and parity rule in AGENTS.md
* style(mcp): apply ruff format at repo line-length (120)
* fix(mcp): add mcp_tool_search_enabled to ObjectPermissionDict and customer test fixture
* chore: trigger CI
* fix(mcp): mirror pre-call pipeline, guard imports, coerce top_k, honor include_disabled_tools
- SSE mcp_tool_call now runs common_processing_pre_call_logic so it spend-logs and runs guardrails like the REST path (P1)
- coerce_top_k avoids ValueError on non-integer top_k from clients (both REST and SSE)
- guard mcp.types import in tool_search behind runtime/TYPE_CHECKING per package convention
- admin list with include_disabled_tools returns the real catalog even when mcp_tool_search_enabled is set
Replace bare Optional[dict] on the object_permission validator surfaces with
a typed TypedDict mirror of LiteLLM_ObjectPermissionBase. The TypedDict shape
matches the Pydantic model field-for-field and supports .get() and item
assignment, so the mutation in _rewrite_object_permission_mcp_identifiers
continues to work at runtime (TypedDict is a plain dict).
Propagated through the surfaces this PR touches: _object_permission_to_dict,
_validate_mcp_servers_for_key_update, validate_key_mcp_servers_against_team,
validate_key_search_tools_against_team, validate_key_vector_stores_against
_team, the five _extract_requested_* helpers, and the two
_rewrite_object_permission_mcp_* mutators. attach_object_permission_to_dict,
handle_update_object_permission_common, and _set_object_permission keep
their wider dict typing because they handle the full key/team data_json,
which is a superset of ObjectPermissionDict and pre-dates this PR.
No behavior change. 373 tests pass; ruff strict + type discipline gates
green.
Non-admin callers could create or update a personal key (no team_id)
with arbitrary access_group_ids, mcp_toolsets, vector_stores, or
search_tools in object_permission. The server persisted the values
without ownership validation; runtime authorization then trusted the
IDs because they were stored on the key, allowing cross-tenant access
to other teams' restricted models, MCP toolsets, and vector stores.
The personal-key gate now mirrors the team-key path.
enforce_member_can_assign_access_groups raises 403 for non-admin
teamless callers. validate_key_mcp_servers_against_team rejects
non-empty mcp_toolsets on personal non-admin keys. A new
validate_key_vector_stores_against_team enforces the same rule for
vector_stores. validate_key_search_tools_against_team gains the same
gate for search_tools. The four validators are wired into
/key/generate, /key/update, and /key/regenerate. Proxy admins keep
their existing carve-out across all fields; team keys are unaffected.
Endpoint-level regression coverage lives in
tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py
(six new parametrised cases through generate_key_fn and
_validate_update_key_data) and helper-level coverage in
tests/test_litellm/proxy/management_helpers/. Deleting any of the
validator calls in _common_key_generation_helper or unmoving the
enforce gate in _validate_update_key_data breaks the suite.
/team/member_add could not set budget_duration on an individual member
budget. add_new_member created the budget row with only max_budget and
allowed_models, and TeamMemberAddRequest had no budget_duration field, so
a member added with an explicit per-member budget while the team ran a
recurring member budget got a lifetime cap instead of a recurring
allowance.
Thread budget_duration from TeamMemberAddRequest through
_process_team_members into add_new_member, and pull the member-budget
resolution into a helper that writes budget_duration plus a computed
budget_reset_at. When only a budget_duration is supplied and the team has
a default member budget, the default is cloned and its reset window
overridden so the member keeps the default's max_budget rather than
becoming uncapped; a duration with no team default creates a window-only
budget. Invalid durations are rejected with a 400 before any DB write,
symmetric with /team/member_update.
The available-team self-join bypass only grants the ability to join, so
reject per-member budget and model controls (max_budget_in_team,
budget_duration, allowed_models) for non-admin self-join callers in
_validate_team_member_add_permissions, before any DB write. Otherwise a
self-joining non-admin could set their own cap, reset window, or model
scope past the team default; admins, team admins, and org admins are
unaffected and a clean self-join still inherits the team default budget.
Resolves LIT-4052
Creating or updating a key with a specific (non-allow_all_keys) MCP
server or access group failed with a 403 when the key had no team:
Key is not in a team. Only globally available (allow_all_keys) MCP
servers can be assigned
validate_key_mcp_servers_against_team computed the allowed set as
team servers + allow_all_keys servers. For a teamless key the team
set is empty, so the allowed set collapsed to just allow_all_keys
servers and any explicitly-picked server or access group was rejected.
This was asymmetric with runtime: get_allowed_mcp_servers honors a
teamless key's own object_permission.mcp_servers verbatim, with no
team gate and no allow_all_keys filter. So the create/update path
refused to persist a grant the run path would have served.
Thread is_proxy_admin into the validator from both call sites
(/key/generate and /key/update). When a key has no team and the
caller is a proxy admin, the requested servers and access groups are
folded into the allowed set so the existing subset checks pass. A
proxy admin can already reach every MCP server, so there is nothing
to escalate. Non-admins and every team-scoped key are unchanged.
Resolves LIT-3815
* feat(mcp): scope a key to zero MCP servers with no-mcp-servers sentinel
A key under a team that has MCP servers had no way to opt out of them;
an empty list has always meant "inherit the team". This adds a
no-mcp-servers sentinel (mirroring no-default-models for models) so a key
can declare an explicit zero that overrides team inheritance, additive
grants, and allow_all_keys servers, surfaced as an exclusive "No MCP
Servers" option in the key create/edit UI.
* refactor(ui): centralize no-mcp-servers sentinel in a shared constant
The sentinel string was defined under two different local names and
inlined in two more files; a single exported constant removes the drift
risk flagged in review.
* fix(mcp): enforce no-mcp-servers sentinel on toolset-scoped routes
Toolset scoping replaced a key's mcp_servers with the toolset's servers,
dropping the no-mcp-servers sentinel, so a key opted out of all MCP could
still execute a granted toolset's tools via /toolset/{name}/mcp. Deny
toolset access when the key carries the sentinel, checked before the admin
branch to match get_allowed_mcp_servers.
* fix(mcp): make key.access_group_ids grants additive over team ceiling
A key whose unified access_group_ids grant a private MCP server was
having that grant intersected against its team's MCP ceiling, so a key
in a team scoped to other servers (or with no own scope) lost the
granted server entirely. Resolve access_group_ids once as ungated
additive grants and union them on top of the key/team ceiling instead
of folding them into the key scope that gets intersected.
* test(mcp): align key access-group tests with additive-grant model
The previous commit moved key.access_group_ids resolution out of the
intersected key ceiling (_get_allowed_mcp_servers_for_key) and into the
ungated additive grant path (_get_key_access_group_mcp_server_extras),
unioned on top of the team ceiling. Five tests from #28890/#29195 still
asserted the old gated / in-key-scope contract and failed:
- _get_allowed_mcp_servers_for_key now returns the object_permission
ceiling only and never resolves access_group_ids; two tests now assert
the group resolver is not called from that path (with and without an
object_permission present).
- The extras path is ungated, so a group whose assigned_team_ids /
assigned_key_ids exclude the caller still contributes its servers.
- The end-to-end test asserts the grant surfaces via the extras path
rather than the base key path.
- Dropped test_key_access_group_ids_empty_returns_no_extras; the empty
case is already covered by the extras family's no-groups test.
* feat(auth): gate member access-group assignment on keys behind opt-in
Non-admin team members could attach access_group_ids to keys they create
or update, letting them self-grant resources (MCP servers/models) the team
admin never intended. Add an opt-in KEY_ACCESS_GROUP_ASSIGNMENT team-member
permission (default-deny) enforced at /key/generate and /key/update; proxy
and team admins bypass. Surfaces automatically as a checkbox in the team
Member Permissions UI.
* fix(auth): gate access-group assignment on /key/regenerate too
RegenerateKeyRequest inherits access_group_ids and prepare_key_update_data
persists it, so a non-admin key owner could self-grant access groups by
regenerating. Apply the same opt-in member gate using the existing key's
team.
* test(auth): cover member access-group gate and additive MCP grants
Add unit tests for enforce_member_can_assign_access_groups (deny without
opt-in, allow with opt-in, and proxy-admin / team-admin / non-team-key
bypasses) and for _get_key_access_group_mcp_server_extras (no-auth and
no-resolved-servers return empty, resolved ids are expanded, errors
degrade to no grants).
* fix(mcp): ignore stale server ids during key permission validation
Prevent virtual key save failures when object_permission still includes MCP server IDs that were deleted from the registry. The validator now drops stale IDs before team scope checks and adds coverage for stale-ID and active unauthorized-ID behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): use DB find_many to detect stale server IDs, add tool-permission test
Addresses greptile feedback on PR #29128:
P1 – Race-condition / authoritative-signal concern:
Extract _get_stale_mcp_server_ids() helper that performs a single
DB find_many (LiteLLM_MCPServerTable) when prisma_client is available,
making server-existence checks authoritative even during in-memory
registry warm-up. Falls back to the in-memory registry only when no
DB client is present (config-only deployments / unit tests).
Pass prisma_client through both call sites in key_management_endpoints.
P2 – Missing test coverage for mcp_tool_permissions:
Add test_validate_stale_ids_in_mcp_tool_permissions_silently_dropped
to confirm stale server IDs that appear only as keys in
mcp_tool_permissions are also stripped without raising 403.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): also check in-memory registry when DB is present to avoid misclassifying config-file servers as stale
Config-file MCP servers are never written to LiteLLM_MCPServerTable, so a
DB-only query would classify them as stale and silently drop their IDs from
the authorization check, allowing unauthorized key access.
Fix: treat a server ID as stale only when it is absent from BOTH the DB and
the in-memory registry (which holds config-file servers).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): normalize server aliases on key save
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* fix: remove unused stale MCP helper and capture normalized object_permission on key generation
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* test(mcp): keep validate_key_mcp_servers_against_team stub in sync with return contract
validate_key_mcp_servers_against_team now returns the (normalized) object_permission, and the key-generation helper assigns that return value back into the request data. The two key-generation tests stubbed the function with a bare AsyncMock, whose default MagicMock return value clobbered object_permission and skipped permission-record creation. Make the stubs pass object_permission through unchanged.
* fix(mcp): preserve provided object_permission fields on key update
The key-update path reconstructs data.object_permission from a full
model_dump(), which marks every field as set. Downstream
model_dump(exclude_unset=True) then emits models/blocked_tools/
search_tools as None, and those are non-nullable array columns, so the
Prisma write fails whenever the UI submits object_permission with only a
subset of fields populated (e.g. a TPM/RPM-only edit). Build the dict
with exclude_unset=True so the normalized object_permission keeps the
caller's original field set.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Store search tool allowlists only on object permissions, wire auth/management/UI flows to object_permission.search_tools, and remove legacy team-metadata search credential code and tests.
Made-with: Cursor
Previously, members added to a team without an explicit per-member budget were
all linked to the same `litellm_budgettable` row referenced by the team's
`metadata.team_member_budget_id`. Updating one member's budget via
`/team/member_update` mutated the shared row and silently changed every other
member's budget too.
Now both write paths produce a private, per-member budget:
- `add_new_member` clones the team's default budget into a fresh row when a
member is added without `max_budget_in_team`/`allowed_models`. If no team
default exists, the membership is created with no budget.
- `_upsert_budget_and_membership` detects when an existing membership still
points at the team's default budget id and clones-on-write, relinking the
membership to the new private budget before applying the update.
- `team_member_update` reads `team_member_budget_id` from team metadata and
passes it through so the helper can make this distinction.
Adds unit tests for clone-on-write, in-place update of a private budget, and
the no-default-no-budget add path.
Made-with: Cursor
Block cross-team key update/regenerate operations by raising when the caller is not a member of the target key's team, and add unit coverage for deny/allow team membership paths.
Made-with: Cursor
Rewrite Default Team Settings UI from Tremor to antd with hardcoded fields,
fix default team params not applying during team creation or persisting
across proxy restarts, remove dead code, and add comprehensive tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two fixes based on PR feedback:
1. Move callback dispatch before the prisma_client check so audit logs
still reach S3/Datadog even if the DB is down. Also changed the
prisma_client=None case from raising an exception to logging an error
and returning gracefully.
2. Attach a done_callback to asyncio tasks created for audit log
callbacks so exceptions are logged through verbose_proxy_logger
instead of silently swallowed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Audit logs (CRUD events on keys, teams, users, models) were only stored in
the Prisma DB. This adds a pluggable callback system so audit logs can be
forwarded to external services like S3 for ingestion into security monitoring
tools.
New config key `audit_log_callbacks` under `litellm_settings` reuses the
existing callback infrastructure. Any CustomLogger subclass can opt in by
overriding `async_log_audit_log_event()`. S3Logger (s3_v2) is implemented
as the first handler, storing audit logs under `audit_logs/{date}/` prefix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use fastuuid helper across the codebase
First batch of changes, simple drop in replacement.
* second batch of changes
* fixed: script mistake on helper file
* added mcp tools on internal user and divide it by teams
* add support for server api call
* Added frontend for test key
* added tools used output
* fix ui for servers
* All servers to personal
* change columns format
* revert ui logic
* Added vertical align
* fix mapped tests
* fix lint
* fix lint
* remove extra file
* fix ui test
* comments fixes
* change query type
* change query type
* mcp acces group init
* add ability to change server display on ui through access groups
* Mcp access group names UI (#12486)
* Added ui changes to reflect mcp_access_groups
* fix edit mcp page
* change to string array (#12491)
* change to string array
* Remove print
* add ability to change server display on ui through access groups
* Litellm mcp access groups accesses (#12498)
* added mcp access groups for keys and teams
* added access groups above servers
* fixed ruff
* fixed mypy
* revert couple changes
* fix test
* fixed double asterisks
* Litellm mcp groups UI (#12522)
* add ui for teams
* fix object permissions
* fix mcp servers test object permission
* remove print
* add helper method
* add tests + remove logs
commit 440bc02725
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 23:04:11 2025 -0700
fix: fix check
commit 89a7451cb9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 22:42:30 2025 -0700
fix: fix test
commit 1322e3b349
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:52:40 2025 -0700
style: add more tooltips
commit 172738b98b
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:51:09 2025 -0700
feat(team_member_view.tsx): add a tooltip
commit 895eb28deb
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:46:49 2025 -0700
fix(teams.tsx): support setting team member budget on create
commit 003cc54a6d
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:49 2025 -0700
fix(team_member_view.tsx): style improvements
commit a627a044f2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:01 2025 -0700
fix(team_member_view.tsx): handle scientific notation in string
commit c5a3b7bd84
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:34:25 2025 -0700
feat(team_membership_view.tsx): show team member spend + max budget on UI
commit e986d12ad5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:28:06 2025 -0700
feat(team_member_view.tsx): show team member spend + budget on team info
commit 8e398607b2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:18:16 2025 -0700
feat(team_info.tsx): show team member budget on team info
commit 1f56886b59
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:15:30 2025 -0700
feat(team_endpoints.py): get team budget table on team info
allows user to see max budget set for team members
commit 0a4320bbfa
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:10:06 2025 -0700
feat(team_endpoints.py): return team member budget on team info
allows ui to display this to admin / team member
commit 6a4e29f87b
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:57:20 2025 -0700
fix(team_endpoints.py): support updating team budget on UI
commit 53f0fff340
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:38:17 2025 -0700
feat(proxy/_types.py): return team member spend
update pydantic object to include spend
Allows showing spend of team member within team on UI
commit ef2a1a43ec
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:31:42 2025 -0700
feat(team_endpoints.py): support 'team_member_budget' param on `/team/update`
enables budget working across all team members
commit 512999f124
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:20:04 2025 -0700
test: add unit tests for default team member budget
commit 90fa3f61a2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:37:51 2025 -0700
feat(team_endpoints.py): support using default team member budget id, if set
allows all team members to use the same budget id
commit acef5324b1
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:22:30 2025 -0700
feat(team_endpoints.py): support `team_member_budget` param on `/team/new`
Allow creating 1 budget for all users within team (makes it easier to increase/reduce budget if needed for all team members)
commit 2e867ac70f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 13:45:06 2025 -0700
fix(ui_sso.py): ensure user is added to team, if set via default internal settings
allows users signed up via SSO to be added to default team
