* feat(agents): authoritative permissions
* fix: enforce authoritative managed agent permissions
* fix(agents): only consult the identity store for managed targets
is_agent_allowed entered the identity-store path whenever a prisma client
was configured, so an ordinary agent paired with an internal user returned
503 instead of 200. Classify the target from the registry first and fall
back to the store only when the registry has no entry, so an unmanaged
target never depends on the store being reachable.
* fix(agents): gate the managed path on an admitted policy object
Ten call sites branched on `managed_agent_policy is not None`, which any
MagicMock attribute satisfies, so the managed path fired on unmanaged
subjects and died in Pydantic validation as a 503. Route every check
through a shared helper that requires a real AgentResponse.
* test(mcp): stub the writer replica the fresh-policy reads use
reload_admitted_user now passes check_db_only through to get_user_object,
so the user row is read from writer_db. Point the mocks at the replica the
code actually reads and give each parametrized case its own user id.
* fix(agents): cap a managed agent at the invoking team's agents
resolve_agent_access returned the managed policy's grants before the
agent_caller ceiling was applied, so a managed agent acting on behalf of a
user reached agents that user's team was never granted. Intersect with the
caller ceiling the unmanaged path already honours.
* fix(agents): restore token narrowing and scope the private-access suppressions
The managed-model check lost its valid_token narrowing when it moved to the
shared helper. Make the caller-access resolver public rather than reaching
into it from module scope, and give each remaining private access a reason.
* docs(agents): drop the comment claiming admins skip the A2A permission check
The check has never had an admin bypass on this path, so the comment
described behaviour the code does not implement.
* test(proxy): stub the writer reads and restore the MCP manager singleton
Fresh-policy user lookups read writer_db, so the team and rest-endpoint
mocks stubbed a replica the code no longer reads, and the dashboard
session fake still had the pre-kwarg signature. The manager reload also
rebound global_mcp_server_manager in every MCP module without restoring
it, leaking an empty manager into later files.
* style: sort imports under the litellm package ruff config
* fix(mcp): cap a managed agent's servers and tools at the invoking caller
managed_agent_servers and managed_agent_tools returned the agent's own
grants without the agent_caller ceiling the unmanaged resolvers apply, so
a managed agent reached MCP servers and tools the echoed caller could not.
Call the existing ceiling helpers on both axes.
* refactor(mcp): return the caller-capped tools without an interim list
The ceiling helper already returns a sequence, so materializing it into a
list added a mutable collection for nothing. Sort at the return sites
instead, which also makes the tool order stable across both branches.
* fix(agents): preserve actor ceilings during managed target checks
* fix(agents): keep managed permission ceilings authoritative
* fix(mcp): fail closed on authoritative caller team outages
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* refactor(auth): bind UI/CLI session tokens to their own AES-GCM context
UI and CLI session tokens are now always encrypted with AES-256-GCM and a
fixed session associated-data value, and the session-token check only accepts
AES-GCM values carrying that same value. Stored secrets keep their current
encryption and decrypt unchanged, so nothing needs migrating.
encrypt_value_helper and decrypt_value_helper take an optional aad. XSalsa20
cannot bind associated data, so an AAD-bound value is always written as
AES-256-GCM, and an AAD-bound decrypt refuses the legacy format.
Session tokens issued before the upgrade stop validating, so UI and CLI users
sign in once more after upgrading.
* test(e2e): cover real SSO login through the dashboard and the lite CLI
Adds two specs under tests/e2e/ui/oidc, run by playwright.oidc.config.ts
against a live Keycloak stack. The dashboard spec checks that the SSO
session authorizes the Virtual Keys and Models data requests. The CLI
spec runs a real lite login in an isolated HOME with the keyring
disabled, then lists models and sends one chat completion with the
stored session. The main Playwright config now ignores oidc/.
* fix(auth): encode UI/CLI session tokens as unpadded base64url
Session tokens carried the v2:gcm: storage prefix and base64 padding. Basic-auth parsers split on the first colon and browsers reject ':' and '=' in WebSocket subprotocols, so Langfuse pass-through and the realtime playground could not use them
Tokens are now plain unpadded base64url, the same header-safe shape as any bearer token
* fix(auth): prefix UI/CLI session tokens with litellm_login_
A prefix-less token starts with sk- about once in 262,144 logins and is then routed as a virtual key, so that login gets a 401. The prefix also makes session tokens easy to spot in logs
The prefix doubles as the token's AES-GCM associated data, so the visible kind and the encrypted kind cannot disagree
---------
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Identity objects (key, end user) load through the request MGET and their write-backs, the registry
reads and the management-object SETs ride the request pipeline. A team refresh invalidates its alias
with a pipelined DEL instead of a synchronous DEL plus a duplicate async one, and an MGET miss is
remembered so no per-key GET follows it in the same request.
Resolves LIT-9012
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* feat(cost_calculator): add cost_per_second for chat per-second pricing
Keep legacy input_cost_per_second and output_cost_per_second as aliases for chat, completion, embedding and responses. When both legacy fields are set, input_cost_per_second wins
Move Bedrock commitment rows to cost_per_second so they bill once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost_calculator): drop legacy per-second fields from chat paths
Keep Azure chat token pricing generic and update inert Voxtral rates and SageMaker examples
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost_calculator): recognize output-only per-second rates
Include output_cost_per_second when checking whether a deployment cost entry has pricing so output-only legacy aliases remain attached to the deployment during cost selection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(pricing): cover cost_per_second and legacy per-second aliases through the proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost_calculator): drop output_cost_per_second as a chat per-second alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost_calculator): restore output_cost_per_second as a chat per-second fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost-map): keep input_cost_per_second on bedrock commitment rows for older clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`_is_model_cost_zero()` reads a group's cost through `Router.get_model_group_info()`,
which resolves `model_group_alias`, and then gates that on `_is_cost_explicitly_configured()`,
which scanned `Router.model_list` for an exact `model_name` match. Alias names live only in
`Router.model_group_alias` and are never `model_name` entries, so the scan found nothing and
returned False. That False means "the zero cost was defaulted, not configured" (the sparse
auto-registration gate added for #24770), so a model priced explicitly at 0 had budget
enforced against it when requested through an alias, while the same deployment under its own
name was exempt. Both names route to the same deployment and add nothing to spend.
The two lookups in one function disagreeing is the bug, so they now share one resolution:
`_is_cost_explicitly_configured()` resolves through `Router.get_model_list()`, the same
alias-aware path `get_model_group_info()` takes. That also reaches a deployment which prices
itself through its `model_info` block, whose cost-map entry lands under the deployment id.
`_group_declares_explicit_cost()` was an alias-aware copy of this function, wired only into
`model_has_no_cost_mapping()` and never into the budget path; its body is what
`_is_cost_explicitly_configured()` now carries, and both callers share it so the two cannot
drift apart again.
`_has_ptu_flat_cost()` scanned `model_list` the same way and runs after the gate above, so
resolving one without the other would let an aliased PTU group — explicit zero per-token
price alongside a flat capacity cost — pass as free. It resolves the same way now.
Tests cover the predicate and the request path it feeds: over-budget requests through
`_should_skip_budget_checks()` into `common_checks()` for an aliased free model (allowed) and
an aliased paid model (refused), the predicate for free, paid, PTU, hidden and dangling
aliases, and `model_has_no_cost_mapping()` through an alias so the other caller of the shared
check stays covered.
Unchanged: priced groups (the predicate returns False before the gate), unmapped groups whose
zero cost was defaulted (#24770), hidden aliases and aliases pointing at a nonexistent group
(`get_model_group_info()` returns None for both, so the cost is unknown and budget is
enforced), and non-aliased PTU groups.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(proxy): email alerts at configured percentages of a team member budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(alerting): label team member budget crossings as team member budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(auth): cover the team member alert dispatch from _check_team_member_budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(email): drop the emoji from the team member budget alert template
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): ignore team member alert thresholds outside 1 to 100 on both the backend and the dashboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): bound team member alert threshold key length before int parsing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the legacy covers marker from the team member alert test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): reject malformed team_member_max_budget_alert_emails on team writes
Thresholds outside 1-100, non-list recipients, and invalid emails now return 422 on
/team/new, /team/update and PATCH /team/{id} instead of being stored and silently
ignored. The value is stored canonically. Read-side LiteLLM_TeamTable is unchanged,
and the PATCH body stays a raw merge patch so a null threshold still deletes it.
* fix(auth): enforce and alert on team member budgets only in common_checks
The builder re-checked the team member budget inline before common_checks ran the same
check, so one request that crossed a team_member_max_budget_alert_emails threshold
dispatched two alerts. Drop the inline check; common_checks is the single authorization
point and already covers per-member rows, the team default member budget, zero-cost
skips and the cross-pod spend counter. Its 422 message now uses the TeamMember=user:team
form the builder and budget reservation already returned.
* Revert "fix(team): reject malformed team_member_max_budget_alert_emails on team writes"
This reverts commit 703e754b46.
* fix(alerting): keep BaseBudgetAlertType.get_event_message zero-arg
Requiring user_info broke existing callers and out-of-tree subclasses. The team member
label now comes from SlackAlerting.budget_alerts, so the interface and its Readme are
unchanged from main.
* fix(mcp): keep team member budget enforcement on the MCP OAuth auth dependency
The MCP OAuth dependency stops at _user_api_key_auth_builder and never reaches common_checks, so removing the builder's inline member budget check would have let over-budget members through there. Enforce it explicitly for that caller.
* fix(auth): keep main's team member budget enforcement, alert once per request
Restore the builder's team member budget check and 422 message exactly as on main and drop the MCP-only gate. The builder sends the member alert only on the request it rejects; common_checks sends it for requests that get past the builder, so no request alerts twice.
* test(integration): read team member alert deliveries without a shared accumulator
* test(integration): match team member alert deliveries by subject so other alerts cannot race the count
* refactor(proxy): build the team member alert threshold config without mutable collections
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): collapse the alert recipient isinstance checks into one call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read the SMTP sink through lock-guarded snapshots and assert the exact deliveries
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall
Under a load burst with a slow authentication database every request parked inside
the pod with no deadline while /health/readiness kept answering 200 (its own ping
gets a fresh connection), so the load balancer kept sending traffic until the pod hit
its memory limit, and the parked requests completed against the provider minutes
after every client had hung up
Every pre-request read (key, team, user, end user, budget, membership, organization,
object permission, jwt mapping, project, proxy budget, spend counter reseed) now runs
under one deadline, PROXY_DB_LOOKUP_DEADLINE_SECONDS (default 10 s). A lookup that
hits it fails the request with the existing 503 "authentication database is
temporarily unreachable" answer, honours allow_requests_on_db_unavailable, and never
triggers the transport reconnect (the transport is fine, the query is slow), which is
what turned the repro's stall into "too many clients". Writes stay unbounded
A deadline hit marks the pod stalled for PROXY_DB_LOOKUP_STALL_WINDOW_SECONDS
(default 30 s, 0 disables), during which /health/readiness answers 503 with
"db": "stalled" behind the same fail-open gate, so the pod leaves rotation before it
fills its memory. The existing litellm_in_flight_requests gauge already exposes the
parked set on /metrics
The deadline is enforced on the wall clock: bounded_db_lookup waits on the lookup
task with asyncio.wait and raises DBLookupDeadlineExceeded when the deadline passes
even if the lookup absorbs its cancellation, where asyncio.wait_for on 3.12+ would
sit on the cancelled task for as long as it takes
The failure spend-log row no longer re-runs the key and team lookups when the
failure itself is a database connection or deadline error, so a request that hit
the deadline is answered after one deadline instead of two
* fix(proxy): bound the spend counter gate wait and narrow the stalled lookup shortcut
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the global spend lookup on the prisma client handle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: add dashboard and core smoke checks across supported Python versions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: tighten merge smoke harness and keep mapped test diffs additive
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: terminate proxy on readiness timeout and use contextlib.suppress in teardown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(cost-map): remove models past their deprecation date
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost-calc): drop the empty parametrize left behind by the gemini web search removal
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost-map): drop merge base block left by conflict resolution
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost-calc): drop gemini image cost tests pinned on removed model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_user_object wrapped every failed read, a refused connection included, in
ValueError("User doesn't exist in db ..."), so JWT callers got a 401 naming a
missing user while Postgres was down and virtual-key callers got 503
no_db_connection for the same outage. A connection or transport error now
propagates as-is and the auth exception mapper answers 503 no_db_connection;
a genuinely missing row and query-level errors still answer 401.
The MCP auth and token-exchange docstrings and the exception-chain helper's
docstring described the old wrap and are updated to the new contract.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
An agent key that echoes the x-litellm-user-id / x-litellm-team-id headers
forwarded by /a2a is capped at that user's and team's models, MCP servers
and agents, on top of its own grants and access group ceiling. The echoed
ids only narrow, and nested A2A hops forward the original human caller
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The last-known org copy was written only on get_org_object's DB-read path. The
virtual-key auth prefetch fills the same 5s org entry directly, so with keys and
JWTs of one org on the same worker the JWT lookup always hit the cache, never
wrote the copy, and a DB outage turned that JWT traffic into 503s again.
get_org_object_for_request now writes the copy itself whenever this worker holds
none, under the management-object TTL, and get_org_object is back to its shape
on main.
A JWT whose team sits in an org resolves the org on every request, and the
org row is cached for only DEFAULT_IN_MEMORY_TTL seconds while the team and
user rows ride the 60s management-object TTL. A few seconds into a database
outage the org lookup failed closed and that traffic got 503s while the same
request through a virtual key kept succeeding on its cached team.
get_org_object now also keeps a last-known copy of the org row under the
management-object TTL, and get_org_object_for_request serves that copy when
the database is unreachable, so JWT traffic degrades the same way the team
lookup does. A missing copy keeps the previous behaviour: fail closed unless
allow_requests_on_db_unavailable is set.
A project max_budget of 0 was treated as unbudgeted by #41354, while key budgets block at 0 and null is the unlimited value. Drop the <= 0 skip so 0 blocks and null stays unlimited
The lint job failed on one new typing.cast (LIT006) in the cost callback, and the test-quality gate behind it would have failed next on a test whose only assertion inspected a mock (TQ002). project_id is now narrowed with isinstance, and the zero and negative max_budget cases run through the existing parametrized budget test, which asserts the raised error or a clean admit with no alert
get_user_object skipped the database for db_cache_expiry seconds after a miss on the same worker even when the caller asked for check_db_only, so the token exchange mint could answer no_active_key for a user JWT auth had just created. A database-only read now always reaches the database.
A temporary-only member update no longer clones the team default budget into the private row. The row stores just the temp pair and auth, spend admission and reservation add the active increase to the current shared default, so a later lowering of the default reaches members with an active grant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A service-account key can now carry end_user_budget_id in its metadata. When a request through that key names a customer that does not exist yet, the key's budget is applied to the new customer from the first request and wins over the proxy-wide max_end_user_budget_id. A customer with an explicitly assigned budget keeps it. The Admin UI exposes the setting on service-account key creation and key edit, and only proxy admins may set or clear it.
Resolves LIT-7996
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A registered gateway DCR client can now POST /token with
grant_type=urn:ietf:params:oauth:grant-type:token-exchange and an IdP
JWT as subject_token. The gateway proves the JWT the way its JWT auth
does, resolves the user and team, and answers with the proxy-API
credential and a refresh token, so a fresh laptop with only an IdP login
gets a gateway key without a browser round trip.
"/token" joins mcp_inference_routes so the default JWT team allowlist
reaches the exchange, and the JWT auth builder accepts any header
mapping so the request headers pass through unchanged.
Project-scoped keys never wrote spend to LiteLLM_ProjectTable, so
/project/info stayed at 0 and project budgets could not block. Wire the
PROJECT entity through the spend queue, redis buffer, and db writer,
reserve and increment a spend:project counter, reseed it from the
project row, reset project spend in the budget cascade, and read the
live counter in the project max budget check. Team member budgets keep
gating project-scoped keys alongside the project budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Drop the model_access_denied_message setting, its {model} template, the DB
override entry and the Admin UI field. Model access denials now always return
the fixed client message while the allowlist diagnostic is logged at the final
HTTP, realtime and MCP boundaries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Move the WARNING that carries the internal denial detail out of the message formatter and into the auth exception handler. The denial exceptions now carry internal_message so access-group probes and fallback paths that catch and recover from the denial no longer log a false denial for an allowed request
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add litellm_settings.model_access_denied_message, a template ({model} placeholder) returned to clients instead of the detailed "can only access models=[...]" text on key/team/user/org/project and team-member model access denials. The full denial reason is still written to the proxy logs at WARNING. Unset keeps the existing detailed message, status codes and error types are unchanged.
Expose the new setting and the existing expose_router_debug_in_errors flag in the Admin UI general settings (String editor, Boolean toggle with an explicit True default) and allow both as safe DB overrides so they persist and propagate across workers.
Resolves LIT-5283
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
