The previous per-provider guards in HuggingFace, Oobabooga, and Gemini
files lived in the SDK layer, breaking SDK callers who legitimately pass
URL-valued model identifiers. Move the check to the proxy boundary in
add_litellm_data_to_request so SDK users keep working while proxy users
default-deny URL-valued model and file_id, with admin opt-in via
litellm.provider_url_destination_allowed_hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflict resolution for #26968 dropped the `Iterator` typing import
(NameError at module load), left a dead `fallback_models = cast(...)`
block, and the new tests called `_enforce_key_and_fallback_model_access`
without the now-required `request` kwarg.
Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the
per-row Metadata pane in the UI no longer shows the stack trace when the
opt-in env var is set, matching the existing console-side suppression.
https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6
Adds LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS env var. When set to true and the
proxy log level is INFO or above, spend-tracking error paths emit a single
ERROR line without the full traceback. Stack traces are preserved at DEBUG
and the Sentry / proxy_logging_obj.failure_handler path is unchanged.
The new spend_log_error helper is wired through the spend write hot path:
- DBSpendUpdateWriter (update_database, _update_*_db, batch upsert,
redis-commit fallbacks)
- _ProxyDBLogger._PROXY_track_cost_callback
- get_logging_payload exception path
- update_spend / update_daily_tag_spend / spend logs queue monitor
Resolves LIT-2704.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(proxy): re-validate user_id ownership after /user/info re-parses query
The route-level access check in `RouteChecks.non_proxy_admin_allowed_routes_check`
reads `request.query_params.get("user_id")`, which decodes literal `+` to
spaces. The endpoint then re-parses the raw query string with `urllib.unquote`
in `get_user_id_from_request` to preserve `+` characters (so plus-addressed
emails work as user_ids). Those two paths produce different ids: a caller
who registered a user_id containing a literal space could pass the route
check and then read another user's row by sending the encoded `+` form.
Add `_enforce_user_info_access` and call it after `_normalize_user_info_user_id`
returns the final id. Proxy admin / view-only admin still bypass; everyone
else must match the resolved user_id (or have no user_id, which falls back
to the caller's own id later in the handler).
Tests cover the admin bypass, owner-match path, and the cross-user lookup
that this change blocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(proxy): apply user_info ownership check to PROXY_ADMIN_VIEW_ONLY
`_enforce_user_info_access` was bypassing both PROXY_ADMIN and
PROXY_ADMIN_VIEW_ONLY, but the upstream route check in
`RouteChecks.non_proxy_admin_allowed_routes_check` only treats
PROXY_ADMIN as a true admin for the `/user/info` route — view-only
admins go through the `user_id == valid_token.user_id` enforcement
along with regular users. Mirroring that asymmetry left the same
encoded-`+` bypass open for view-only admins whose user_id contains a
literal space.
Drop the PROXY_ADMIN_VIEW_ONLY exemption so the post-decode re-check
matches the upstream rule. Update tests: a view-only admin must now
be blocked from cross-user lookups but still allowed to read their
own row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count non-chat tokens and validate every model in batch file
Two security control bypasses on POST /v1/batches:
1. `_get_batch_job_input_file_usage` only summed tokens for
`body.messages` (chat completions). Embedding (`input`) and text
completion (`prompt`) batches reported zero, letting massive
non-chat workloads slip past TPM rate limits. Extend the counter
to handle string and list shapes for both fields.
2. The batch input file was forwarded to the upstream provider
without inspecting the models named inside the JSONL — only the
outer `model` query parameter was checked against the caller's
allowlist. A caller restricted to gpt-3.5 could submit a batch
targeting gpt-4o and the upstream would execute it under the
proxy's shared API key.
Add `_get_models_from_batch_input_file_content` (returns the
distinct `body.model` values) and call it from
`_enforce_batch_file_model_access` in the pre-call hook, which runs
each model through `can_key_call_model` so the same allowlist
semantics (wildcards, access groups, all-proxy-models, team aliases)
the proxy enforces on `/chat/completions` apply here too. Any
unauthorized model raises a 403 before the file is forwarded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count pre-tokenized prompt/input shapes, classify 403 logs
Two follow-ups from the Greptile review on the batch validation PR:
1. P1 TPM bypass via integer token arrays. The OpenAI batch schema
accepts ``prompt`` and ``input`` as ``list[int]`` (a single
pre-tokenized prompt) or ``list[list[int]]`` (multiple) in addition
to the string and ``list[str]`` shapes. Pre-fix only the string
shapes were counted, so a caller could submit a batch with hundreds
of millions of pre-tokenized tokens and the rate limiter would
record zero. Extract the per-field logic into
``_count_prompt_or_input_tokens`` and count each int as one token.
2. P2 access-denial logs were indistinguishable from I/O failures.
``count_input_file_usage`` caught every exception under a generic
"Error counting input file usage" message, so an intentional 403
from ``_enforce_batch_file_model_access`` looked the same in the
logs as a missing file or a Prisma timeout. Catch ``HTTPException``
separately and log 403s at WARNING level with a security-relevant
message before re-raising.
Tests cover the new shapes: single ``list[int]``, ``list[list[int]]``
(the worst-case bypass vector), and embeddings ``input`` with
pre-tokenized arrays.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reset the policy ID index during policy engine test cleanup so stale policy versions cannot leak between tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
Mark runtime-created policies and attachments initialized so global policy attachments created from the policy builder apply immediately without requiring a restart.
Co-authored-by: Cursor <cursoragent@cursor.com>
Many MCP integrations (Zapier, etc.) embed an upstream API key
directly in the server URL, e.g.
``https://actions.zapier.com/mcp/<api-key>/sse``. The list and
single-server endpoints were returning the full URL to any
authenticated user — `_redact_mcp_credentials` only stripped the
explicit ``credentials`` field, and `_sanitize_mcp_server_for_virtual_key`
only ran for restricted virtual keys. Non-admin internal users could
read the dashboard, click the unmask toggle, and exfiltrate the raw
token.
Add `_sanitize_mcp_server_for_non_admin` that runs on top of the
existing credential redaction and clears the credential-bearing
fields:
- ``url`` (the primary leak vector)
- ``spec_path`` (OpenAPI spec URLs that may carry tokens)
- ``static_headers`` / ``extra_headers`` (Authorization)
- ``env`` (arbitrary secrets)
- ``authorization_url`` / ``token_url`` / ``registration_url``
Identity fields (``server_id``, ``alias``, ``mcp_info``, etc.) are
preserved so the UI can still list servers a non-admin's team has
access to.
Apply the new sanitizer in `fetch_all_mcp_servers` and the per-server
fetch path right after the existing virtual-key branch. Update the
existing `test_list_mcp_servers_non_admin_user_filtered` assertions
that previously checked URL visibility.
Frontend defense-in-depth: hide the URL unmask toggle on
`mcp_server_view.tsx` unless the viewer is a proxy admin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a team grants /key/list via team_member_permissions, non-admin members
should see all keys for that team — same as a team admin. Previously the
classification in list_keys() only checked admin status, so permitted
members fell into the service-account-only path and could not see other
members' personal keys. Routes those members into the full-visibility set.
Greptile P1: the unsafe-method branch of `_check_proxy_admin_viewer_access`
ended with a blanket `if route in management_routes: return`. That set is a
mix of reads (info/list — handled via the safe-method GET branch above) and
writes. The fallback let Admin Viewer POST to write endpoints not enumerated
in `_ADMIN_VIEWER_BLOCKED_WRITE_ROUTES`, including:
- /team/block, /team/unblock, /team/permissions_update
- /jwt/key/mapping/{new,update,delete}
- /key/bulk_update
- /key/{key_id}/reset_spend
Remove the fallback. The two remaining allow sets (admin_viewer_routes and
global_spend_tracking_routes) are both read-only, so removal does not affect
the legitimate POST-as-read cases (e.g. /spend/calculate, which is in
spend_tracking_routes ⊂ admin_viewer_routes).
Tests:
- 8 new parametrized cases pinning each previously-leaking management write
endpoint to 403 on POST for PROXY_ADMIN_VIEW_ONLY.
The reservation path (PR #26845) atomically pre-fills `spend:user:{user_id}`
and admits at the strict-`<` boundary. The legacy `_PROXY_MaxBudgetLimiter`
pre-call hook re-reads the same counter with `>=`, so a reservation that
fills the counter to exactly `max_budget` (e.g. a request without a
`max_tokens` cap that falls back to reserving the smallest remaining
headroom) is rejected by the hook even though the reservation already
admitted it.
Skip the hook when the request's active `budget_reservation` covers
`spend:user:{user_id}`. The reservation is the source of truth for that
counter cross-pod; the legacy `>=` path remains in place for requests
without a reservation (e.g. paths that bypass the reservation entirely).
Reproduces as `tests/otel_tests/test_prometheus.py::test_user_budget_metrics`
on a fresh user with `max_budget=10` calling `fake-openai-endpoint` without
`max_tokens`. Adds focused unit coverage in
`tests/test_litellm/proxy/hooks/test_max_budget_limiter.py`.
Greptile flagged two follow-ups on the OpenAPI/local-registry pre-call
check:
1. **P1 runtime crash via None proxy_logging_obj.**
`kwargs.get("proxy_logging_obj")` is `None` on the MCP entry path,
and `pre_call_tool_check` calls `proxy_logging_obj._create_mcp_request_object_from_kwargs`
unconditionally after the security checks, which would have crashed
every legitimate call with `AttributeError`. Source the logging
object from `litellm.proxy.proxy_server` the same way
`_handle_managed_mcp_tool` already does.
2. **P2 authorization-bypass window when mcp_server is None.**
Previously the new check was guarded by `if mcp_server is not None`,
so any local tool whose registry entry had no resolvable server (a
startup-race window before `_initialize_tool_name_to_mcp_server_name_mapping`
completes, or an orphaned registry entry) ran without the security
check. Tools registered via openapi_to_mcp_generator are always tied
to a server, so a missing one is a configuration/timing fault — fail
the call with 503 instead of dispatching unguarded.
Tests: existing two pass with an added assertion that
`proxy_logging_obj` is non-None at the call site, plus a new test that
covers the 503 deny branch when the tool→server mapping is missing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged that the new `created_by` fallback in
`/agent/daily/activity` resolves to `WHERE created_by IS NULL` when
`user_api_key_dict.user_id` is `None`, which would expose every
ownerless agent's rows to a service-account-style caller without a
user_id. Skip the fallback query entirely in that case so the caller
is treated as having no permitted agents (empty page, no DB hit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`execute_mcp_tool` dispatches in two ways: managed MCP servers go
through `_handle_managed_mcp_tool`, which calls
`MCPServerManager.pre_call_tool_check` to enforce allowed/banned tool
lists, key/team `object_permission` tool grants, and parameter
validation. OpenAPI-backed tools, however, were resolved via
`global_mcp_tool_registry` and dispatched directly to
`_handle_local_mcp_tool` — entirely skipping `pre_call_tool_check`.
A caller could invoke any registered OpenAPI tool regardless of their
key/team permissions, including administrative or destructive
operations on the upstream API.
Run `pre_call_tool_check` before the local-registry dispatch whenever
the resolved server is set (the same condition used to surface server
context to the managed path). Honor any guardrail-modified arguments
the hook returns. Errors raised by the hook propagate up before
`_handle_local_mcp_tool` runs.
Tests cover both directions: the pre-call hook fires when the local
tool resolves alongside a server, and a hook-raised HTTPException
prevents the local handler from being invoked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related cross-tenant leaks in the daily-activity endpoints:
1. `/team/daily/activity` set a single `has_full_team_view` flag that
went True if the caller was admin/perm-holder of ANY one of the
requested teams. An admin of team A could pass `team_ids=A,B` and
read team B's per-API-key breakdown even when they were only a
plain member of team B. Require admin/permission on EVERY requested
team for the unfiltered view; otherwise force fallback to the
caller's own API keys for the whole request. Callers wanting wider
coverage can split into separate requests.
2. `/agent/daily/activity` initialized an empty `where_condition` and
returned every agent's spend/token rows on the proxy when
`agent_ids` was omitted — the dashboard's "Top Agents Driving
Spend" panel triggered this for any authenticated user. For
non-admin callers, scope the query to agents they're permitted to
invoke (`AgentRequestHandler.get_allowed_agents`) or, when their
key/team has no explicit agent allowlist, to agents they created
(`created_by`). Explicit `agent_ids` is intersected with the same
permitted set rather than trusted. When the resolved set is empty,
return an empty paginated page without issuing the query.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related authorization gaps in management endpoints:
1. `/project/update` evaluated permission against the team_id supplied in
the request body. By passing `data.team_id` pointing at a team they
admin, a caller could hijack any project — `_check_user_permission_for_project`
was given the attacker's team_object and happily checked admin
membership against that. Drop the team_object kwarg so the helper
re-fetches the existing project's team. Also require admin rights on
the destination team when reassigning a project across teams, so a
team admin cannot shed projects into another team's namespace.
2. `/key/update` accepted any `organization_id` and only checked that
the org existed before applying limits. A caller could thereby point
their key at an arbitrary org. Add `_validate_caller_can_assign_key_org`
which enforces the same membership rule already applied on the
`/key/list` filter path (`validate_key_list_check`); proxy admins and
no-change updates skip the check.
Tests cover both helpers in isolation: existing-team-admin allow,
unrelated-team admin deny, proxy-admin shortcut, org-member allow,
non-member deny, missing user_id deny, no-memberships deny.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The GitHub merge conflict resolver concatenated both test sets but left
`from litellm.proxy._lazy_openapi_snapshot import _normalize_operation_ids`
stranded between functions instead of at the top of the file.
A non-admin scoped to ["model-a"] could call /health?model_id=id-b
(where id-b belongs to a deployment outside their scope) and the
background-cache code path would return id-b's cached health entry. The
helper returned {model_id} unconditionally, so the cache filter was
driven by an unvalidated id and the global cache leaked the entry — the
ternary `targeted_ids if not None else allowed_model_ids` skipped any
intersection with the caller's allowed deployments.
Make _resolve_targeted_model_ids walk the supplied model_list for both
the model and model_id branches. Callers pass an already-scoped list
(filtered to allowed model_names for non-admins, full list for admins),
so an out-of-scope model_id resolves to an empty set and the cache
filter drops every entry — matching the live path's existing behavior.
When JWT auth is enabled but `JWT_AUDIENCE` is unset, `auth_jwt`
disabled audience verification entirely. Tokens minted by any other
application that shared the same IdP signing keys (Azure AD, Okta,
etc.) were accepted as long as their signature checked out, even
though their `aud` and `iss` claims pointed at unrelated apps. The
proxy then fell into the no-team / no-user branch where access checks
default-allow.
This change:
1. Adds support for the `JWT_ISSUER` env var. When set, PyJWT verifies
the token's `iss` claim — turning on the same defense for tokens
that share an audience but come from a different IdP tenant.
2. Refactors the duplicated `jwt.decode` calls (RSA/EC/OKP path and
x509 path) into a single `_build_decode_kwargs` helper that
computes audience, issuer, and the corresponding `verify_*` opt-outs
once per call.
3. Logs a single startup-time warning when JWT auth is enabled but
neither `JWT_AUDIENCE` nor `JWT_ISSUER` is configured, so operators
running the insecure default see a flag in their logs without
getting spammed per-request.
Default behavior (no env vars) is preserved for backward compatibility.
Setting `JWT_AUDIENCE` and/or `JWT_ISSUER` opts into the verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
