The previous per-provider guards in HuggingFace, Oobabooga, and Gemini
files lived in the SDK layer, breaking SDK callers who legitimately pass
URL-valued model identifiers. Move the check to the proxy boundary in
add_litellm_data_to_request so SDK users keep working while proxy users
default-deny URL-valued model and file_id, with admin opt-in via
litellm.provider_url_destination_allowed_hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflict resolution for #26968 dropped the `Iterator` typing import
(NameError at module load), left a dead `fallback_models = cast(...)`
block, and the new tests called `_enforce_key_and_fallback_model_access`
without the now-required `request` kwarg.
Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the
per-row Metadata pane in the UI no longer shows the stack trace when the
opt-in env var is set, matching the existing console-side suppression.
https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6
Adds LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS env var. When set to true and the
proxy log level is INFO or above, spend-tracking error paths emit a single
ERROR line without the full traceback. Stack traces are preserved at DEBUG
and the Sentry / proxy_logging_obj.failure_handler path is unchanged.
The new spend_log_error helper is wired through the spend write hot path:
- DBSpendUpdateWriter (update_database, _update_*_db, batch upsert,
redis-commit fallbacks)
- _ProxyDBLogger._PROXY_track_cost_callback
- get_logging_payload exception path
- update_spend / update_daily_tag_spend / spend logs queue monitor
Resolves LIT-2704.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(proxy): re-validate user_id ownership after /user/info re-parses query
The route-level access check in `RouteChecks.non_proxy_admin_allowed_routes_check`
reads `request.query_params.get("user_id")`, which decodes literal `+` to
spaces. The endpoint then re-parses the raw query string with `urllib.unquote`
in `get_user_id_from_request` to preserve `+` characters (so plus-addressed
emails work as user_ids). Those two paths produce different ids: a caller
who registered a user_id containing a literal space could pass the route
check and then read another user's row by sending the encoded `+` form.
Add `_enforce_user_info_access` and call it after `_normalize_user_info_user_id`
returns the final id. Proxy admin / view-only admin still bypass; everyone
else must match the resolved user_id (or have no user_id, which falls back
to the caller's own id later in the handler).
Tests cover the admin bypass, owner-match path, and the cross-user lookup
that this change blocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(proxy): apply user_info ownership check to PROXY_ADMIN_VIEW_ONLY
`_enforce_user_info_access` was bypassing both PROXY_ADMIN and
PROXY_ADMIN_VIEW_ONLY, but the upstream route check in
`RouteChecks.non_proxy_admin_allowed_routes_check` only treats
PROXY_ADMIN as a true admin for the `/user/info` route — view-only
admins go through the `user_id == valid_token.user_id` enforcement
along with regular users. Mirroring that asymmetry left the same
encoded-`+` bypass open for view-only admins whose user_id contains a
literal space.
Drop the PROXY_ADMIN_VIEW_ONLY exemption so the post-decode re-check
matches the upstream rule. Update tests: a view-only admin must now
be blocked from cross-user lookups but still allowed to read their
own row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count non-chat tokens and validate every model in batch file
Two security control bypasses on POST /v1/batches:
1. `_get_batch_job_input_file_usage` only summed tokens for
`body.messages` (chat completions). Embedding (`input`) and text
completion (`prompt`) batches reported zero, letting massive
non-chat workloads slip past TPM rate limits. Extend the counter
to handle string and list shapes for both fields.
2. The batch input file was forwarded to the upstream provider
without inspecting the models named inside the JSONL — only the
outer `model` query parameter was checked against the caller's
allowlist. A caller restricted to gpt-3.5 could submit a batch
targeting gpt-4o and the upstream would execute it under the
proxy's shared API key.
Add `_get_models_from_batch_input_file_content` (returns the
distinct `body.model` values) and call it from
`_enforce_batch_file_model_access` in the pre-call hook, which runs
each model through `can_key_call_model` so the same allowlist
semantics (wildcards, access groups, all-proxy-models, team aliases)
the proxy enforces on `/chat/completions` apply here too. Any
unauthorized model raises a 403 before the file is forwarded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count pre-tokenized prompt/input shapes, classify 403 logs
Two follow-ups from the Greptile review on the batch validation PR:
1. P1 TPM bypass via integer token arrays. The OpenAI batch schema
accepts ``prompt`` and ``input`` as ``list[int]`` (a single
pre-tokenized prompt) or ``list[list[int]]`` (multiple) in addition
to the string and ``list[str]`` shapes. Pre-fix only the string
shapes were counted, so a caller could submit a batch with hundreds
of millions of pre-tokenized tokens and the rate limiter would
record zero. Extract the per-field logic into
``_count_prompt_or_input_tokens`` and count each int as one token.
2. P2 access-denial logs were indistinguishable from I/O failures.
``count_input_file_usage`` caught every exception under a generic
"Error counting input file usage" message, so an intentional 403
from ``_enforce_batch_file_model_access`` looked the same in the
logs as a missing file or a Prisma timeout. Catch ``HTTPException``
separately and log 403s at WARNING level with a security-relevant
message before re-raising.
Tests cover the new shapes: single ``list[int]``, ``list[list[int]]``
(the worst-case bypass vector), and embeddings ``input`` with
pre-tokenized arrays.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reset the policy ID index during policy engine test cleanup so stale policy versions cannot leak between tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
The router_code_coverage CI check requires every function in router.py to
be referenced by at least one test under tests/{local_testing,
router_unit_tests,test_litellm} in a file with "router" in its name.
The recently-extracted helper had no direct test, so the check failed
with "0.45% of functions in router.py are not tested".
Add a focused test that exercises the four return paths: model already
in self.model_names, no fallback applies, pattern-router match, and
default_deployment substitution (also asserting the stored default
isn't mutated).
https://claude.ai/code/session_019AVp1XL7RT9RxRe4qRLkay
Mark runtime-created policies and attachments initialized so global policy attachments created from the policy builder apply immediately without requiring a restart.
Co-authored-by: Cursor <cursoragent@cursor.com>
Many MCP integrations (Zapier, etc.) embed an upstream API key
directly in the server URL, e.g.
``https://actions.zapier.com/mcp/<api-key>/sse``. The list and
single-server endpoints were returning the full URL to any
authenticated user — `_redact_mcp_credentials` only stripped the
explicit ``credentials`` field, and `_sanitize_mcp_server_for_virtual_key`
only ran for restricted virtual keys. Non-admin internal users could
read the dashboard, click the unmask toggle, and exfiltrate the raw
token.
Add `_sanitize_mcp_server_for_non_admin` that runs on top of the
existing credential redaction and clears the credential-bearing
fields:
- ``url`` (the primary leak vector)
- ``spec_path`` (OpenAPI spec URLs that may carry tokens)
- ``static_headers`` / ``extra_headers`` (Authorization)
- ``env`` (arbitrary secrets)
- ``authorization_url`` / ``token_url`` / ``registration_url``
Identity fields (``server_id``, ``alias``, ``mcp_info``, etc.) are
preserved so the UI can still list servers a non-admin's team has
access to.
Apply the new sanitizer in `fetch_all_mcp_servers` and the per-server
fetch path right after the existing virtual-key branch. Update the
existing `test_list_mcp_servers_non_admin_user_filtered` assertions
that previously checked URL visibility.
Frontend defense-in-depth: hide the URL unmask toggle on
`mcp_server_view.tsx` unless the viewer is a proxy admin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When _get_all_deployments returns 0 candidates and the litellm-model
fallback branch (_get_deployment_by_litellm_model) finds deployments that
the access-group filter then empties, _access_group_filter_emptied_candidates
remained False (it was captured before that branch ran). The router would
then proceed to default fallbacks; the fallback model could have no
access_groups and short-circuit the filter, silently serving a caller
blocked by access-group restrictions.
Update the flag inside the litellm-model branch when filtering empties a
non-empty candidate set so the default-fallback guard still triggers.
When a team grants /key/list via team_member_permissions, non-admin members
should see all keys for that team — same as a team admin. Previously the
classification in list_keys() only checked admin status, so permitted
members fell into the service-account-only path and could not see other
members' personal keys. Routes those members into the full-visibility set.
Greptile P1: the unsafe-method branch of `_check_proxy_admin_viewer_access`
ended with a blanket `if route in management_routes: return`. That set is a
mix of reads (info/list — handled via the safe-method GET branch above) and
writes. The fallback let Admin Viewer POST to write endpoints not enumerated
in `_ADMIN_VIEWER_BLOCKED_WRITE_ROUTES`, including:
- /team/block, /team/unblock, /team/permissions_update
- /jwt/key/mapping/{new,update,delete}
- /key/bulk_update
- /key/{key_id}/reset_spend
Remove the fallback. The two remaining allow sets (admin_viewer_routes and
global_spend_tracking_routes) are both read-only, so removal does not affect
the legitimate POST-as-read cases (e.g. /spend/calculate, which is in
spend_tracking_routes ⊂ admin_viewer_routes).
Tests:
- 8 new parametrized cases pinning each previously-leaking management write
endpoint to 403 on POST for PROXY_ADMIN_VIEW_ONLY.
The reservation path (PR #26845) atomically pre-fills `spend:user:{user_id}`
and admits at the strict-`<` boundary. The legacy `_PROXY_MaxBudgetLimiter`
pre-call hook re-reads the same counter with `>=`, so a reservation that
fills the counter to exactly `max_budget` (e.g. a request without a
`max_tokens` cap that falls back to reserving the smallest remaining
headroom) is rejected by the hook even though the reservation already
admitted it.
Skip the hook when the request's active `budget_reservation` covers
`spend:user:{user_id}`. The reservation is the source of truth for that
counter cross-pod; the legacy `>=` path remains in place for requests
without a reservation (e.g. paths that bypass the reservation entirely).
Reproduces as `tests/otel_tests/test_prometheus.py::test_user_budget_metrics`
on a fresh user with `max_budget=10` calling `fake-openai-endpoint` without
`max_tokens`. Adds focused unit coverage in
`tests/test_litellm/proxy/hooks/test_max_budget_limiter.py`.
Greptile flagged two follow-ups on the OpenAPI/local-registry pre-call
check:
1. **P1 runtime crash via None proxy_logging_obj.**
`kwargs.get("proxy_logging_obj")` is `None` on the MCP entry path,
and `pre_call_tool_check` calls `proxy_logging_obj._create_mcp_request_object_from_kwargs`
unconditionally after the security checks, which would have crashed
every legitimate call with `AttributeError`. Source the logging
object from `litellm.proxy.proxy_server` the same way
`_handle_managed_mcp_tool` already does.
2. **P2 authorization-bypass window when mcp_server is None.**
Previously the new check was guarded by `if mcp_server is not None`,
so any local tool whose registry entry had no resolvable server (a
startup-race window before `_initialize_tool_name_to_mcp_server_name_mapping`
completes, or an orphaned registry entry) ran without the security
check. Tools registered via openapi_to_mcp_generator are always tied
to a server, so a missing one is a configuration/timing fault — fail
the call with 503 instead of dispatching unguarded.
Tests: existing two pass with an added assertion that
`proxy_logging_obj` is non-None at the call site, plus a new test that
covers the 503 deny branch when the tool→server mapping is missing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged that the new `created_by` fallback in
`/agent/daily/activity` resolves to `WHERE created_by IS NULL` when
`user_api_key_dict.user_id` is `None`, which would expose every
ownerless agent's rows to a service-account-style caller without a
user_id. Skip the fallback query entirely in that case so the caller
is treated as having no permitted agents (empty page, no DB hit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
