The blocklist check in _check_proxy_admin_viewer_access only fires for
routes that match LiteLLMRoutes.management_routes — the bulk-update
endpoint was missing from that list, so the test for view-only admins
on /team/permissions_bulk_update fell through to "allow."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If record_container_owner raises after the upstream container is created,
the user previously got a 500 with no usable container — they were billed
for an unreachable resource. Move ownership recording into the create
path's exception handling and split the two failure modes:
- HTTPException from the recorder (auth conflicts) propagates verbatim
so the client sees the real status code, not a generic LLM error.
- Unexpected exceptions are logged and swallowed; the response is
returned to the caller so they aren't billed for a container they
can't address. The DB row stays untracked until an operator reconciles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
test_anthropic_messages_streaming_cost_injection hits the proxy's
/v1/messages route, which routes via the anthropic/* wildcard to
api.anthropic.com. The 404 surfaced in the test was Anthropic's own
not_found_error propagated back through the proxy (visible from the
x-litellm-model-id hash on the response — the proxy did route).
Same root cause as the prior commit: the legacy claude-4-sonnet-20250514
alias is no longer recognized by Anthropic's main API under the new key.
Swap to claude-haiku-4-5-20251001 — same routing path, canonical model.
The previous per-provider guards in HuggingFace, Oobabooga, and Gemini
files lived in the SDK layer, breaking SDK callers who legitimately pass
URL-valued model identifiers. Move the check to the proxy boundary in
add_litellm_data_to_request so SDK users keep working while proxy users
default-deny URL-valued model and file_id, with admin opt-in via
litellm.provider_url_destination_allowed_hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three live-API tests pinned to claude-4-sonnet-20250514, which is a
non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API
no longer resolves the legacy form under freshly issued keys, so the
tests fail with not_found_error. The token counter test pinned to
claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks
out) was on borrowed time too.
Bump all four to claude-haiku-4-5-20251001 — capability superset for what
these tests exercise (streaming, parallel tool calling, extended thinking,
token counting), no upcoming deprecation, cheaper per-token.
Mirrors the membership rule on /key/update so that /key/generate and
/key/{key}/regenerate apply the same `_validate_caller_can_assign_key_org`
gate when the caller specifies an `organization_id`. Proxy admins bypass.
The check no-ops when `organization_id` is not being set.
Conflict resolution for #26968 dropped the `Iterator` typing import
(NameError at module load), left a dead `fallback_models = cast(...)`
block, and the new tests called `_enforce_key_and_fallback_model_access`
without the now-required `request` kwarg.
Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the
per-row Metadata pane in the UI no longer shows the stack trace when the
opt-in env var is set, matching the existing console-side suppression.
https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6
Bugbot caught a regression: the previous error log here was a single-line
verbose_proxy_logger.error(...) with no traceback. spend_log_error attaches
the active exception's traceback by default (when the suppression env var
is unset), so swapping it in changed default behavior. Revert this one site
to its original .error() call to keep the PR strictly opt-in.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
This call site previously logged a single-line error via verbose_proxy_logger.error()
with no traceback. Switching it to spend_log_error(..., exc=e) caused a full stack
trace to render by default (when LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS is unset),
which contradicts the PR goal of leaving default behavior unchanged. Revert this
specific site to the original error log call.
Adds LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS env var. When set to true and the
proxy log level is INFO or above, spend-tracking error paths emit a single
ERROR line without the full traceback. Stack traces are preserved at DEBUG
and the Sentry / proxy_logging_obj.failure_handler path is unchanged.
The new spend_log_error helper is wired through the spend write hot path:
- DBSpendUpdateWriter (update_database, _update_*_db, batch upsert,
redis-commit fallbacks)
- _ProxyDBLogger._PROXY_track_cost_callback
- get_logging_payload exception path
- update_spend / update_daily_tag_spend / spend logs queue monitor
Resolves LIT-2704.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(proxy): re-validate user_id ownership after /user/info re-parses query
The route-level access check in `RouteChecks.non_proxy_admin_allowed_routes_check`
reads `request.query_params.get("user_id")`, which decodes literal `+` to
spaces. The endpoint then re-parses the raw query string with `urllib.unquote`
in `get_user_id_from_request` to preserve `+` characters (so plus-addressed
emails work as user_ids). Those two paths produce different ids: a caller
who registered a user_id containing a literal space could pass the route
check and then read another user's row by sending the encoded `+` form.
Add `_enforce_user_info_access` and call it after `_normalize_user_info_user_id`
returns the final id. Proxy admin / view-only admin still bypass; everyone
else must match the resolved user_id (or have no user_id, which falls back
to the caller's own id later in the handler).
Tests cover the admin bypass, owner-match path, and the cross-user lookup
that this change blocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(proxy): apply user_info ownership check to PROXY_ADMIN_VIEW_ONLY
`_enforce_user_info_access` was bypassing both PROXY_ADMIN and
PROXY_ADMIN_VIEW_ONLY, but the upstream route check in
`RouteChecks.non_proxy_admin_allowed_routes_check` only treats
PROXY_ADMIN as a true admin for the `/user/info` route — view-only
admins go through the `user_id == valid_token.user_id` enforcement
along with regular users. Mirroring that asymmetry left the same
encoded-`+` bypass open for view-only admins whose user_id contains a
literal space.
Drop the PROXY_ADMIN_VIEW_ONLY exemption so the post-decode re-check
matches the upstream rule. Update tests: a view-only admin must now
be blocked from cross-user lookups but still allowed to read their
own row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count non-chat tokens and validate every model in batch file
Two security control bypasses on POST /v1/batches:
1. `_get_batch_job_input_file_usage` only summed tokens for
`body.messages` (chat completions). Embedding (`input`) and text
completion (`prompt`) batches reported zero, letting massive
non-chat workloads slip past TPM rate limits. Extend the counter
to handle string and list shapes for both fields.
2. The batch input file was forwarded to the upstream provider
without inspecting the models named inside the JSONL — only the
outer `model` query parameter was checked against the caller's
allowlist. A caller restricted to gpt-3.5 could submit a batch
targeting gpt-4o and the upstream would execute it under the
proxy's shared API key.
Add `_get_models_from_batch_input_file_content` (returns the
distinct `body.model` values) and call it from
`_enforce_batch_file_model_access` in the pre-call hook, which runs
each model through `can_key_call_model` so the same allowlist
semantics (wildcards, access groups, all-proxy-models, team aliases)
the proxy enforces on `/chat/completions` apply here too. Any
unauthorized model raises a 403 before the file is forwarded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(batches): count pre-tokenized prompt/input shapes, classify 403 logs
Two follow-ups from the Greptile review on the batch validation PR:
1. P1 TPM bypass via integer token arrays. The OpenAI batch schema
accepts ``prompt`` and ``input`` as ``list[int]`` (a single
pre-tokenized prompt) or ``list[list[int]]`` (multiple) in addition
to the string and ``list[str]`` shapes. Pre-fix only the string
shapes were counted, so a caller could submit a batch with hundreds
of millions of pre-tokenized tokens and the rate limiter would
record zero. Extract the per-field logic into
``_count_prompt_or_input_tokens`` and count each int as one token.
2. P2 access-denial logs were indistinguishable from I/O failures.
``count_input_file_usage`` caught every exception under a generic
"Error counting input file usage" message, so an intentional 403
from ``_enforce_batch_file_model_access`` looked the same in the
logs as a missing file or a Prisma timeout. Catch ``HTTPException``
separately and log 403s at WARNING level with a security-relevant
message before re-raising.
Tests cover the new shapes: single ``list[int]``, ``list[list[int]]``
(the worst-case bypass vector), and embeddings ``input`` with
pre-tokenized arrays.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reset the policy ID index during policy engine test cleanup so stale policy versions cannot leak between tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
The router_code_coverage CI check requires every function in router.py to
be referenced by at least one test under tests/{local_testing,
router_unit_tests,test_litellm} in a file with "router" in its name.
The recently-extracted helper had no direct test, so the check failed
with "0.45% of functions in router.py are not tested".
Add a focused test that exercises the four return paths: model already
in self.model_names, no fallback applies, pattern-router match, and
default_deployment substitution (also asserting the stored default
isn't mutated).
https://claude.ai/code/session_019AVp1XL7RT9RxRe4qRLkay
Mark runtime-created policies and attachments initialized so global policy attachments created from the policy builder apply immediately without requiring a restart.
Co-authored-by: Cursor <cursoragent@cursor.com>
Many MCP integrations (Zapier, etc.) embed an upstream API key
directly in the server URL, e.g.
``https://actions.zapier.com/mcp/<api-key>/sse``. The list and
single-server endpoints were returning the full URL to any
authenticated user — `_redact_mcp_credentials` only stripped the
explicit ``credentials`` field, and `_sanitize_mcp_server_for_virtual_key`
only ran for restricted virtual keys. Non-admin internal users could
read the dashboard, click the unmask toggle, and exfiltrate the raw
token.
Add `_sanitize_mcp_server_for_non_admin` that runs on top of the
existing credential redaction and clears the credential-bearing
fields:
- ``url`` (the primary leak vector)
- ``spec_path`` (OpenAPI spec URLs that may carry tokens)
- ``static_headers`` / ``extra_headers`` (Authorization)
- ``env`` (arbitrary secrets)
- ``authorization_url`` / ``token_url`` / ``registration_url``
Identity fields (``server_id``, ``alias``, ``mcp_info``, etc.) are
preserved so the UI can still list servers a non-admin's team has
access to.
Apply the new sanitizer in `fetch_all_mcp_servers` and the per-server
fetch path right after the existing virtual-key branch. Update the
existing `test_list_mcp_servers_non_admin_user_filtered` assertions
that previously checked URL visibility.
Frontend defense-in-depth: hide the URL unmask toggle on
`mcp_server_view.tsx` unless the viewer is a proxy admin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When _get_all_deployments returns 0 candidates and the litellm-model
fallback branch (_get_deployment_by_litellm_model) finds deployments that
the access-group filter then empties, _access_group_filter_emptied_candidates
remained False (it was captured before that branch ran). The router would
then proceed to default fallbacks; the fallback model could have no
access_groups and short-circuit the filter, silently serving a caller
blocked by access-group restrictions.
Update the flag inside the litellm-model branch when filtering empties a
non-empty candidate set so the default-fallback guard still triggers.
When a team grants /key/list via team_member_permissions, non-admin members
should see all keys for that team — same as a team admin. Previously the
classification in list_keys() only checked admin status, so permitted
members fell into the service-account-only path and could not see other
members' personal keys. Routes those members into the full-visibility set.
Replaces the previous intersect-with-team.access_group_ids check, which
made the override unreachable in practice (the team-gate fallback already
covered every case the intersection allowed). The override now resolves
each of the key's access_group_ids via get_access_object and accepts the
group only if its assigned_team_ids includes the key's team_id, or its
assigned_key_ids includes the key's token. This fulfills the original ask
(a key can extend a team's allow-list via a group the admin granted to
that team or that specific key) while still rejecting foreign groups
referenced by team members of other teams.
`os.path.relpath` with no `start` arg uses the current working
directory, so running pytest from a subdirectory produced a
different Redis key than running from the repo root. CI-recorded
cassettes and locally-replayed runs would silently miss each
other's cache.
Anchor the path to the repo root (derived from `__file__`) so the
key is stable regardless of CWD.
https://claude.ai/code/session_018uCx7pcrkdUJZrCVMaTdPx
