* fix duplicate cost callbacks for anthropic streaming pass-through
Two bugs caused _PROXY_track_cost_callback to see stream=True +
complete_streaming_response=None on every streaming pass-through request,
making the dedup guard in dispatch_success_handlers permanently inactive:
1. pass_through_endpoints.py created the Logging object with stream=False
for all requests. _is_assembled_stream_success short-circuits on
self.stream is not True, so has_dispatched_final_stream_success was
never set and any second dispatch went through unchecked.
Fix: set logging_obj.stream = True after stream detection.
2. _create_anthropic_response_logging_payload set complete_streaming_response
inside the try block after litellm.completion_cost(), so a pricing error
caused an early return without setting it on model_call_details.
Fix: set complete_streaming_response before the try block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix stream
* add stream to logging obj
* test(pass_through): give mock logging object a real model_call_details dict
The anthropic passthrough logging payload now records the assembled
response on model_call_details before cost calculation, which requires
model_call_details to support item assignment. In production it is always
a dict; the existing unit test stubbed the logging object with a bare Mock
whose attribute is not subscriptable, so the new assignment raised
TypeError. Use a real dict to match the production logging object.
* test(pass_through): cover streaming logging-obj stream flag
The streaming branch of pass_through_request that marks the logging object
as streaming (logging_obj.stream and model_call_details["stream"]) had no
unit coverage, so the patch coverage gate flagged it. Add a regression test
that drives a streaming pass-through request through pass_through_request and
asserts the logging object is flagged as a stream before dispatch.
* test(pass_through): cover SSE-response stream flag fallback branch
The auto-detected streaming branch of pass_through_request (when a request
that was not flagged as streaming returns a text/event-stream response) sets
logging_obj.stream and model_call_details["stream"] but had no unit coverage,
so the codecov patch gate failed at 60%. Drive a non-streaming pass-through
request whose upstream response is SSE through pass_through_request and assert
the logging object is flagged as a stream before dispatch.
* fix(pass_through): gate complete_streaming_response on stream flag
perform_redaction only scrubs complete_streaming_response when
model_call_details["stream"] is True. Setting it unconditionally for
non-streaming Anthropic pass-through responses left the assembled
response unredacted in model_call_details, which is handed to logging
callbacks as kwargs when message logging is disabled. Only record it for
actual streaming responses so redaction always applies.
---------
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit 2bbdbfa5c3)
* fix(vertex): strip output_config.effort for models that reject it
Haiku 4.5 on Vertex AI does not support output_config.effort and 400s with
"output_config.effort: Extra inputs are not permitted". PR #27074 emptied
VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS so effort would forward for Opus/Sonnet
4.6+, but that made the strip unconditional across every Vertex Anthropic
model, including ones that don't support it. Claude Code injects effort into
its default Messages payload, so `claude --model claude-haiku-4.5` started
failing.
Make the sanitizer model-aware: drop output_config.effort for models that
don't advertise output_config support (or any reasoning effort level) while
forwarding it for those that do. The fix covers both the chat-completion and
Messages pass-through transformation paths since they share the helper.
* chore(vertex): log at debug when dropping unsupported output_config.effort
Operators pointing an unregistered Vertex Claude alias that does support
effort would otherwise see it stripped with no signal. Debug level keeps it
out of normal logs since Claude Code sends effort on every request.
(cherry picked from commit cc55662e5f)
* fix(key_generate): allow team members to create keys on org-scoped teams
When a virtual key is created for a team, enterprise logic inherits the
team's organization_id onto the key (add_team_organization_id). Since the
VERIA-55 org-IDOR fix, /key/generate then required the caller to be an
explicit LiteLLM_OrganizationMembership member of that org, returning
403 "Caller is not a member of organization_id=<uuid>". Admins normally
only add users to teams (not orgs), so self-serve key creation regressed
for any user on an org-scoped team (regression since v1.84.0-rc.1).
Skip the org-membership check when organization_id was inherited from the
key's team (organization_id == team_table.organization_id). Team-level
authorization already gates this path, so team membership is sufficient.
The membership check still runs when a caller assigns an organization_id
that did not come from the key's team, preserving the IDOR protection.
Adds regression tests covering both the team-inherited (allowed) and
foreign-org (still blocked) cases.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(key_generate): cover mismatched team org IDOR path on generate
Add test_generate_key_foreign_org_with_mismatched_team_still_enforces_membership
for the case where a team is present but request organization_id differs from
team_table.organization_id. Enterprise inheritance is no-op'd in the test so
the guard is exercised directly; membership validation must still run.
Addresses Greptile review on #29310.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit b11833c737)
* fix(proxy): map stripped batch body.model to proxy alias for auth
replace_model_in_jsonl rewrites JSONL body.model to the provider id before
upload; batch file access checks must resolve that id back to model_name
so keys granted the proxy alias are not rejected with 403.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(proxy): surface resolved proxy alias in batch file 403 detail
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 70d2748d80)
* fix(azure): preserve AD token refresh in v1 OpenAI client path
The /openai/v1/ code path (api_version in {"v1", "latest", "preview"})
constructs a plain OpenAI/AsyncOpenAI client, but only forwarded
`api_key` from `azure_client_params`. When `enable_azure_ad_token_refresh`
is set (or any AD-only auth), `api_key` is None and the client
constructor raised "The api_key client option must be set...", breaking
every Azure call with a v1 api_version.
The OpenAI SDK (>=2.20.0) accepts a callable for `api_key` and re-invokes
it on every request via `_refresh_api_key`, so we now forward
`azure_ad_token_provider` directly — preserving the per-request token
refresh behavior of the regular AzureOpenAI client and avoiding the
expiry hole that resolving the token once at client-creation time would
introduce. Static `azure_ad_token` strings fall through to `api_key`.
For the async path we wrap the sync provider returned by azure-identity
in an async function since AsyncOpenAI expects `Callable[[], Awaitable[str]]`.
Fixes#27945https://claude.ai/code/session_01UnzrDSFUUgp5T2wRoPMxq5
* fix(azure): offload sync token provider to thread in v1 async wrapper
* fix(azure): include AD credential identity in v1 client cache key
---------
Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit 96a2e8b16d)
* fix(reset_budget): write only {spend, budget_reset_at} and stop pre-zeroing counter
ResetBudgetJob's batched update_data path shipped the full key/user/team
model on each reset. Prisma rejects object_permission_id and budget_limits
on the update input type, so any row carrying those fields detonated the
entire batch -- spend never reset, budget_reset_at never advanced. After
v1.84.0 started populating object_permission_id on UI-created keys, this
fires routinely.
_reset_budget_common also zeroed the cross-pod spend counter before the
DB write, so failed resets left enforcement reading 0 from the counter
while the DB still held the over-budget spend, admitting requests past
the cap until the counter naturally re-saturated from new reservations.
Switch the write to per-row narrow updates ({spend, budget_reset_at})
via db.batch_, and move the counter invalidation out of
_reset_budget_common so it only fires after the DB write commits. On
DB-write failure the counter is left untouched, enforcement continues
to block, and the next scheduler tick can retry without leaving a
bypass window.
Fixes#27730.
* fix(reset_budget): address Greptile review on #29358
- Strengthen the bypass-half regression test: replace the for-loop over
call_args_list (vacuously true when empty) with assert_not_called(),
so the test would actually flag a re-introduction of counter-zeroing
via any code path.
- Add the same explanatory docstring on _write_user_reset_updates and
_write_team_reset_updates that _write_key_reset_updates already has,
so all three helpers point future maintainers at #27730.
* test(reset_budget): update test_proxy_budget_reset for new batch-write path
Same shape as the previous test_reset_budget_job.py update: keys/users/teams
now write through prisma.db.batch_().<table>.update, not update_data, so the
tests need a batcher mock and updated assertions. Adds:
- _wire_batcher_for_test helper that returns a list which accumulates per-row
batch updates captured from prisma_client.db.batch_().
- _attrify helper that wraps dict fixtures so getattr(item, "token") works
alongside the dict item-access the fake_reset_* mocks rely on. The new
narrow-write helpers use getattr to pull out the row's id, and would
silently skip plain dicts otherwise.
- Updates 3 partial_failure tests to assert against the batch-call list
(rows by id, payload contains only {spend, budget_reset_at}) instead of
update_data.assert_awaited_once + data_list inspection.
- Updates test_reset_budget_continues_other_categories_on_failure: only
budget + enduser still flow through update_data; key/user/team go through
the batch path now.
- Wires the batcher mock into 3 service_logger_*_success tests so commit()
is actually awaitable and the success hook fires.
These tests were silently passing locally only because the editable install
in .venv pointed at the main repo, not the worktree — running pytest with
PYTHONPATH overridden to the worktree (matching CI) reproduces the failures.
* [internal copy of #29089] fix: duplicate claude code traces (#29311)
* refactor(proxy/auth): normalize Bearer prefix in safe-hash helper (#29343)
* refactor(proxy/auth): normalize Bearer prefix in safe-hash helper
UserAPIKeyAuth._safe_hash_litellm_api_key now strips a leading
"Bearer "/"bearer " prefix before its existing sk-/JWT classification, so
the helper produces the same hashed output regardless of whether the
caller stripped the Authorization header prefix or passed the header
value through unchanged.
* refactor(proxy/auth): make Bearer-prefix strip case-insensitive
Per RFC 7235 the HTTP authorization scheme token is case-insensitive.
Replace the two-prefix loop with a single case-insensitive check so the
helper normalizes "Bearer ", "bearer ", "BEARER ", and any mixed-case
variant before classifying the remainder as sk- or JWT. The contract
test gains coverage of "BEARER " and "BeArEr ".
* test(mcp): align auth-handler test expectations with safe-hash helper
The two MCP auth tests asserted that UserAPIKeyAuth(api_key="Bearer ...")
retained the raw header bytes on the api_key field. _safe_hash_litellm_api_key
now normalizes that input — stripping the Bearer prefix and hashing the
resulting sk- key — so the expectations move to the normalized form:
the bare token in the parametrize case, and hash_token("sk-...") in the
backward-compat assertion. This matches what the real auth flow produces
(the builder strips Bearer and the DB stores the hashed token), so the
mocks now line up with production rather than with the un-normalized
validator output.
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix(docker): restore npm to non_root builder image (#28519)
The non_root builder stage installs `nodejs` but not `npm`. Without `npm`
on PATH, prisma-python falls back to downloading a Node runtime via
nodeenv from nodejs.org, and that downloaded binary fails to load
`libatomic.so.1` — breaking `prisma generate` and the image build.
`npm` was dropped from this apk list in ca52e346b0. Restoring it lets
prisma-python use the system Node + npm, matching docker/Dockerfile
which already installs `npm` for the same reason.
* bump: version 1.84.2 → 1.84.3
* chore: uv lock after version bump 1.84.2 → 1.84.3
* chore(proxy): cherry-pick #28547 onto patch/v1.84.1
Backport of #28547 (`d480ffda3c`) onto the `patch/v1.84.1` branch.
Routes the remaining path-dependent call sites in auth, ACL, routing,
and audit-log decisions through `get_request_route(request)` so they
read from the ASGI `scope["path"]` instead of `request.url.path`. The
helper itself already exists on v1.84.1 (added by #27904 / #27878);
this PR extends the helper's usage to the additional sites listed
below.
Sites routed through get_request_route:
- _experimental/mcp_server/auth/user_api_key_auth_mcp.py
- management_endpoints/mcp_management_endpoints.py
- vector_store_endpoints/utils.py
- pass_through_endpoints/pass_through_endpoints.py
- auth/route_checks.py
- litellm_pre_call_utils.py
- spend_tracking/spend_management_endpoints.py
- common_utils/http_parsing_utils.py
- management_helpers/utils.py
- health_endpoints/_health_endpoints.py
Regression tests in tests/proxy_unit_tests/test_proxy_routes.py
construct a Request with scope["path"] set to a benign route and the
Host header crafted so url.path would resolve differently; each
site's decision is asserted against scope["path"].
Conflict resolution
-------------------
Two files conflicted because v1.84.1's base predates the
delegate_auth_to_upstream feature (#27834 — not on v1.84.1):
1. _experimental/mcp_server/auth/user_api_key_auth_mcp.py
The cherry-pick brought in a `_target_servers_delegate_auth_to_upstream`
elif branch in `process_mcp_request`. That branch is feature drift
from #27834 and is irrelevant to the path-resolution change. Dropped
the elif block; kept the get_request_route swap on the existing
well-known/_target_servers_use_oauth2 call sites.
2. management_endpoints/mcp_management_endpoints.py
The cherry-pick brought in the entire `_mcp_oauth_user_api_key_auth`
function. That function does not exist on v1.84.1 (added by #27834);
the #28547 change inside it is just a `request.url.path` →
`get_request_route` swap. Dropped the function entirely.
The other 8 production files and the test file auto-merged cleanly
and contain only `request.url.path` → `get_request_route(request)`
swaps plus the lazy auth_utils import (no feature drift).
* bump: version 1.84.1 → 1.84.2
* chore: uv lock after version bump 1.84.1 → 1.84.2
* fix(vertex_ai): omit function_call id on Vertex Gemini 3.5+ tool turns
Vertex AI rejects `id` on function_call/function_response parts; only Google AI Studio accepts it for Gemini 3.5+ strict tool matching.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Update litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(vertex_ai): forward custom_llm_provider in context caching
Pass custom_llm_provider through to _gemini_convert_messages_with_history
in the context caching path so Gemini 3.5+ tool-call `id` forwarding
behaves consistently between cached and non-cached completions on Google
AI Studio.
Co-authored-by: Claude <claude@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <claude@anthropic.com>
(cherry picked from commit fecf212d70)
* fix(spend_counter): seed Redis counter via SET NX to prevent cross-pod double-seed
Symptom
-------
Customers on multi-pod deployments see team `spend` jump to ~2x (or N x
the pod count) shortly after a Redis cache miss / TTL expiry, triggering
spurious "Budget Crossed" alerts and blocked requests until the value is
manually reset.
Root cause
----------
`SpendCounterReseed.coalesced` warmed the primary spend counter by
calling `redis.async_increment(key, value=db_spend, refresh_ttl=True)`,
which lowers to Redis `INCRBYFLOAT`. That is additive, not idempotent.
The per-counter `asyncio.Lock` only coalesces seeders inside one
process. With N pods sharing one Redis, on a cold key (cold start, TTL
expiry, manual delete) every pod independently passes its lock + Redis
re-check, reads the same `db_spend`, and issues `INCRBYFLOAT db_spend`.
Final value: N x db_spend.
Fix
---
Use `redis.async_set_cache(key, value=db_spend, nx=True)` for the seed.
SET NX is atomic across pods: exactly one writer initializes the key;
losers read the winner's value via `async_get_cache`. This is the same
idiom already used by `coalesced_window` in the same file, so the two
seed paths are now consistent.
Per-request deltas continue to use `INCRBYFLOAT` (correct - additive
behaviour is what we want for increments, not for initial seed).
Verification
------------
Live two-process repro against the same Postgres + Redis (DB
spend = 506):
Unpatched: 4/4 runs -> Redis counter = ~1012 (~2 x db_spend)
Patched: 12/12 runs -> Redis counter = ~506
Unit tests (`test_proxy_server.py`):
- New `test_primary_spend_counter_redis_concurrent_seed_does_not_double_seed`
patches `_get_lock` to return a fresh lock per caller (otherwise the
per-process lock masks the race), races two `coalesced` calls, and
asserts final = 506 with exactly one of two SET NX attempts winning.
- 4 existing tests updated for the new seed contract (SET NX for the
seed, INCRBYFLOAT only for the per-request delta).
- Full `spend_counter or reseed or budget` slice: 22 passed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(spend_counter): make SET NX mock atomic so loser branch is exercised
Greptile flagged that `redis_set_cache` in
test_primary_spend_counter_redis_concurrent_seed_does_not_double_seed
placed `await asyncio.sleep(0)` AFTER the NX membership check. Both
concurrent tasks observed an empty `redis_store`, passed the guard, and
both returned True - so the loser branch (else: read back winner's value)
was never exercised.
Fix the mock to model real atomic Redis SET NX:
- Yield BEFORE the membership check so two concurrent callers interleave
the way real SET NX does (first to resume runs check + write atomically
and wins; second resumes after the key exists and loses).
- Track set_cache return values; assert sorted([loser, winner]) so we
know exactly one task wins and one loses.
- Track async_get_cache calls that happen AFTER at least one SET NX has
completed; assert at least one such read - that is the loser-path
fallback (`current_value = float(cached)` when seeded is False).
Verified by temporarily reverting the mock to the old order: the test
now fails with `expected exactly one SET NX winner and one loser, got
[True, True]`, exactly the failure mode Greptile described.
No production code change.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(spend_counter): mock async_set_cache to populate redis_store in concurrent read+write test
`test_concurrent_read_and_write_paths_share_one_db_query` mocks
`async_increment` to populate the in-memory `redis_store`, but did not
mock `async_set_cache`. After the SET-NX seed change in `coalesced()`,
the seed step writes via `async_set_cache(nx=True)` (default AsyncMock,
no `redis_store` write), so the simulated Redis stays empty after the
first reseed. The second `get_current_spend` then sees a clean Redis
miss, re-enters the DB read path, and the test fails with
`expected 1 DB query, got 2`.
Fix: add a `redis_set_cache` side_effect that updates `redis_store` on
`nx=True` (and rejects when the key already exists), matching the
pattern used by the four sibling tests fixed in this branch's first
commit. Pre-existing assertions are unchanged.
Full `tests/test_litellm/proxy/test_proxy_server.py`: 158 passed.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit 0fb710400f)
* fix: patch Host-header auth bypass in get_request_route
Starlette reconstructs request.url from the Host header. A malformed
Host like `localhost/?x=1` causes Starlette to build the full URL as
`http://localhost/?x=1/health`, which url-parses to path="/". Since "/"
is in LiteLLMRoutes.public_routes, all protected routes became reachable
without authentication.
Fix: read scope["path"] (set by uvicorn from the HTTP request line,
not derivable from headers) instead of request.url.path. Sub-path
deployments are handled via scope["app_root_path"] / scope["root_path"],
mirroring Starlette's own base_url construction logic.
Affected variants confirmed fixed:
Host: localhost/?x=1
Host: localhost:4000/?x=1
Host: localhost/#test
Host: localhost:4000/#test
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* style: reduce comments in route fix
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: block credential fields in RAG ingest vector_store options
Credential fields (vertex_credentials, aws_access_key_id, api_key, etc.)
in ingest_options.vector_store are now rejected at the API boundary with
a 400 error. Credentials must be configured server-side.
Previously any authenticated user could supply a vertex_credentials dict
with type=external_account pointing credential_source.file at an
arbitrary path (e.g. /proc/1/environ) and token_url at an
attacker-controlled server. google-auth's identity_pool.Credentials
refresh() would read the file and POST its contents to the attacker.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: block /key/update self-escalation by assigned users
Non-admin users who were assigned a key (created_by != caller) could
update any non-budget field — models, rpm_limit, guardrails, etc. —
without admin authorization, allowing privilege self-escalation.
Gate: only the key creator (created_by == caller) may edit their own
key without admin check; budget changes always require admin regardless
of creator status. All other callers must pass _check_key_admin_access.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: block user-controlled api_base in RAG ingest vector_store options
A user-supplied api_base in ingest_options.vector_store caused the server
to forward its configured provider credentials (Gemini, OpenAI) to an
attacker-controlled endpoint via SSRF.
Add api_base to the blocked credential params set alongside api_key and
the existing credential fields.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: restrict /utils/transform_request to PROXY_ADMIN and apply body safety check
Any authenticated internal_user could POST arbitrary provider config
(aws_sts_endpoint, api_base, etc.) to /utils/transform_request and have
the server forward its credentials to an attacker-controlled endpoint.
- Gate the endpoint on PROXY_ADMIN role (403 for all other roles)
- Call is_request_body_safe() to reject banned params even for admins
- Convert ValueError from safety check to HTTP 400
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: apply banned-param check to /utils/transform_request
Without is_request_body_safe(), any authenticated user could pass
aws_sts_endpoint, api_base, or aws_web_identity_token to
/utils/transform_request and have the server forward its configured
provider credentials to an attacker-controlled endpoint during SDK
credential resolution.
Applies the same banned-param blocklist already used by LLM endpoints.
Endpoint remains accessible to all authenticated users.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: block SSRF via api_base in /prompts/test dotprompt YAML frontmatter
Any frontmatter key not in ["model","input","output"] flowed into
optional_params and was merged into the LLM call data dict, bypassing
is_request_body_safe. An attacker with any bearer key could set
api_base in YAML to redirect the outbound LLM request — including the
provider API key — to an attacker-controlled host.
Fix: call is_request_body_safe on the constructed data dict after
optional_params are merged, before invoking ProxyBaseLLMRequestProcessing.
ValueError from the banned-param check is surfaced as HTTP 400.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* Update litellm/proxy/rag_endpoints/endpoints.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix: coerce nested config strings before banned-param check
_NESTED_CONFIG_KEYS descent used isinstance(nested, dict) which silently
skipped litellm_embedding_config when delivered as a JSON string via
multipart/form-data. Banned params (api_base, aws_sts_endpoint, etc.)
nested inside the stringified value were invisible to is_request_body_safe.
_NESTED_METADATA_KEYS already used _coerce_metadata_to_dict which parses
JSON strings before checking. Apply the same coercion to _NESTED_CONFIG_KEYS.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: replace substring match with prefix match in is_llm_api_route
mapped_pass_through_routes used `_llm_passthrough_route in route` (substring)
so any admin-only path whose URL contained a provider name (openai, anthropic,
azure, bedrock, etc.) was misclassified as an LLM API route and bypassed the
admin gate in non_proxy_admin_allowed_routes_check.
Confirmed live: non-admin key could GET /credentials/by_name/openai (read
masked provider API key) and DELETE /credentials/openai (delete credential).
Fix: use exact match or startswith(prefix + "/") — the same pattern used
everywhere else in RouteChecks — so only routes that actually start with a
passthrough prefix are allowed through.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: stabilize PR #27878 test failures
- key_management_endpoints: extend can_skip_admin_check to team keys so
team members with /key/update permission can update non-budget fields.
can_team_member_execute_key_management_endpoint already validates team
membership + permission and raises if unauthorized; reaching the admin
check on a team key means the caller was authorized.
- test: set created_by on mock key in
test_update_key_non_budget_fields_allowed_for_internal_user so
caller_is_creator resolves correctly (MagicMock default ≠ user_id).
- auth_utils.get_request_route: guard against non-dict request.scope
(e.g. MagicMock in unit tests) to prevent a MagicMock leaking into
UserAPIKeyAuth.request_route and failing Pydantic validation.
- ci: assign test_multipart_bypass_repro.py to the proxy-runtime shard
in test-unit-proxy-db.yml to satisfy the shard-coverage check.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(lint): add explicit str() cast in get_request_route for MyPy
scope.get() returns Any|None which MyPy cannot coerce to str implicitly.
Wrap both scope.get() calls in str() to satisfy the type checker.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: guard bare-/ root_path strip + make total_spend migration idempotent
auth_utils.get_request_route: when Starlette sets scope["app_root_path"]
to "/" (e.g. behind some middleware), the old stripping logic would
remove the leading slash from every path ("/team/new" → "team/new"),
breaking route matching and causing auth to misclassify protected routes.
Skip stripping when root_path is bare "/".
migration: add IF NOT EXISTS to total_spend ALTER TABLE so the migration
is safe to replay when a prior partial run already created the column.
Without this guard, prisma migrate deploy fails on CI DBs that were
partially migrated, causing all subsequent DB operations (including
/team/new) to 500.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: require creator still owns key for personal-key bypass in /key/update
caller_is_creator now requires both created_by == caller AND user_id ==
caller. Previously checking only created_by let a demoted admin who
originally created a key for another user continue editing non-budget
fields on it after reassignment, bypassing _check_key_admin_access.
Adds regression test: creator whose key was reassigned is blocked (403).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: extract auth checks to fix PLR0915 + broaden max_budget assertion
internal_user_endpoints._update_single_user_helper exceeded 50 statements
(PLR0915). Extract authorization checks into _check_user_update_authz helper
to bring statement count under the limit.
test_validate_max_budget: assert "negative" (substring of both the local
"cannot be negative" and the CI "non-negative finite number" messages) so
the test is stable regardless of which exact wording the function uses.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
aws_sts_endpoint, aws_web_identity_token, and aws_bedrock_runtime_endpoint
in ingest_options.vector_store were passed directly to the Bedrock ingestion
class, which reads them into boto3 STS client construction. Any authenticated
caller could redirect AssumeRole calls to an attacker-controlled server,
leaking the proxy's instance profile credentials.
Calls is_request_body_safe() on ingest_options["vector_store"] before
forwarding to litellm.aingest(). Same banned-params list and admin opt-in
escape hatch (allow_client_side_credentials) as the /chat/completions path.
ValueError from the safety check is caught and re-raised as HTTP 400.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Authenticated clients could supply CustomPricingLiteLLMParams fields
(input_cost_per_token, output_cost_per_token, etc.) in the request body.
These were forwarded to register_model() in main.py, permanently mutating
the shared global litellm.model_cost dict for all users on the instance.
Adds all CustomPricingLiteLLMParams fields to _BANNED_REQUEST_BODY_PARAMS
so is_request_body_safe() rejects them before they reach completion().
New pricing fields added to CustomPricingLiteLLMParams are auto-covered.
Admin opt-in via allow_client_side_credentials or
configurable_clientside_auth_params still works as before.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two small follow-ups on the readiness/details migration:
- Set `retry: false` on the query. The payload feeds a passive
navbar tag and a debug banner; a 401 from an expired token
shouldn't fan out into three retries against the proxy.
- Add navbar specs that assert the `accessToken` prop is forwarded
into the hook (matches the DebugWarningBanner spec). Without
this, the navbar could silently regress to passing `undefined`
and the existing tests wouldn't catch it.
The proxy moved `litellm_version`, `is_detailed_debug`, and other
diagnostic fields off the public `/health/readiness` payload behind
an auth-gated `/health/readiness/details` endpoint. The navbar
version tag and the detailed-debug-mode banner stopped working
because they were still reading those fields from the unauthed
response, which no longer contains them.
Replace `useHealthReadiness` with a `useHealthReadinessDetails`
hook that takes an `accessToken` argument and sends a Bearer header
to the auth-gated endpoint. The hook stays disabled while
`accessToken` is falsy, so the navbar can keep rendering on the
public model hub (where the token is null) without triggering an
auth redirect or a 401-loop.
Backport of #27866 onto litellm_1.84.0rc2.
External readiness probes consumed the legacy detailed payload's `db`
field to drive alerting and pod-rotation decisions. Stripping the body
to {"status": "healthy"} broke those probes silently — the HTTP code
still flipped to 503, but probes checking body.db == "connected"
treated the response as healthy.
Add `db` back to the unauthenticated payload. The rest of the diagnostic
fields (litellm_version, callbacks, cache, log_level) stay behind
/health/readiness/details so the recon-leak gate from #26912 holds.
Values match the legacy contract: "connected", "disconnected",
"Not connected". The 503-on-DB-disconnect behavior from LIT-2607 is
preserved.
Backport of #27793 onto litellm_1.84.0rc2.
A non-admin caller could rebind their own key's user_id via /key/regenerate.
_execute_virtual_key_regeneration had org/team guards but no user_id guard,
and prepare_key_update_data did not strip the field — it survived
model_dump(exclude_unset=True) into the Prisma update. On the next request,
_return_user_api_key_auth_obj resolved the rebound user_id against
litellm_usertable and returned PROXY_ADMIN whenever the target row's
user_role was admin.
/key/update had the equivalent guard inline at _validate_update_key_data;
extract it to a shared helper _validate_caller_can_change_key_ownership and
call from both /key/update and _execute_virtual_key_regeneration.
Also tighten the premium gate that allowed the master-key rotation branch to
skip the enterprise check. The previous predicate was a field-presence test,
not an identity check. Verify the caller actually holds the master key via
_is_master_key before allowing the non-premium path.
Block explicit-null user_id and empty-string user_id as removal attempts;
both 403-reject for non-admin callers.
SERVER_ROOT_PATH is a process-startup env var. Read it once in
__init__ instead of calling get_server_root_path() + rstrip on every
request that arrives before all lazy features have loaded.
LazyFeatureMiddleware compared the raw scope path against registered
prefixes (e.g. /policies), so requests under a server root path like
/api/v1/policies/... never matched, the feature never loaded, and the
endpoint returned 404. Strip the configured root path before matching,
normalizing trailing slashes and enforcing a component boundary so
/api does not falsely match /apiv2.
Cherry-pick of #27762 onto litellm_1.84.0rc2.
* chore: reject bare str at file-input sinks to prevent local-file read (#27667)
* fix: use os.PathLike in ocr sink and check truthy reasoningSummary for bridge
- ocr/main.py: widen Path check to os.PathLike for consistency with other sinks
- main.py: bridge condition checks truthiness of reasoning_summary, not just None
* fix: remove unused pathlib.Path import in ocr/main.py
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
The tag-strip block was removed in the parent commit but two surrounding
comments still referenced "tags without opt-in" and "runs AFTER the
strip". Update them to describe the remaining user_api_key_* and
_pipeline_managed_guardrails strip that the snapshot/merge ordering
actually protects against.
Caller-supplied tags (`x-litellm-tags` header, body `tags`, `metadata.tags`)
were silently dropped unless the key/team had
`metadata.allow_client_tags: true` set. Restore the documented behavior:
tags from the request always flow into `metadata.tags` and union with any
admin-configured static tags from key/team/project metadata.
Removes the `allow_client_tags` opt-in flag from the pre-call pipeline.
The flag was only ever read here; it has no schema or endpoint footprint,
so leftover values in existing key metadata are inert.
Test cleanup mirrors the simplification: drop the three tests that
verified the strip-when-not-opted-in path, drop the `allow_client_tags`
fixture lines from the merge/union tests.
Match the existing MCP invariant in merge_mcp_headers and the managed MCP
path: operator-configured static headers always override caller-forwarded
headers on name conflict, with case-insensitive comparison so different
casing cannot bypass the precedence. _request_auth_header (BYOK) still
overrides Authorization last.
Addresses Veria review on PR #27383.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
