Flip the litellm_settings dict merge from {**incoming, **existing} to
{**existing, **incoming} so the caller's value for any pre-existing key
is what gets persisted. The previous direction silently no-op'd a
request like {"litellm_settings": {"drop_params": false}} when the DB
already held drop_params: true — the endpoint returned 200 OK but the
stored value never changed. router_settings (immediately below) had
been doing the right thing all along; this brings the two sections into
alignment.
success_callback semantics are unchanged: it is still always normalized
to lowercase, and still unioned with any existing list (callbacks are
additive — a caller sends the new entry, not the full set).
Adds a regression test (drop_params: True in DB, request flips to
False, expect persisted False with other keys preserved).
* chore(auth): validate clientside api_base against SSRF guard; clear admin secrets on base override
Two related issues with how the proxy handles client-supplied
``api_base`` / ``base_url`` overrides on chat-completion requests:
1. **SSRF gate bypass** — ``check_complete_credentials()`` returned
``True`` for any non-empty ``api_key``, allowing the
``is_request_body_safe`` ``banned_params`` loop to admit ``api_base``
/ ``base_url`` values that point at private (RFC 1918), loopback,
link-local, or cloud-metadata addresses. Now: when the gate sees a
client-supplied ``api_base`` / ``base_url``, it runs the URL through
``litellm_core_utils.url_utils.validate_url`` (DNS-resolves, blocks
internal/IMDS/LL networks, defends against rebinding). Rejection
raises with a clear message.
2. **Admin-config leak on base override** —
``get_dynamic_litellm_params`` only carried the three clientside keys
(``api_key``, ``api_base``, ``base_url``) from request to upstream
call. Other admin-configured fields on ``litellm_params`` —
``organization``, ``extra_body``, ``extra_headers``, ``api_version``,
``azure_ad_token``, AWS / Vertex creds, etc. — flowed through
unchanged. With base redirected to a client-controlled server, those
admin secrets were sent to the attacker. Now: when ``api_base`` /
``base_url`` is in ``request_kwargs``, drop those admin-config
fields from ``litellm_params`` unless the caller re-supplied them.
Tests cover the SSRF-target rejection per URL field, the admin-secret
clearing on base override, the don't-clear case when only ``api_key``
is overridden (BYOK pattern), and the don't-overwrite case when the
caller resupplies fields like ``organization`` themselves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(vertex-batches): wrap api_base GET in safe_get for defense-in-depth
The vertex batches status-poll fetches an attacker-influenceable
``api_base`` URL with a raw ``sync_handler.get()``. The proxy auth gate
already validates clientside ``api_base`` before reaching this sink, so
the proxy flow is covered. This adds the per-sink wrap so SDK callers
and any future code path that bypasses the proxy gate pick up the same
SSRF defense from ``url_utils.safe_get``.
Operators with a legitimate private Vertex base can either allowlist
the host via ``litellm.user_url_allowed_hosts`` or disable validation
with ``litellm.user_url_validation = False``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(auth): hoist url_utils import; derive admin-config field list from CredentialLiteLLMParams
/simplify pass:
- Move ``from litellm.litellm_core_utils.url_utils import SSRFError, validate_url``
to module top in ``proxy/auth/auth_utils.py``. CLAUDE.md prefers
module-level imports unless avoiding a circular dependency, and
there's no cycle here (``url_utils`` doesn't depend on ``proxy.auth``).
- Replace the hardcoded ``_ADMIN_CONFIG_FIELDS_TO_CLEAR_ON_BASE_OVERRIDE``
literal with ``_admin_config_fields_to_clear_on_base_override()`` that
derives the typed-field portion from
``CredentialLiteLLMParams.model_fields``. Adds three fields the
hardcoded list missed (``aws_bedrock_runtime_endpoint``,
``watsonx_region_name``, ``region_name``) and stays in sync as new
provider fields are declared on the model. The kwargs-only set
(``organization``, ``extra_body``, ``azure_ad_token``, ``aws_session_token``,
``aws_sts_endpoint``, ``aws_web_identity_token``, ``aws_role_name``, …)
remains explicit since those fields aren't on the typed model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): close field-echo bypass; gate URL check on toggle; cover async batch path
Three issues from review:
1. ``get_dynamic_litellm_params`` used ``if field not in request_kwargs:
pop`` to clear admin-set provider config when the caller redirected
``api_base``. A caller could *echo* any clear-list field name (with any
value, including an empty string) to skip the pop, leaving the admin's
value in ``litellm_params`` to be forwarded to the redirected upstream.
Fix: always pop, then write the caller's value back if they resupplied
the field.
2. ``check_complete_credentials`` called ``validate_url`` directly. That
helper doesn't itself consult ``litellm.user_url_validation``; the
toggle is honoured by ``safe_get`` / ``async_safe_get``. Mirror that
here so admins who explicitly disabled URL validation aren't blocked
at the proxy boundary.
3. ``VertexAIBatchesHandler._async_retrieve_batch`` still used a bare
``await client.get(api_base, ...)`` while the sync sibling was wrapped
in ``safe_get``. Wrap the async call in ``async_safe_get`` so SDK
callers on the async path get the same DNS-rebind / private /
cloud-metadata defenses as the sync path.
Tests:
- ``TestCheckCompleteCredentialsBlocksSSRF`` is now mock-only; an autouse
fixture flips the toggle on, ``validate_url`` is patched in the
parametrized blocking tests, and the positive path no longer makes a
real DNS call to api.openai.com.
- ``test_skips_url_validation_when_toggle_is_off`` documents the new
toggle-off behaviour and asserts ``validate_url`` is not called.
- ``test_caller_resupplied_value_overrides_admin_value_on_base_override``
replaces the prior test that asserted the buggy
preserve-admin-value-on-echo behaviour.
- ``test_field_echo_does_not_preserve_admin_value`` is a focused
regression test for the empty-string echo vector.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): close provider-confusion credential exfil; expand banned-params; cover OCI
Three additions on top of the entry-point URL gate so the cluster is
fully closed against caller-supplied ``api_base`` redirection:
1. ``get_llm_provider_logic.py`` matched registered openai-compatible
endpoints against ``api_base`` with an unanchored substring search
(``if endpoint in api_base:``). A caller could pass an api_base like
``https://attacker.com/api.groq.com/openai/v1`` to coerce the proxy
into reading ``GROQ_API_KEY`` from the environment and forwarding it
as a Bearer credential to the attacker's host. Replaced with parsed-
URL semantics (hostname exact-match plus segment-bounded path-prefix)
in a new ``_endpoint_matches_api_base`` helper.
2. ``is_request_body_safe`` rejects ``api_base`` / ``base_url`` /
``user_config`` / a handful of AWS / vertex fields, but the list
omitted three other endpoint-targeting fields:
* ``aws_bedrock_runtime_endpoint`` — Bedrock endpoint redirect
* ``langsmith_base_url`` / ``langfuse_host`` — observability callback
hostnames; attacker-controlled values exfiltrate the entire request
payload (incl. message content) via the logging hook.
Added all three to the blocklist.
3. ``_admin_config_fields_to_clear_on_base_override`` derives its typed-
field list from ``CredentialLiteLLMParams.model_fields``, which does
not declare any of the OCI provider's auth fields. Added
``oci_signer``, ``oci_user``, ``oci_fingerprint``, ``oci_tenancy``,
``oci_key``, and ``oci_key_file`` to the kwargs-only fixed list so
they are cleared on caller-redirected ``api_base`` like the AWS /
Azure / Vertex equivalents.
Tests:
- ``TestEndpointMatchesApiBase`` — direct unit tests on the new
matcher: legitimate provider URLs (5 shapes) match; attacker
smuggling via path injection, suffix label, prefix label, userinfo
``@`` injection, and path-segment lookalikes (7 shapes) do not.
- ``TestGetLlmProviderRejectsAttackerSmuggledApiBase`` — end-to-end
invariant that ``GROQ_API_KEY`` is never read against an attacker-
controlled host while the legitimate ``api.groq.com`` path still
resolves the provider correctly.
- ``TestIsRequestBodySafeBlocksEndpointTargetingFields`` — parametrized
coverage that each of the three new banned-params raises a clear
rejection naming the offending field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): remove implicit api-key bypass + add posthog/braintrust/slack to blocklist
The historical ``check_complete_credentials`` clause inside
``is_request_body_safe`` was a third, *implicit*, *caller-controlled*
BYOK path: any caller that supplied a non-empty ``api_key`` caused the
entire banned-params blocklist to be skipped. That turned every missing
entry on the blocklist into an exploitable SSRF / credential-exfil hole
and is the root cause of the chain of api_base advisories that have
been re-discovered with each new integration:
* GHSA-jh89-88fc-qrfp (critical, triage) — env-var exfil via api_base
* GHSA-3frq-6r6h-7j64 (high, triage) — admin org / extra_body leak
* veria-admin Dv_m860l, b_yRJeQ5, stN90yjP, LBlyOAc8, U2TD78kg —
variations on "list X is missing field Y"
Two explicit, admin-controlled BYOK paths already exist and remain:
``general_settings.allow_client_side_credentials = true`` (proxy-wide)
and ``configurable_clientside_auth_params: [...]`` per deployment.
Removing the implicit bypass converts the failure mode of a missing
blocklist entry from "live credential leak" to "predictable 400 with
a clear remediation message," which is the structural fix.
Also adds the three remaining endpoint-targeting fields the dynamic
callback layer reads from request body: ``posthog_host``,
``braintrust_host``, ``slack_webhook_url``. ``slack_webhook_url`` in
particular was a direct exfil channel (caller-set webhook → proxy
mirrors every request to attacker's Slack).
Tests:
- ``test_api_key_does_not_bypass_blocklist`` — parametrized regression
asserting api_key=anything no longer skips the gate for any of the
five highest-risk fields.
- ``test_admin_opt_in_proxy_wide_still_allows`` — confirms the
documented BYOK opt-in still works.
- Extends ``test_endpoint_targeting_field_in_request_body_is_rejected``
to cover posthog / braintrust / slack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): block sagemaker_base_url, s3_endpoint_url, deployment_url
Provider-specific endpoint overrides surfaced by a wider audit of
``optional_params`` consumers in ``litellm/llms/``. Same threat as
``api_base``: a caller-supplied value redirects the outbound request
to an attacker host.
* ``s3_endpoint_url`` — read in ``litellm/llms/bedrock/files/transformation.py``
to build the S3 upload URL for Bedrock files. Caller redirects file
uploads to attacker-controlled S3.
* ``sagemaker_base_url`` — read in ``litellm/llms/sagemaker/{chat,completion}/*``.
Caller redirects SageMaker traffic. This is the primary vector
described in veria-admin mNqEBBtG.
* ``deployment_url`` — popped in ``litellm/llms/sap/chat/transformation.py``.
Caller redirects SAP deployment requests.
Tests parametrize ``test_endpoint_targeting_field_in_request_body_is_rejected``
to cover the three new fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LiteLLM_MCPUserCredentials.credential_b64 stored both BYOK API keys and
OAuth2 access tokens as plain urlsafe-base64 of the raw value. Any DB
read could recover the upstream-provider key.
Run all writes through encrypt_value_helper (nacl SecretBox, the same
helper used for the server-level credentials column) and read back via
a small dual-path helper that tries decryption first, then falls back to
plain base64 so existing rows keep working until they get rewritten.
Folds the three near-identical "decode -> json.loads -> check type ==
oauth2" sites into _decode_oauth_payload, which simplifies the BYOK
guard inside store_user_oauth_credential.
* feat(schema): add workflow run tracking tables (LiteLLM_WorkflowRun, LiteLLM_WorkflowEvent, LiteLLM_WorkflowMessage)
* feat(proxy): add /v1/workflows/runs endpoints for durable agent workflow tracking
* feat(proxy): register workflow management router in proxy_server
* docs(workflows): add README for workflow run tracking API
* test(workflows): add unit tests for /v1/workflows/runs endpoints
* fix(workflows): atomic event+status update via tx(), run_id 404 guard, sequence retry on collision
* test(workflows): add tx mock, 404 on unknown run_id, retry-on-collision tests
* fix(workflows): constrain status to Literal enum, rename total→count in list responses
* add tenant isolation and bounded limits to workflow endpoints
* add created_by column and index to LiteLLM_WorkflowRun
* add ownership and bounded-limit tests for workflow endpoints
* Fix workflow run ownership for null owners
* guard prisma import in workflow_management_endpoints
* sync schema.prisma copies with workflow run models
* black: format workflow_management_endpoints.py
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Greptile flagged the unused ``from unittest.mock import patch``
left over from before the ``configure_proxy`` fixture refactor (the
fixture uses ``monkeypatch``, no ``patch`` calls remain). Also pruned
the now-stale "premium gate" paragraph from the module docstring
since that gate was removed in fbcfd59b1a.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Read user_id and team_id from the request's litellm_params metadata when
fabricating the UserAPIKeyAuth handed to the managed_files hook, so
batches created via passthrough are attributed to the real requester
instead of a hardcoded fallback. Adds parametrized regression coverage
for both the populated-metadata and empty-kwargs cases.
When a litellm_settings row already holds mixed-case names (e.g.
["Langfuse"]) — written by another code path or by hand — the
union-on-update path was running set([...]) over the raw existing list
plus the lowercase-normalized incoming list, so "Langfuse" and
"langfuse" survived as duplicates. delete_callback uses a lowercase
lookup, leaving the mixed-case entry unreachable.
Normalize the existing list with normalize_callback_names before the
union so the merged list converges to lowercase. Adds a regression test
covering the case where the DB starts with ["Langfuse", "SQS"] and the
caller submits ["langfuse"].
Pass-through endpoints configured in
``general_settings.pass_through_endpoints`` defaulted to ``auth: false``
and the safe ``auth: true`` setting was rejected at startup unless the
operator had a LiteLLM Enterprise license. Net result: OSS deployments
had **no safe configuration** — every pass-through admins added without
remembering ``auth: true`` shipped an unauthenticated forwarder, and
remembering ``auth: true`` raised a hard "enterprise-only" error.
Three changes:
* ``litellm/proxy/_types.py`` — flip
``PassThroughGenericEndpoint.auth`` default to ``True``. Operators
who add a pass-through with no explicit ``auth`` value now get a
safe, authenticated forwarder by default. Setting ``auth: false``
remains supported for genuine public-forwarder use cases (e.g.
webhook receivers).
* ``litellm/proxy/pass_through_endpoints/pass_through_endpoints.py``
— drop the ``premium_user`` gate around ``auth: true``. An
unauthenticated forwarder is a deployment choice operators should
be allowed to make explicitly, but the safe option must always be
free. The product-tier decision (which features sit behind the
enterprise license) is separate from "OSS users must always have a
safe option."
* ``litellm/proxy/auth/user_api_key_auth.py`` — the runtime dispatch
pulls pass-through endpoints from ``general_settings`` as raw
dicts, so the Pydantic default doesn't apply. Switched
``endpoint.get("auth")`` to ``endpoint.get("auth", True)`` so a
config dict without an explicit ``auth`` key still requires
authentication at request time.
Tests:
- ``test_passthrough_auth_defaults_to_true`` — Pydantic default is
now safe.
- ``test_passthrough_auth_can_still_be_explicitly_disabled``
— opt-in to ``auth=False`` still works for legitimate
public-forwarder use cases.
- ``test_register_passthrough_with_auth_true_works_for_oss``
— ``premium_user=False`` no longer rejects ``auth=true``.
- ``test_runtime_check_treats_missing_auth_key_as_authenticated``
— raw dict without an ``auth`` key now requires auth (the
previously-unauthenticated forwarder).
- ``test_runtime_check_explicit_auth_false_still_skips_validation``
— explicit opt-in still works.
Closes GHSA-7h34-mmrh-6g58.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged the ``premium_user is not True`` check as a hard
backwards-incompatible break for OSS users currently running
``enable_oauth2_proxy_auth=True``. They were right: unlike the
api_base case (where the docs already required admin opt-in), this
path was documented as available to OSS users. Adding the gate would
have closed a documented feature, not fixed a vuln.
Reframed the change:
* The **identity-only allowlist** (``ALLOWED_OAUTH2_PROXY_FIELDS`` =
``{user_id, user_email, team_id, team_alias, org_id, models}``) is
the actual security fix — it closes the privesc by rejecting any
mapping to a non-identity field at request time. This is unchanged.
* The **premium gate** was parity-with-siblings (a product decision,
not a security one). Removed. BerriAI can re-add it on their own
schedule with a proper deprecation cycle if they want enterprise-
only gating.
Tests: removed ``test_rejects_when_not_premium``; everything else
(allowlist enforcement, identity passthrough, attack-shape
regression) still passes — 14 tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant analysis on the team-callback IDOR (GHSA-xxv2-fprq-9x93)
surfaced the same shape on organization-scoped endpoints. Each takes
``organization_id`` from the request body, looks up the org, and
performs reads or writes — without checking whether the caller can
manage that org.
Affected endpoints:
* ``PATCH /organization/update`` — any authenticated key holder could
rewrite any org's metadata, budgets, and object permissions.
* ``POST /organization/member_add`` — docstring promises "Only
proxy_admin or org_admin allowed" but the code never enforced it;
any caller could add members to any org.
* ``POST /organization/member_update`` — only the
``modify-PROXY_ADMIN-target-only`` defense was in place; non-admin
members in any org could be re-roled by any caller.
* ``POST /organization/member_delete`` — no access check at all; any
caller could remove any user from any org.
Each handler now runs the existing ``_verify_org_access`` helper
(proxy-admin / org-admin hierarchy already used by ``GET
/organization/info`` and ``POST /organization/info``) before the read
or write.
Tests:
- ``test_organization_member_add_rejects_unauthorized_caller`` —
internal user not on the org gets 403; DB write never happens.
- ``test_organization_member_update_rejects_unauthorized_caller`` —
same.
- ``test_organization_member_delete_rejects_unauthorized_caller`` —
same.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged that the denylist was incomplete: ``user_max_budget``,
``user_tpm_limit``, ``user_rpm_limit``, and ``user_spend`` were not on
it. Inspection of the auth model showed dozens more privileged fields
across the ``LiteLLM_VerificationTokenView`` hierarchy (team / org /
end-user / region budget / spend / limit fields, plus
``allowed_model_region``, ``rpm_limit_per_model``, etc.) — a denylist
of "privileged fields" is unmaintainable here.
Inverted the model. ``ALLOWED_OAUTH2_PROXY_FIELDS`` is now an
identity-only allowlist: ``user_id``, ``user_email``, ``team_id``,
``team_alias``, ``org_id``, ``models``. Any mapping to a non-identity
field is rejected at request time. Default-secure: a future field
added to ``UserAPIKeyAuth`` is automatically blocked from
header-trust.
Use case for OAuth2-proxy auth is identity assertion from a trusted
upstream. Anything beyond that (privileges, budgets, rate limits) is
policy and should be authenticated with a signature, not a header —
operators who need this should switch to JWT auth.
Tests:
- ``test_refuses_to_map_non_identity_fields`` parametrized over 22
fields including all four ``user_*`` Greptile flagged, plus
team/org/end-user budget/limit fields, plus a fabricated field name
to confirm "anything not on the allowlist" is the rule.
- ``test_allowlist_is_identity_only`` locks in the allowlist's intent
so future additions of budget / role / permission entries are caught
in review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cleanups from the /simplify review pass:
* The header-mapping loop had a special-case ``if key == "max_budget":
auth_data[key] = float(value)`` branch. Since ``max_budget`` is now
in ``PRIVILEGED_OAUTH2_PROXY_FIELDS``, the denylist check rejects
the configuration before the loop runs — the float-conversion
branch is unreachable. Removed.
* Four tests independently called
``monkeypatch.setattr(proxy_server, "premium_user", ...)`` and
``monkeypatch.setattr(proxy_server, "general_settings", ...)`` with
almost-identical bodies. Replaced with a ``configure_proxy`` fixture
that yields a single callable —
``configure_proxy(premium=False)`` /
``configure_proxy(mappings={...})`` — so each test's setup is one
line. The previously-unused ``premium_proxy_settings`` fixture is
removed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``handle_oauth2_proxy_request`` reads HTTP request headers per the
admin-set ``oauth2_config_mappings`` and constructs a
``UserAPIKeyAuth`` from the values. Two failure modes:
1. **Premium parity.** Sibling auth paths
(``enable_oauth2_auth``, ``enable_jwt_auth``) require
``premium_user``; this path did not, so any open-source deployment
could turn the feature on without realising it requires a hardened
reverse-proxy topology. Added the ``premium_user`` gate.
2. **Privileged-field denylist.** Without a denylist, an admin who
maps the wrong header to ``user_role`` (or whose reverse proxy
leaks the header from upstream user input) lets any caller send
``X-User-Role: proxy_admin`` and gain full admin access — Pydantic
coerces the string into the ``LitellmUserRoles.PROXY_ADMIN`` enum.
Mapping any field in ``PRIVILEGED_OAUTH2_PROXY_FIELDS``
(``user_role``, ``api_key``, ``token``, ``permissions``,
``allowed_routes``, budget/limit fields, ``metadata``) raises at
request time so the misconfiguration surfaces loudly rather than
as a silent privesc.
Operators who genuinely need a trusted upstream to assert one of
these privileged fields should switch to JWT auth (signature-validated)
rather than header-trust.
Tests:
- ``test_returns_auth_for_simple_user_id_mapping``: legitimate
identity-only mapping still works.
- ``test_rejects_when_not_premium``: open-source deployments get a
clear enterprise-feature error.
- ``test_refuses_to_map_privileged_fields``: parametrized over every
entry in the denylist — each is rejected at request time.
- ``test_user_role_header_forgery_attack_is_blocked``: end-to-end
shape of the GHSA-5c3m-qffq-4r9m attack; rejected before auth
object construction.
- ``test_safe_fields_still_pass_through``: documented usage
(``user_id``, ``user_email``, ``team_id``, ``models``) is
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cleanups from the /simplify review pass:
* ``Response`` was imported inside the ``except OSError`` branch in
``/get_image`` and at the top of ``/get_favicon``. Per the project's
no-inline-imports rule (CLAUDE.md), hoisted to the existing
``from fastapi.responses import (...)`` block at the top of
``proxy_server.py``.
* The test class's ``_patches()`` helper returned a 2-element list of
patch context managers and tests indexed into them via
``self._patches(...)[0], self._patches()[1]`` — two distinct calls
with confusing aliasing semantics. Restructured to:
- module-level ``_patch_async_safe_get(...)`` that returns a single
patch context manager
- autouse fixture that patches ``get_async_httpx_client`` for every
test in the file (it's the same patch in every case)
- small ``_image_response(...)`` factory to deduplicate Mock setup
Tests now read as ``with _patch_async_safe_get(return_value=...):``
with no list-indexing or duplicate Mock construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three review items addressed:
* **Veria (Medium): SSRF via redirect.** ``fetch_validated_image_bytes``
was calling ``validate_url(url)`` once and then fetching with the
default httpx client, so a 3xx to an internal IP would have been
followed unvalidated. Switched to ``async_safe_get`` (the existing
SSRF primitive used elsewhere in the codebase) which walks each
redirect hop, re-validates, and rejects redirects to blocked
networks. Default ``litellm.user_url_validation`` is True so
protection is on out of the box.
* **Greptile (P2): SVG can embed JS.** Removed ``image/svg+xml`` from
the allowed-Content-Type set. The hardcoded response media type
(``image/jpeg`` / ``image/x-icon``) means a real SVG body wouldn't
render as SVG anyway in modern browsers — the allowlist entry was
giving up XSS surface for no actual SVG-rendering benefit. If real
SVG support is wanted later, that's a deliberate feature PR with CSP
/ nosniff bundled.
* **Greptile (P2): cache-write OSError drops validated bytes.** When
the upstream fetch succeeded but ``open(cache_path, "wb")`` raised
(read-only assets dir), the bytes were discarded and the default
logo was served — a silent regression for that deployment. Now
serve the validated bytes inline via ``Response(...)`` as a fallback
before falling back to default.
Tests:
- Replaced low-level mocks of ``validate_url`` with mocks of
``async_safe_get`` directly, exercising the helper's contract
rather than the SSRF primitive's internals.
- New ``test_rejects_svg_content_type`` confirms SVG is blocked.
- ``test_get_image_cache_logic`` fixture now sets
``mock_response.is_redirect = False`` so ``async_safe_get`` doesn't
treat the Mock's truthy attribute as a redirect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Use key-in-dict membership instead of truthy value lookup so explicitly
supplied empty/falsy payloads still trigger the permission check.
Adds parametrized regression coverage across all gated keys.
Greptile flagged that ``disable_team_logging`` and ``get_team_callbacks``
re-wrap any ``HTTPException`` (including the 403 from the access guard)
through a catch-all that logs at ``.error()`` before re-raising — so
every legitimate access-denied response would pollute alerting
dashboards as a "server error".
Add explicit ``except HTTPException: raise`` and
``except ProxyException: raise`` branches before the catch-all (matching
the pattern already used in ``add_team_callbacks``). 4xx now propagates
quietly; only genuinely unexpected exceptions still hit the
error-level log.
Tests assert ``HTTPException`` is now the surfaced shape (instead of
the previous ``ProxyException`` re-wrap).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three CI failures from the previous push, all addressed:
* ``lint`` (mypy): ``async_client.get(url, **request_kwargs)`` confused
mypy because ``AsyncHTTPHandler.get``'s second positional arg is typed
``bool | None``. Switched to an explicit branch:
``await async_client.get(rewritten_url, headers={"host": host_header})``
for the HTTP-rewritten case, plain ``get(rewritten_url)`` otherwise.
* ``proxy-infra`` /
``test_get_image_custom_local_logo_bypasses_cache``: the existing
test set ``UI_LOGO_PATH=/app/custom_logo.jpg`` with no
``LITELLM_ASSETS_PATH``, asserting the path was served verbatim. That
was the LFI behaviour the new path-containment guard closes. Updated
the test to set ``LITELLM_ASSETS_PATH=/app`` so the path is inside an
allowed root, and patched the helper's ``realpath`` / ``isfile`` to
go along with the mocked filesystem. Test intent (bypass cache when
``UI_LOGO_PATH`` is local) is preserved.
* ``auth-and-jwt`` / ``test_get_image_cache_logic``: existing test
built a ``Mock`` response without ``headers``, so the new
Content-Type check tripped on ``Mock().split(";")[0]``. Two fixes:
1. Set ``mock_response.headers = {"content-type": "image/jpeg"}``
on the test (matches the real upstream contract — a logo CDN
always sets a Content-Type).
2. Make ``fetch_validated_image_bytes`` defensive: if the
Content-Type header is missing or non-string, treat as non-image
and fall back to default. Closes a subtle hole — pre-fix, an
upstream that omits Content-Type entirely would have served
arbitrary bytes under the ``image/jpeg`` wrapper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three endpoints in ``team_callback_endpoints.py`` accept a ``team_id``
from the URL but never check whether the authenticated caller can
manage that team:
* ``POST /team/{team_id}/callback`` — write Langfuse / Langsmith / GCS
credentials to any team
* ``POST /team/{team_id}/disable_logging`` — silence audit logging
for any team
* ``GET /team/{team_id}/callback`` — read back another team's stored
third-party API credentials
Each handler now runs the existing ``_verify_team_access`` helper
(proxy-admin / org-admin / team-admin hierarchy already used by sibling
endpoints in ``team_endpoints.py``) on the resolved team row before
the read or write.
Tests:
- ``test_add_team_callbacks_rejects_unauthorized_caller`` — internal
user not on the team gets 403; DB write never happens.
- ``test_disable_team_logging_rejects_unauthorized_caller`` — same.
- ``test_get_team_callbacks_rejects_unauthorized_caller`` — same on the
read path; victim team's callback data stays inaccessible.
- ``test_proxy_admin_can_add_team_callbacks`` — proxy admin still
passes through to the DB write (sanity that the guard didn't
over-rotate).
- ``test_team_admin_of_target_team_can_add_callbacks`` — team admin of
the target team still passes through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves merge conflict in tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
by keeping both the new bedrock tool-result file/document tests and the
transform_response body-leak regression test.
Also addresses Greptile P2 comment: when BedrockImageProcessor returns a
block with neither 'image' nor 'document' keys on the tool-result path
(image_url and file content types), log a warning instead of silently
dropping the block.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The unauthenticated ``/get_logo_url`` endpoint returned the
``UI_LOGO_PATH`` env var verbatim. For HTTP(S) URLs this is intended —
the dashboard loads the logo directly from a public/internal CDN. For
local filesystem paths it was an information disclosure: any caller
could fetch ``/get_logo_url`` and read admin-only filesystem details
like ``UI_LOGO_PATH=/etc/litellm/secret-config.json``.
Now the endpoint returns the URL only when it begins with
``http://`` or ``https://``. For local paths (or unset) it returns an
empty string — the dashboard falls back to ``/get_image`` which
serves the file via the path-containment guard added in the previous
commit.
Tests parametrize the disclosure-blocked cases (``/etc/...``,
``/proc/self/environ``, relative paths) and confirm HTTP / HTTPS URLs
still pass through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The unauthenticated ``/get_image`` and ``/get_favicon`` endpoints accept
the admin-set env vars ``UI_LOGO_PATH`` and ``LITELLM_FAVICON_URL`` and
return whatever bytes they resolve to, with a hard-coded ``image/jpeg``
or ``image/x-icon`` content-type. Two attack shapes:
* ``UI_LOGO_PATH=/etc/passwd`` (or any other readable file path) — any
unauthenticated caller exfiltrates the file via ``GET /get_image``.
The previous gate was ``os.path.exists(logo_path)`` which fires on
every readable file. Same shape for the favicon endpoint.
* ``UI_LOGO_PATH=http://169.254.169.254/iam`` (or any internal HTTP
service the admin pointed at) — the proxy fetches it server-side
and streams the response body to the unauthenticated caller. No
URL validation, no Content-Type validation; ``application/json``
AWS metadata gets tunneled out under the ``image/jpeg`` wrapper.
New helper module ``litellm/proxy/common_utils/static_asset_utils.py``:
* ``resolve_local_asset_path(candidate, allowed_roots)`` — returns the
resolved absolute path only if it lives within one of the allowed
asset roots. Uses ``realpath`` so symlinks pointing outside the roots
are caught.
* ``fetch_validated_image_bytes(url)`` — runs the URL through
``validate_url`` (rejecting private / cloud-metadata / loopback
targets) and only returns the response body if the upstream
Content-Type is in a small allowlist of image MIME types.
Both ``/get_image`` and ``/get_favicon`` are wired through the helpers.
The SSRF gate is enforced unconditionally — these endpoints are
unauthenticated, so the admin-facing ``litellm.user_url_validation``
toggle does not apply (an admin who opted out of URL validation for
LLM provider paths shouldn't also expose ``/get_image`` to SSRF).
Tests:
- ``TestResolveLocalAssetPath``: 10 cases covering legitimate paths,
``/etc/passwd``, ``/proc/self/environ``, symlink-out, ``..``
traversal, directories, missing files, and root list edge cases.
- ``TestFetchValidatedImageBytes``: 7 cases covering SSRF block, non-
image content-type rejection, valid image passthrough, non-200
response, fetch exception, empty URL, and parametrized coverage of
every allowed image MIME type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Proxy decoded `model` from the encoded batch_id but never passed it
to `litellm.aretrieve_batch`.
- Without `model` in kwargs, litellm cannot load `BedrockBatchesConfig`
and falls into the legacy provider switch, which 400s for bedrock.
- Fix: set `data["model"] = model_from_id` before the litellm call in
the SCENARIO 1 (encoded batch_id) branch.
- Also corrects the error string in
`_handle_retrieve_batch_providers_without_provider_config` (said
`'create_batch'` despite being raised from the retrieve path).
- Adds tests covering retrieve + file_content round-trip for bedrock-
encoded IDs.
AWS Bedrock pricing publishes a separate 1-hour prompt-cache write rate for
Claude 4.5 / 4.6 / 4.7 (1.6x the 5-minute rate). Without
`cache_creation_input_token_cost_above_1hr`, cost tracking for 1-hour-TTL
prompt caching on Bedrock falls back to the 5-minute rate and undercounts
spend by ~60%.
Adds the field to the spot-checked Global and US-region entries:
- anthropic.claude-opus-4-7 (Global $10.00 / MTok)
- anthropic.claude-opus-4-6-v1 (Global $10.00 / MTok)
- anthropic.claude-opus-4-5-... (Global $10.00 / MTok)
- anthropic.claude-sonnet-4-6 (Global $6.00 / MTok)
- anthropic.claude-sonnet-4-5-... (Global $6.00 / MTok regular,
$12.00 / MTok long-context >200K)
- anthropic.claude-haiku-4-5-... (Global $2.00 / MTok)
- global.anthropic.* mirrors of the above
- us.anthropic.* mirrors at the US +10% premium
Also updates the long-context (>200K) variants of Sonnet 4.5 with
`cache_creation_input_token_cost_above_1hr_above_200k_tokens`.
The mirrored entries in `litellm/model_prices_and_context_window_backup.json`
are updated in lockstep.
EU / AU / APAC / JP / us-gov regional variants are out of scope for this
change pending separate verification against AWS Bedrock pricing for those
regions.
Adds tests/test_litellm/test_bedrock_anthropic_1hr_cache_pricing.py to lock
in the expected values and the 1.6x ratio invariant.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Two issues from the previous push's review:
1. **Greptile P1**: ``get_vector_store_info`` had the same catch-all
``except Exception`` pattern as ``update_vector_store``, so the
HTTPException(403/404) raised by both the in-memory access check and
the new ``_fetch_and_authorize_vector_store`` helper was rewritten as
500. Mirror the ``except HTTPException: raise`` guard from
``update_vector_store``.
2. **code-quality CI** (``tests/code_coverage_tests/recursive_detector.py``)
flagged ``_redact_sensitive_litellm_params`` as an unallowlisted
recursive function. Match the convention of other allowlisted
helpers ("max depth set"): bound recursion at depth 10 (well above
any plausible nesting level for real ``litellm_params`` payloads),
return the redaction sentinel on overflow, and add the function
name to ``IGNORE_FUNCTIONS``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the previous commit:
1. **Veria — Medium**: ``litellm_params`` carries a nested
``litellm_embedding_config`` dict (auto-resolved from the model
registry on create / update) which itself holds ``api_key`` /
``aws_*`` / ``vertex_credentials``. The previous redactor only
inspected top-level keys, so the nested values passed through
unredacted. Recurse into nested dicts.
2. **Greptile — P2**: when ``litellm_params`` is a JSON-serialized
string (the in-memory registry occasionally stores it that way), the
previous redactor silently no-op'd via the ``isinstance(..., dict)``
guard and echoed the raw payload back. Now: parse, redact, re-serialize.
If the string is not valid JSON, replace it with the redaction
sentinel rather than echo it.
3. **mypy** flagged ``_redact_sensitive_litellm_params``'s
``Optional[Dict[str, Any]]`` signature as incompatible with the
``object``-typed call site. Widened to ``Any -> Any`` to reflect the
actual contract (the function now handles dict / str / None / other).
Also fixes a related test regression in
``test_remove_sensitive_info_from_deployment_with_excluded_keys``: the
``"credentials"`` plural addition to ``SensitiveDataMasker`` defaults
caused the first call (without ``excluded_keys``) to mutate the input
dict's ``litellm_credentials_name`` to a masked value. The second call
(with ``excluded_keys``) then saw the already-masked value rather than
the original. Construct fresh input for each call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two architectural extensions to the credential-redaction in the previous
commit:
1. ``/vector_store/update`` had two gaps:
- No per-store access control. Any authenticated principal that
passed the premium-feature gate could mutate *any* vector store,
including stores belonging to other teams.
- The response returned the full DB row including ``litellm_params``,
so the caller could read another team's persisted provider
credentials by submitting a no-op metadata change.
Mirror the access-control check ``/vector_store/info`` already
performs (``_check_vector_store_access`` against the existing row),
redact ``litellm_params`` in the response, and add an
``except HTTPException: raise`` guard so the 403/404 responses don't
get rewritten as 500 by the catch-all.
2. ``SensitiveDataMasker``'s default ``sensitive_patterns`` set used
segment-exact matching, so ``credential`` matched ``vertex_credential``
but not ``vertex_credentials`` (the actual Vertex field name). The
previous commit worked around this with a per-call extension; this
commit puts the plural in the upstream defaults so every caller
(Redis config dump, MCP debug headers, cache routes, ...) gets the
correct behavior. The local override in
``vector_store_endpoints/management_endpoints.py`` is removed.
Also updates ``test_excluded_keys_exact_match`` which relied on
``credentials`` *not* being a sensitive pattern to demonstrate
case-sensitive ``excluded_keys`` matching. The intent of the test
(case-sensitive match) is preserved; the assertion now reflects that
when ``excluded_keys`` fails to apply (wrong case), the field falls
through to standard pattern-based masking instead of being passed
through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/simplify pass:
- Remove the single-call-site ``_redact_vector_store`` wrapper. Inline
the two-line redaction at its only caller in ``list_vector_stores``;
``get_vector_store_info`` was already calling the inner helper directly.
- Inherit ``SensitiveDataMasker``'s default sensitive-key set instead of
duplicating the 12-element list, then add only the plural
``credentials`` extension. Won't drift if upstream defaults change.
- Trim the over-explained docstring on ``_redact_sensitive_litellm_params``
to a one-paragraph summary; the WHY (credential-leakage class) belongs
in the commit message, not in every consumer's IDE tooltip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``LiteLLM_ManagedVectorStore.litellm_params`` carries the upstream provider
credential — OpenAI ``api_key``, AWS ``aws_access_key_id`` /
``aws_secret_access_key``, GCP ``vertex_credentials``, etc. ``GET
/vector_store/list`` and ``POST /vector_store/info`` return these
verbatim to any authenticated principal. Because both routes are in
``openai_routes``, ``RouteChecks.is_llm_api_route`` short-circuits the
standard role gate, so even read-only users and narrowly-scoped keys can
read every stored credential.
Replace credential-bearing values with the ``REDACTED_BY_LITELM``
sentinel in both responses while preserving non-secret keys
(``api_base``, ``region``, ``model``, ``api_version``) so callers can
still see *which* upstream is configured. Detection reuses
``SensitiveDataMasker.is_sensitive_key`` with the default heuristics
plus the plural ``credentials`` pattern (covers Vertex's
``vertex_credentials`` field, which the singular ``credential`` pattern
misses on segment-exact matching).
Applied at:
- ``list_vector_stores`` (``GET /vector_store/list``,
``GET /v1/vector_store/list``)
- ``get_vector_store_info`` (``POST /vector_store/info``), both the
in-memory-registry path and the prisma-DB fallback
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI surfaced two issues from the previous commit:
1. ``general_settings`` and ``master_key`` were still imported at the top
of ``get_logging_payload`` but had no remaining users after the
master-key hash-detection blocks were removed. Drop the import.
2. ``tests/proxy_unit_tests/test_user_api_key_auth.py::test_x_litellm_api_key``
and ``tests/proxy_unit_tests/test_key_generate_prisma.py::test_master_key_hashing``
asserted ``valid_token.token == hash_token(master_key)`` — the
pre-alias behavior. The new contract is
``valid_token.token == LITELLM_PROXY_MASTER_KEY_ALIAS`` (and !=
``hash_token(master_key)``), since the master key (and its hash)
must not propagate to the verification-token column or any other
downstream consumer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related changes to how the master-key auth path interacts with
downstream consumers of UserAPIKeyAuth.api_key:
1. The master-key auth branch in user_api_key_auth.py now sets
`valid_token.api_key` to a stable alias
(`LITELLM_PROXY_MASTER_KEY_ALIAS = "litellm_proxy_master_key"`) instead
of the raw master key. Downstream consumers — spend logging,
Prometheus metrics, audit trails, rate limiting, cost tracking — now
receive the alias instead of the master key (which they would
previously hash and propagate). Neither the raw master key nor its
hash flows past the auth layer.
2. `_is_master_key` in spend_tracking_utils.py is reduced to a strict
raw-only constant-time comparison. The hashed form is no longer
considered equivalent.
Side effects:
- The two hash-detection blocks in `get_logging_payload` are removed.
They were re-detecting the master key per spend-log write to swap in
the alias; that detection happens once at the auth layer now.
- The `disable_adding_master_key_hash_to_db` general setting becomes a
no-op. Operators can remove it from their config; existing config is
still accepted.
- Operator dashboards that filter Prometheus metrics by the master-key
hash will need to switch to the `api_key="litellm_proxy_master_key"`
label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
