Remove parameters that may contain credentials from the messages built
inside broad except handlers. These messages can surface in HTTP error
responses, so caller-supplied secrets and integration tokens shouldn't
be interpolated into them.
Two cleanups from the /simplify review pass:
* ``Response`` was imported inside the ``except OSError`` branch in
``/get_image`` and at the top of ``/get_favicon``. Per the project's
no-inline-imports rule (CLAUDE.md), hoisted to the existing
``from fastapi.responses import (...)`` block at the top of
``proxy_server.py``.
* The test class's ``_patches()`` helper returned a 2-element list of
patch context managers and tests indexed into them via
``self._patches(...)[0], self._patches()[1]`` — two distinct calls
with confusing aliasing semantics. Restructured to:
- module-level ``_patch_async_safe_get(...)`` that returns a single
patch context manager
- autouse fixture that patches ``get_async_httpx_client`` for every
test in the file (it's the same patch in every case)
- small ``_image_response(...)`` factory to deduplicate Mock setup
Tests now read as ``with _patch_async_safe_get(return_value=...):``
with no list-indexing or duplicate Mock construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three review items addressed:
* **Veria (Medium): SSRF via redirect.** ``fetch_validated_image_bytes``
was calling ``validate_url(url)`` once and then fetching with the
default httpx client, so a 3xx to an internal IP would have been
followed unvalidated. Switched to ``async_safe_get`` (the existing
SSRF primitive used elsewhere in the codebase) which walks each
redirect hop, re-validates, and rejects redirects to blocked
networks. Default ``litellm.user_url_validation`` is True so
protection is on out of the box.
* **Greptile (P2): SVG can embed JS.** Removed ``image/svg+xml`` from
the allowed-Content-Type set. The hardcoded response media type
(``image/jpeg`` / ``image/x-icon``) means a real SVG body wouldn't
render as SVG anyway in modern browsers — the allowlist entry was
giving up XSS surface for no actual SVG-rendering benefit. If real
SVG support is wanted later, that's a deliberate feature PR with CSP
/ nosniff bundled.
* **Greptile (P2): cache-write OSError drops validated bytes.** When
the upstream fetch succeeded but ``open(cache_path, "wb")`` raised
(read-only assets dir), the bytes were discarded and the default
logo was served — a silent regression for that deployment. Now
serve the validated bytes inline via ``Response(...)`` as a fallback
before falling back to default.
Tests:
- Replaced low-level mocks of ``validate_url`` with mocks of
``async_safe_get`` directly, exercising the helper's contract
rather than the SSRF primitive's internals.
- New ``test_rejects_svg_content_type`` confirms SVG is blocked.
- ``test_get_image_cache_logic`` fixture now sets
``mock_response.is_redirect = False`` so ``async_safe_get`` doesn't
treat the Mock's truthy attribute as a redirect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Use key-in-dict membership instead of truthy value lookup so explicitly
supplied empty/falsy payloads still trigger the permission check.
Adds parametrized regression coverage across all gated keys.
Greptile flagged that ``disable_team_logging`` and ``get_team_callbacks``
re-wrap any ``HTTPException`` (including the 403 from the access guard)
through a catch-all that logs at ``.error()`` before re-raising — so
every legitimate access-denied response would pollute alerting
dashboards as a "server error".
Add explicit ``except HTTPException: raise`` and
``except ProxyException: raise`` branches before the catch-all (matching
the pattern already used in ``add_team_callbacks``). 4xx now propagates
quietly; only genuinely unexpected exceptions still hit the
error-level log.
Tests assert ``HTTPException`` is now the surfaced shape (instead of
the previous ``ProxyException`` re-wrap).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three CI failures from the previous push, all addressed:
* ``lint`` (mypy): ``async_client.get(url, **request_kwargs)`` confused
mypy because ``AsyncHTTPHandler.get``'s second positional arg is typed
``bool | None``. Switched to an explicit branch:
``await async_client.get(rewritten_url, headers={"host": host_header})``
for the HTTP-rewritten case, plain ``get(rewritten_url)`` otherwise.
* ``proxy-infra`` /
``test_get_image_custom_local_logo_bypasses_cache``: the existing
test set ``UI_LOGO_PATH=/app/custom_logo.jpg`` with no
``LITELLM_ASSETS_PATH``, asserting the path was served verbatim. That
was the LFI behaviour the new path-containment guard closes. Updated
the test to set ``LITELLM_ASSETS_PATH=/app`` so the path is inside an
allowed root, and patched the helper's ``realpath`` / ``isfile`` to
go along with the mocked filesystem. Test intent (bypass cache when
``UI_LOGO_PATH`` is local) is preserved.
* ``auth-and-jwt`` / ``test_get_image_cache_logic``: existing test
built a ``Mock`` response without ``headers``, so the new
Content-Type check tripped on ``Mock().split(";")[0]``. Two fixes:
1. Set ``mock_response.headers = {"content-type": "image/jpeg"}``
on the test (matches the real upstream contract — a logo CDN
always sets a Content-Type).
2. Make ``fetch_validated_image_bytes`` defensive: if the
Content-Type header is missing or non-string, treat as non-image
and fall back to default. Closes a subtle hole — pre-fix, an
upstream that omits Content-Type entirely would have served
arbitrary bytes under the ``image/jpeg`` wrapper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three endpoints in ``team_callback_endpoints.py`` accept a ``team_id``
from the URL but never check whether the authenticated caller can
manage that team:
* ``POST /team/{team_id}/callback`` — write Langfuse / Langsmith / GCS
credentials to any team
* ``POST /team/{team_id}/disable_logging`` — silence audit logging
for any team
* ``GET /team/{team_id}/callback`` — read back another team's stored
third-party API credentials
Each handler now runs the existing ``_verify_team_access`` helper
(proxy-admin / org-admin / team-admin hierarchy already used by sibling
endpoints in ``team_endpoints.py``) on the resolved team row before
the read or write.
Tests:
- ``test_add_team_callbacks_rejects_unauthorized_caller`` — internal
user not on the team gets 403; DB write never happens.
- ``test_disable_team_logging_rejects_unauthorized_caller`` — same.
- ``test_get_team_callbacks_rejects_unauthorized_caller`` — same on the
read path; victim team's callback data stays inaccessible.
- ``test_proxy_admin_can_add_team_callbacks`` — proxy admin still
passes through to the DB write (sanity that the guard didn't
over-rotate).
- ``test_team_admin_of_target_team_can_add_callbacks`` — team admin of
the target team still passes through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves merge conflict in tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
by keeping both the new bedrock tool-result file/document tests and the
transform_response body-leak regression test.
Also addresses Greptile P2 comment: when BedrockImageProcessor returns a
block with neither 'image' nor 'document' keys on the tool-result path
(image_url and file content types), log a warning instead of silently
dropping the block.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Align ``/get_favicon``'s allowed-root list with ``/get_image``'s. Both
endpoints now accept paths under any of:
* ``LITELLM_ASSETS_PATH`` (or its default — ``/var/lib/litellm/assets``
for non-root, the package dir otherwise)
* the package's bundled-asset dir (``proxy/_experimental/out`` for the
default favicon, ``proxy/`` for the default logo)
* the proxy package dir (``current_dir``) as a final fallback
Without this, an admin who put a custom favicon under
``LITELLM_ASSETS_PATH`` (e.g. mounted into the container at
``/var/lib/litellm/assets/favicon.ico``) would have the favicon
endpoint silently fall back to the default after the previous commit's
path-containment guard. The logo endpoint already accepted this root.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The unauthenticated ``/get_logo_url`` endpoint returned the
``UI_LOGO_PATH`` env var verbatim. For HTTP(S) URLs this is intended —
the dashboard loads the logo directly from a public/internal CDN. For
local filesystem paths it was an information disclosure: any caller
could fetch ``/get_logo_url`` and read admin-only filesystem details
like ``UI_LOGO_PATH=/etc/litellm/secret-config.json``.
Now the endpoint returns the URL only when it begins with
``http://`` or ``https://``. For local paths (or unset) it returns an
empty string — the dashboard falls back to ``/get_image`` which
serves the file via the path-containment guard added in the previous
commit.
Tests parametrize the disclosure-blocked cases (``/etc/...``,
``/proc/self/environ``, relative paths) and confirm HTTP / HTTPS URLs
still pass through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The unauthenticated ``/get_image`` and ``/get_favicon`` endpoints accept
the admin-set env vars ``UI_LOGO_PATH`` and ``LITELLM_FAVICON_URL`` and
return whatever bytes they resolve to, with a hard-coded ``image/jpeg``
or ``image/x-icon`` content-type. Two attack shapes:
* ``UI_LOGO_PATH=/etc/passwd`` (or any other readable file path) — any
unauthenticated caller exfiltrates the file via ``GET /get_image``.
The previous gate was ``os.path.exists(logo_path)`` which fires on
every readable file. Same shape for the favicon endpoint.
* ``UI_LOGO_PATH=http://169.254.169.254/iam`` (or any internal HTTP
service the admin pointed at) — the proxy fetches it server-side
and streams the response body to the unauthenticated caller. No
URL validation, no Content-Type validation; ``application/json``
AWS metadata gets tunneled out under the ``image/jpeg`` wrapper.
New helper module ``litellm/proxy/common_utils/static_asset_utils.py``:
* ``resolve_local_asset_path(candidate, allowed_roots)`` — returns the
resolved absolute path only if it lives within one of the allowed
asset roots. Uses ``realpath`` so symlinks pointing outside the roots
are caught.
* ``fetch_validated_image_bytes(url)`` — runs the URL through
``validate_url`` (rejecting private / cloud-metadata / loopback
targets) and only returns the response body if the upstream
Content-Type is in a small allowlist of image MIME types.
Both ``/get_image`` and ``/get_favicon`` are wired through the helpers.
The SSRF gate is enforced unconditionally — these endpoints are
unauthenticated, so the admin-facing ``litellm.user_url_validation``
toggle does not apply (an admin who opted out of URL validation for
LLM provider paths shouldn't also expose ``/get_image`` to SSRF).
Tests:
- ``TestResolveLocalAssetPath``: 10 cases covering legitimate paths,
``/etc/passwd``, ``/proc/self/environ``, symlink-out, ``..``
traversal, directories, missing files, and root list edge cases.
- ``TestFetchValidatedImageBytes``: 7 cases covering SSRF block, non-
image content-type rejection, valid image passthrough, non-200
response, fetch exception, empty URL, and parametrized coverage of
every allowed image MIME type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Remove unused search_provider parameter from
SearchAPIRouter._resolve_search_provider_credentials. The function
only reads tool_litellm_params; the docstring already omitted
search_provider, confirming it was unintentional dead code.
- Drop redundant hasAgents/hasSearchTools conditions from the outer
object_permission guard in OldTeams.tsx. Both agent and search-tool
handling already run independently below this block with their own
object_permission initialization, so including them in the outer
guard caused an empty object_permission to be created prematurely
and never populated within that block.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
- Proxy decoded `model` from the encoded batch_id but never passed it
to `litellm.aretrieve_batch`.
- Without `model` in kwargs, litellm cannot load `BedrockBatchesConfig`
and falls into the legacy provider switch, which 400s for bedrock.
- Fix: set `data["model"] = model_from_id` before the litellm call in
the SCENARIO 1 (encoded batch_id) branch.
- Also corrects the error string in
`_handle_retrieve_batch_providers_without_provider_config` (said
`'create_batch'` despite being raised from the retrieve path).
- Adds tests covering retrieve + file_content round-trip for bedrock-
encoded IDs.
AWS Bedrock pricing publishes a separate 1-hour prompt-cache write rate for
Claude 4.5 / 4.6 / 4.7 (1.6x the 5-minute rate). Without
`cache_creation_input_token_cost_above_1hr`, cost tracking for 1-hour-TTL
prompt caching on Bedrock falls back to the 5-minute rate and undercounts
spend by ~60%.
Adds the field to the spot-checked Global and US-region entries:
- anthropic.claude-opus-4-7 (Global $10.00 / MTok)
- anthropic.claude-opus-4-6-v1 (Global $10.00 / MTok)
- anthropic.claude-opus-4-5-... (Global $10.00 / MTok)
- anthropic.claude-sonnet-4-6 (Global $6.00 / MTok)
- anthropic.claude-sonnet-4-5-... (Global $6.00 / MTok regular,
$12.00 / MTok long-context >200K)
- anthropic.claude-haiku-4-5-... (Global $2.00 / MTok)
- global.anthropic.* mirrors of the above
- us.anthropic.* mirrors at the US +10% premium
Also updates the long-context (>200K) variants of Sonnet 4.5 with
`cache_creation_input_token_cost_above_1hr_above_200k_tokens`.
The mirrored entries in `litellm/model_prices_and_context_window_backup.json`
are updated in lockstep.
EU / AU / APAC / JP / us-gov regional variants are out of scope for this
change pending separate verification against AWS Bedrock pricing for those
regions.
Adds tests/test_litellm/test_bedrock_anthropic_1hr_cache_pricing.py to lock
in the expected values and the 1.6x ratio invariant.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Two issues from the previous push's review:
1. **Greptile P1**: ``get_vector_store_info`` had the same catch-all
``except Exception`` pattern as ``update_vector_store``, so the
HTTPException(403/404) raised by both the in-memory access check and
the new ``_fetch_and_authorize_vector_store`` helper was rewritten as
500. Mirror the ``except HTTPException: raise`` guard from
``update_vector_store``.
2. **code-quality CI** (``tests/code_coverage_tests/recursive_detector.py``)
flagged ``_redact_sensitive_litellm_params`` as an unallowlisted
recursive function. Match the convention of other allowlisted
helpers ("max depth set"): bound recursion at depth 10 (well above
any plausible nesting level for real ``litellm_params`` payloads),
return the redaction sentinel on overflow, and add the function
name to ``IGNORE_FUNCTIONS``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the previous commit:
1. **Veria — Medium**: ``litellm_params`` carries a nested
``litellm_embedding_config`` dict (auto-resolved from the model
registry on create / update) which itself holds ``api_key`` /
``aws_*`` / ``vertex_credentials``. The previous redactor only
inspected top-level keys, so the nested values passed through
unredacted. Recurse into nested dicts.
2. **Greptile — P2**: when ``litellm_params`` is a JSON-serialized
string (the in-memory registry occasionally stores it that way), the
previous redactor silently no-op'd via the ``isinstance(..., dict)``
guard and echoed the raw payload back. Now: parse, redact, re-serialize.
If the string is not valid JSON, replace it with the redaction
sentinel rather than echo it.
3. **mypy** flagged ``_redact_sensitive_litellm_params``'s
``Optional[Dict[str, Any]]`` signature as incompatible with the
``object``-typed call site. Widened to ``Any -> Any`` to reflect the
actual contract (the function now handles dict / str / None / other).
Also fixes a related test regression in
``test_remove_sensitive_info_from_deployment_with_excluded_keys``: the
``"credentials"`` plural addition to ``SensitiveDataMasker`` defaults
caused the first call (without ``excluded_keys``) to mutate the input
dict's ``litellm_credentials_name`` to a masked value. The second call
(with ``excluded_keys``) then saw the already-masked value rather than
the original. Construct fresh input for each call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/simplify pass:
- ``update_vector_store`` (newly added) and ``get_vector_store_info``'s
DB-fallback path duplicated the same shape: ``find_unique`` →
``model_dump`` → ``LiteLLM_ManagedVectorStore(**)`` →
``_check_vector_store_access`` → raise 404/403. Extract into
``_fetch_and_authorize_vector_store`` so the pattern lives in one
place; future endpoints that need the same gate get it via one call.
- The ``except HTTPException: raise`` guard added in the prior commit is
retained — the helper raises HTTPException(403/404) and the catch-all
``except Exception`` would otherwise rewrite them as 500.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two architectural extensions to the credential-redaction in the previous
commit:
1. ``/vector_store/update`` had two gaps:
- No per-store access control. Any authenticated principal that
passed the premium-feature gate could mutate *any* vector store,
including stores belonging to other teams.
- The response returned the full DB row including ``litellm_params``,
so the caller could read another team's persisted provider
credentials by submitting a no-op metadata change.
Mirror the access-control check ``/vector_store/info`` already
performs (``_check_vector_store_access`` against the existing row),
redact ``litellm_params`` in the response, and add an
``except HTTPException: raise`` guard so the 403/404 responses don't
get rewritten as 500 by the catch-all.
2. ``SensitiveDataMasker``'s default ``sensitive_patterns`` set used
segment-exact matching, so ``credential`` matched ``vertex_credential``
but not ``vertex_credentials`` (the actual Vertex field name). The
previous commit worked around this with a per-call extension; this
commit puts the plural in the upstream defaults so every caller
(Redis config dump, MCP debug headers, cache routes, ...) gets the
correct behavior. The local override in
``vector_store_endpoints/management_endpoints.py`` is removed.
Also updates ``test_excluded_keys_exact_match`` which relied on
``credentials`` *not* being a sensitive pattern to demonstrate
case-sensitive ``excluded_keys`` matching. The intent of the test
(case-sensitive match) is preserved; the assertion now reflects that
when ``excluded_keys`` fails to apply (wrong case), the field falls
through to standard pattern-based masking instead of being passed
through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/simplify pass:
- Remove the single-call-site ``_redact_vector_store`` wrapper. Inline
the two-line redaction at its only caller in ``list_vector_stores``;
``get_vector_store_info`` was already calling the inner helper directly.
- Inherit ``SensitiveDataMasker``'s default sensitive-key set instead of
duplicating the 12-element list, then add only the plural
``credentials`` extension. Won't drift if upstream defaults change.
- Trim the over-explained docstring on ``_redact_sensitive_litellm_params``
to a one-paragraph summary; the WHY (credential-leakage class) belongs
in the commit message, not in every consumer's IDE tooltip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``LiteLLM_ManagedVectorStore.litellm_params`` carries the upstream provider
credential — OpenAI ``api_key``, AWS ``aws_access_key_id`` /
``aws_secret_access_key``, GCP ``vertex_credentials``, etc. ``GET
/vector_store/list`` and ``POST /vector_store/info`` return these
verbatim to any authenticated principal. Because both routes are in
``openai_routes``, ``RouteChecks.is_llm_api_route`` short-circuits the
standard role gate, so even read-only users and narrowly-scoped keys can
read every stored credential.
Replace credential-bearing values with the ``REDACTED_BY_LITELM``
sentinel in both responses while preserving non-secret keys
(``api_base``, ``region``, ``model``, ``api_version``) so callers can
still see *which* upstream is configured. Detection reuses
``SensitiveDataMasker.is_sensitive_key`` with the default heuristics
plus the plural ``credentials`` pattern (covers Vertex's
``vertex_credentials`` field, which the singular ``credential`` pattern
misses on segment-exact matching).
Applied at:
- ``list_vector_stores`` (``GET /vector_store/list``,
``GET /v1/vector_store/list``)
- ``get_vector_store_info`` (``POST /vector_store/info``), both the
in-memory-registry path and the prisma-DB fallback
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI surfaced two issues from the previous commit:
1. ``general_settings`` and ``master_key`` were still imported at the top
of ``get_logging_payload`` but had no remaining users after the
master-key hash-detection blocks were removed. Drop the import.
2. ``tests/proxy_unit_tests/test_user_api_key_auth.py::test_x_litellm_api_key``
and ``tests/proxy_unit_tests/test_key_generate_prisma.py::test_master_key_hashing``
asserted ``valid_token.token == hash_token(master_key)`` — the
pre-alias behavior. The new contract is
``valid_token.token == LITELLM_PROXY_MASTER_KEY_ALIAS`` (and !=
``hash_token(master_key)``), since the master key (and its hash)
must not propagate to the verification-token column or any other
downstream consumer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related changes to how the master-key auth path interacts with
downstream consumers of UserAPIKeyAuth.api_key:
1. The master-key auth branch in user_api_key_auth.py now sets
`valid_token.api_key` to a stable alias
(`LITELLM_PROXY_MASTER_KEY_ALIAS = "litellm_proxy_master_key"`) instead
of the raw master key. Downstream consumers — spend logging,
Prometheus metrics, audit trails, rate limiting, cost tracking — now
receive the alias instead of the master key (which they would
previously hash and propagate). Neither the raw master key nor its
hash flows past the auth layer.
2. `_is_master_key` in spend_tracking_utils.py is reduced to a strict
raw-only constant-time comparison. The hashed form is no longer
considered equivalent.
Side effects:
- The two hash-detection blocks in `get_logging_payload` are removed.
They were re-detecting the master key per spend-log write to swap in
the alias; that detection happens once at the auth layer now.
- The `disable_adding_master_key_hash_to_db` general setting becomes a
no-op. Operators can remove it from their config; existing config is
still accepted.
- Operator dashboards that filter Prometheus metrics by the master-key
hash will need to switch to the `api_key="litellm_proxy_master_key"`
label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Store search tool allowlists only on object permissions, wire auth/management/UI flows to object_permission.search_tools, and remove legacy team-metadata search credential code and tests.
Made-with: Cursor
Greptile review on #26225 (P2): the docstring said "Called when disconnect()
fails", and the SIGTERM warning log read "after failed disconnect", but
both were stale — `_kill_engine_process` is now invoked on every routine
reconnect (via the unified `recreate_prisma_client` path), not as a
disconnect-failure recovery branch. The misleading wording would have
produced confusing log lines on every reconnect cycle in production.
Update the docstring to explain the actual reason (avoiding the blocking
`disconnect()` event-loop freeze) and reword the SIGTERM warning to "during
reconnect" so it matches reality.
No behavior change; logs only.
Greptile review on #26756 (P2): if `attempt_db_reconnect` itself raises
(e.g. lock cancellation, timer error, unexpected internal failure), the
original `httpx.ReadError` / transport error was lost — `failure_handler`
and `db_exceptions` alerts then logged the reconnect exception instead of
the actual DB transport problem, masking the root cause.
Wrap the reconnect call in a try/except. On reconnect failure, re-raise
the *original* `first_exc` and chain the reconnect error as `__cause__`
so it remains visible for debuggability without becoming the primary
exception observers see.
Adds `test_call_with_db_reconnect_retry_preserves_original_error_when_reconnect_raises`
asserting (a) the propagated exception is the original transport error
and (b) the reconnect exception is attached as `__cause__`.
