feat(guardrails): scan an image on /apply_guardrail with a Bedrock guardrail (#51016)

* fix(guardrails): record the guardrail verdict on a request dict that starts empty

The Bedrock guardrail and the shared `log_guardrail_information` wrapper both picked the request dict with `or {}`. An empty dict is falsy, so a caller that passed one had the verdict written to a throwaway dict. /apply_guardrail passes an empty dict for a request that carries no messages and no metadata, so nothing downstream could see what the guardrail decided. The verdict now goes to the dict the caller passed, and only a missing dict falls back to a new one.

* fix(guardrails): log requests that a guardrail blocks on /apply_guardrail

A request a guardrail blocked on /apply_guardrail was missing from the logs. The endpoint skipped the failure handlers for an HTTPException, the spend log failure hook wrote no row because the route is not an LLM API route, and the guardrail's verdict stayed on a request dict no logger reads.

The endpoint now runs the failure handlers for every exception and copies the verdict onto the logged request, so allowed and blocked requests both carry it along with the guardrail's cost. It marks the failure of a request it had set up with a sentinel, and the spend log failure hook writes its row for that mark. A request rejected before setup, such as one with a bad key or an unknown guardrail, is still not logged. Guardrail records a caller sends in metadata are dropped for both response formats so a verdict cannot be forged in the logs.

* feat(guardrails): scan an image on /apply_guardrail with a Bedrock guardrail

/apply_guardrail only took text. It now takes `images`, one base64 data URI per request, and `text` becomes optional. The image is validated with the attachment reader the Bedrock guardrail already uses for chat completions and sent to ApplyGuardrail through the same call, with source OUTPUT for a response-direction scan. The image is scanned first, and the text scan is skipped only when the request has neither text nor messages.

A request the guardrail cannot judge answers 422 instead of 200 or 400. That covers an unreadable image, a guardrail that is not a Bedrock ApplyGuardrail guardrail, and an answer in which Bedrock does not report the image as guarded.

* test(guardrails): run the image tests through the endpoint's real request setup

The endpoint test helper on main now goes through the real request setup and guardrail lookup, and expects each request to leave one log entry. A body the request model refuses never reaches the endpoint, so those two cases are checked on the model. The guardrail that cannot scan images now carries the name the request asks for.

* fix(guardrails): log an image Bedrock did not evaluate as not run

An ApplyGuardrail image call that Bedrock answered without guarding the image
was logged as a guardrail success, so the request log read as a pass for an
image nobody checked. It now logs as not_run. A Bedrock error inside a 200
body on /apply_guardrail answers 502 naming the AWS error type instead of the
Image modality message

* fix(guardrails): refuse message text a Bedrock guardrail would not scan

* fix(guardrails): answer 422 when Bedrock cannot evaluate an image on /apply_guardrail

* refactor(guardrails): return image scan failures as values and type the test override

* fix(guardrails): keep Bedrock API errors reporting as HTTPException and log image-only prompts

* fix(guardrails): keep the scanned image in the apply_guardrail spend log request

* fix(guardrails): redact the apply_guardrail image when message logging is off and keep Bedrock API errors a plain HTTPException

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
This commit is contained in:
Agung Besti 2026-11-18 01:13:28 +07:00 • committed by GitHub
parent c38b3c3747
commit d7ab7255e2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 676 additions and 38 deletions

View file

@ -504,7 +504,7 @@ def _redact_responses_api_output_dict(output_items: Sequence[object], redacted_s
return [_redacted_responses_api_output_item(item, redacted_str) for item in output_items]
_REDACTED_REQUEST_FIELDS: Final = ("instructions", "query_image", *RETRIEVAL_QUERY_PARAMS)
_REDACTED_REQUEST_FIELDS: Final = ("instructions", "query_image", "images", *RETRIEVAL_QUERY_PARAMS)
_NESTED_REQUEST_FIELDS: Final = ("extra_body", "additional_args", "complete_input_dict")

View file

@ -16338,6 +16338,22 @@
"title": "Guardrail Name",
"type": "string"
},
"images": {
"anyOf": [
{
"items": {
"type": "string"
},
"maxItems": 1,
"type": "array"
},
{
"type": "null"
}
],
"description": "One image to scan, as a base64 data URI. Only Bedrock guardrails scan images.",
"title": "Images"
},
"input_type": {
"default": "request",
"title": "Input Type",
@ -16386,13 +16402,13 @@
"default": "standard"
},
"text": {
"default": "",
"title": "Text",
"type": "string"
}
},
"required": [
"guardrail_name",
"text"
"guardrail_name"
],
"title": "ApplyGuardrailRequest",
"type": "object"

View file

@ -0,0 +1,101 @@
"""Image scanning for the standalone guardrail endpoint."""
from collections.abc import Sequence
from dataclasses import dataclass
from typing import Final, Literal, NoReturn, TypeAlias
from fastapi import HTTPException
from typing_extensions import assert_never
from litellm.integrations.custom_guardrail import CustomGuardrail
from litellm.llms.bedrock.guardrail_attachments import find_request_attachments
from litellm.proxy.guardrails.guardrail_hooks.bedrock_guardrails import (
BedrockGuardrail,
bedrock_error_type,
overrides_bedrock_call,
)
from litellm.types.guardrails import GuardrailEventHooks
@dataclass(frozen=True, slots=True)
class ImageNotScanned:
detail: str
@dataclass(frozen=True, slots=True)
class GuardrailErrorBody:
detail: str
ImageScanFailure: TypeAlias = ImageNotScanned | GuardrailErrorBody
async def scan_images(
guardrail: CustomGuardrail,
images: Sequence[str],
request_data: dict[str, object], # mutable-ok: the guardrail records its verdict on the request dict
input_type: Literal["request", "response"],
) -> None:
"""Raise unless the guardrail evaluated every image and blocked none of them."""
failure: Final = await _image_scan_failure(guardrail, images, request_data, input_type)
if failure is not None:
_raise_public(failure)
async def _image_scan_failure(
guardrail: CustomGuardrail,
images: Sequence[str],
request_data: dict[str, object], # mutable-ok: the guardrail records its verdict on the request dict
input_type: Literal["request", "response"],
) -> ImageScanFailure | None:
name: Final = guardrail.guardrail_name
if not isinstance(guardrail, BedrockGuardrail) or guardrail.checks is not None or overrides_bedrock_call(guardrail):
return ImageNotScanned(f"Guardrail '{name}' does not support image scanning.")
message: Final = {
"role": "user",
"content": [{"type": "image_url", "image_url": {"url": image}} for image in images],
}
attachments: Final = find_request_attachments(
{"messages": [message]}, "acompletion", skip_tool_messages=False, latest_user_message_only=False
)
if attachments.unscannable:
return ImageNotScanned(
f"Guardrail '{name}' cannot scan this image: {', '.join(sorted(frozenset(attachments.unscannable)))}. "
"Send it inline as a base64 PNG or JPEG data URL under 4 MB."
)
is_response: Final = input_type == "response"
try:
answer: Final = await guardrail.async_scan_images(
attachments.images,
request_data=request_data,
source="OUTPUT" if is_response else "INPUT",
event_type=GuardrailEventHooks.post_call if is_response else GuardrailEventHooks.pre_call,
)
except HTTPException as error:
if error.status_code != 400 or isinstance(error.detail, dict):
raise
return ImageNotScanned(f"Guardrail '{name}' could not evaluate the image: {error.detail}")
error_type: Final = bedrock_error_type(answer)
if error_type is not None:
return GuardrailErrorBody(f"Guardrail '{name}' returned {error_type} instead of a verdict on the image.")
coverage: Final = answer.get("guardrailCoverage") or {}
if coverage.get("images", {}).get("guarded", 0) < len(images):
return ImageNotScanned(
f"Guardrail '{name}' did not report evaluating the image. "
"Bedrock evaluates images only for content filters that have the Image modality enabled."
)
return None
def _raise_public(failure: ImageScanFailure) -> NoReturn:
match failure:
case ImageNotScanned(detail=detail):
raise HTTPException(status_code=422, detail=detail)
case GuardrailErrorBody(detail=detail):
raise HTTPException(status_code=502, detail=detail)
case _:
assert_never(failure)
def ignores_messages(guardrail: CustomGuardrail) -> bool:
return isinstance(guardrail, BedrockGuardrail) and not guardrail.experimental_use_latest_role_message_only

View file

@ -10,7 +10,7 @@ import os
from collections.abc import Awaitable, Callable, Mapping, Sequence
from datetime import datetime, timezone
from types import MappingProxyType, UnionType
from typing import TYPE_CHECKING, Any, Final, Protocol, TypeVar, Union, cast, get_args, get_origin
from typing import TYPE_CHECKING, Any, Final, Literal, Protocol, TypeVar, Union, cast, get_args, get_origin
from urllib.parse import urlparse
from fastapi import APIRouter, Depends, HTTPException, Request
@ -32,6 +32,7 @@ from litellm.proxy.common_utils.callback_utils import CALLBACK_VAR_ENCRYPTED_PRE
from litellm.proxy.common_utils.openai_error_payload import status_typed_proxy_exception
from litellm.proxy.common_utils.path_utils import is_within, safe_join
from litellm.proxy.guardrails.content_filter_data import CATEGORIES_DIR, DATA_ROOTS, category_dirs, find_category_file
from litellm.proxy.guardrails.endpoint_image_scan import ignores_messages, scan_images
from litellm.proxy.guardrails.guardrail_hooks.custom_code.bounded_execution import (
ExecutionTimeoutError,
await_with_timeout,
@ -86,9 +87,15 @@ from litellm.types.guardrails import (
with_tolerated_stream_scope,
)
from litellm.types.llms.base import LiteLLMBaseModel
from litellm.types.llms.openai import (
ChatCompletionImageObject,
ChatCompletionTextObject,
ChatCompletionUserMessage,
)
from litellm.types.proxy.guardrails.guardrail_hooks.hide_secrets import (
HideSecretsGuardrailConfigModel,
)
from litellm.types.utils import GenericGuardrailAPIInputs
if TYPE_CHECKING:
from types import CodeType
@ -2510,9 +2517,19 @@ def _patch_logging_obj_for_guardrail(litellm_logging_obj: _GuardrailLoggingObj,
"""Configure the logging object so Langfuse/OTEL extract input and output correctly."""
litellm_logging_obj.call_type = "pass_through_endpoint"
litellm_logging_obj.model_call_details["call_type"] = "pass_through_endpoint"
litellm_logging_obj.update_messages(
request.messages if request.messages else [{"role": "user", "content": request.text}]
litellm_logging_obj.update_messages(request.messages if request.messages else [_logged_user_message(request)])
def _logged_user_message(request: ApplyGuardrailRequest) -> ChatCompletionUserMessage:
if not request.images:
return {"role": "user", "content": request.text}
text_parts: Final[tuple[ChatCompletionTextObject, ...]] = (
({"type": "text", "text": request.text},) if request.text else ()
)
image_parts: Final[tuple[ChatCompletionImageObject, ...]] = tuple(
{"type": "image_url", "image_url": {"url": image}} for image in request.images
)
return {"role": "user", "content": [*text_parts, *image_parts]}
async def _guardrail_response_after_success_hook(
@ -2534,6 +2551,31 @@ async def _guardrail_response_after_success_hook(
return response
async def _run_guardrail(
active_guardrail: CustomGuardrail,
request: ApplyGuardrailRequest,
request_data: dict[str, object], # mutable-ok: the guardrail records its verdict on the request dict
input_type: Literal["request", "response"],
) -> GenericGuardrailAPIInputs:
if request.images:
if request.messages and not request.text and ignores_messages(active_guardrail):
raise HTTPException(
status_code=422,
detail=(
f"Guardrail '{request.guardrail_name}' scans 'text' and ignores 'messages', "
"so put the text to scan in 'text'."
),
)
await scan_images(active_guardrail, request.images, request_data, input_type)
if not request.text and not request.messages:
return {"texts": []}
return await active_guardrail.apply_guardrail(
inputs={"texts": [request.text]},
request_data=request_data,
input_type=input_type,
)
def _record_verdict_for_logging(
request_data: dict[str, object], # mutable-ok: the logging bridge takes the request dict itself
data: dict[str, object], # mutable-ok: the logged request receives the guardrail's verdict
@ -2615,6 +2657,7 @@ async def apply_guardrail(
"guardrail_name": request.guardrail_name,
"input": [request.text],
"messages": request.messages or [],
**({"images": list(request.images)} if request.images else {}),
"metadata": {"route": "/apply_guardrail"},
}
litellm_logging_obj = None
@ -2655,11 +2698,7 @@ async def apply_guardrail(
request_data: Final = dict(response_handler.prepare_request_data(request))
input_type: Final = response_handler.resolve_input_type(request, active_guardrail)
try:
guardrailed_inputs: Final = await active_guardrail.apply_guardrail(
inputs={"texts": [request.text]},
request_data=request_data,
input_type=input_type,
)
guardrailed_inputs: Final = await _run_guardrail(active_guardrail, request, request_data, input_type)
finally:
_record_verdict_for_logging(request_data, data, litellm_logging_obj)
response: Final = response_handler.build_response(request, guardrailed_inputs, request_data)

View file

@ -250,6 +250,18 @@ def _without_image_bytes(content: Sequence[BedrockContentItem]) -> tuple[Bedrock
)
def bedrock_error_type(response: BedrockGuardrailResponse) -> str | None:
error_type: Final = str((response.get("Output") or {}).get("__type", ""))
return error_type if "Exception" in error_type else None
def _apply_guardrail_status(response: BedrockGuardrailResponse, images_sent: int) -> GuardrailStatus:
if bedrock_error_type(response) is not None:
return "guardrail_failed_to_respond"
images: Final = (response.get("guardrailCoverage") or {}).get("images") or {}
return "not_run" if images.get("guarded", 0) < images_sent else "success"
def _allowed_dynamic_body_params(params: Mapping[str, object]) -> Mapping[str, object]:
return {key: value for key, value in params.items() if key not in _BEDROCK_DYNAMIC_BODY_DENYLIST}
@ -258,6 +270,10 @@ def _overridden(guardrail: object, name: str) -> bool:
return getattr(getattr(guardrail, name, None), "__func__", None) is not getattr(BedrockGuardrail, name)
def overrides_bedrock_call(guardrail: object) -> bool:
return _overridden(guardrail, "apply_guardrail") or _overridden(guardrail, "make_bedrock_api_request")
def _is_responses_api_route(request_route: str | None) -> bool:
if request_route is None:
return False
@ -1062,6 +1078,7 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
merged_response: Final = self._merge_bedrock_guardrail_responses(responses)
self._log_apply_guardrail_success(
merged_response=merged_response,
images_sent=sum(1 for item in content if "image" in item),
request_data=request_data,
event_type=event_type,
start_time=start_time,
@ -1397,6 +1414,7 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
def _log_apply_guardrail_success(
self,
merged_response: BedrockGuardrailResponse,
images_sent: int,
request_data: dict[str, object] | None, # mutable-ok: proxy request body dict, mutated by the logging helper
event_type: GuardrailEventHooks,
start_time: "datetime",
@ -1416,11 +1434,7 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
guardrail_provider=self.guardrail_provider,
guardrail_json_response=dict(merged_response),
request_data=request_data if request_data is not None else {},
guardrail_status=(
"guardrail_failed_to_respond"
if "Exception" in str((merged_response.get("Output") or {}).get("__type", ""))
else "success"
),
guardrail_status=_apply_guardrail_status(merged_response, images_sent),
start_time=start_time.timestamp(),
end_time=datetime.now(timezone.utc).timestamp(),
duration=(datetime.now(timezone.utc) - start_time).total_seconds(),
@ -2567,7 +2581,7 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
call_type: CallTypesLiteral,
event_type: GuardrailEventHooks = GuardrailEventHooks.pre_call,
) -> None:
if _overridden(self, "apply_guardrail") or _overridden(self, "make_bedrock_api_request"):
if overrides_bedrock_call(self):
return
attachments: Final = find_request_attachments(
data,
@ -2591,23 +2605,13 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
await self._scan_document_texts(attachments.document_texts, request_data=data, event_type=event_type)
if not images:
return
bedrock_request_data: Final = {
"source": "INPUT",
**_allowed_dynamic_body_params(
self.get_guardrail_dynamic_request_body_params(request_data=data) # pyright: ignore[reportUnknownArgumentType, reportUnknownMemberType] # the base helper returns an untyped dict
),
}
api_key: Final = data.get("api_key")
try:
for start in range(0, len(images), _BEDROCK_APPLY_GUARDRAIL_MAX_IMAGES):
await self._apply_guardrail_to_content(
content=images[start : start + _BEDROCK_APPLY_GUARDRAIL_MAX_IMAGES],
bedrock_request_data=bedrock_request_data,
api_key=api_key if isinstance(api_key, str) else None,
await self.async_scan_images(
images[start : start + _BEDROCK_APPLY_GUARDRAIL_MAX_IMAGES],
request_data=data,
source="INPUT",
event_type=event_type,
start_time=datetime.now(timezone.utc),
allow_chunking=False,
)
except (HTTPException, ModifyResponseException):
raise
@ -2617,6 +2621,29 @@ class BedrockGuardrail(CustomGuardrail, BaseAWSLLM):
verbose_proxy_logger.error("Bedrock Guardrail: Failed to scan attachments: %s", str(e))
raise Exception(f"Bedrock guardrail failed: {e}") from e # noqa: TRY002 # same error the text scan raises
async def async_scan_images(
self,
images: Sequence[BedrockContentItem],
request_data: dict[str, object], # mutable-ok: proxy request body dict, mutated by the logging helper
source: Literal["INPUT", "OUTPUT"],
event_type: GuardrailEventHooks,
) -> BedrockGuardrailResponse:
api_key: Final = request_data.get("api_key")
return await self._apply_guardrail_to_content(
content=images,
bedrock_request_data={
"source": source,
**_allowed_dynamic_body_params(
self.get_guardrail_dynamic_request_body_params(request_data=request_data) # pyright: ignore[reportUnknownArgumentType, reportUnknownMemberType] # the base helper returns an untyped dict
),
},
api_key=api_key if isinstance(api_key, str) else None,
request_data=request_data,
event_type=event_type,
start_time=datetime.now(timezone.utc),
allow_chunking=False,
)
async def _scan_document_texts(
self,
document_texts: Sequence[str],

View file

@ -1570,7 +1570,12 @@ class GuardrailResponseFormat(Enum):
class ApplyGuardrailRequest(LiteLLMBaseModel):
guardrail_name: str
text: str
text: str = ""
images: tuple[str, ...] | None = Field(
default=None,
max_length=1,
description="One image to scan, as a base64 data URI. Only Bedrock guardrails scan images.",
)
language: str | None = None
entities: list[PiiEntityType] | None = None
input_type: str = "request"
@ -1578,6 +1583,12 @@ class ApplyGuardrailRequest(LiteLLMBaseModel):
metadata: dict[str, object] | None = None
response_format: GuardrailResponseFormat = GuardrailResponseFormat.STANDARD
@model_validator(mode="after")
def _require_text_or_images(self) -> "ApplyGuardrailRequest":
if "text" not in self.model_fields_set and not self.images:
raise ValueError("Provide 'text', 'images', or both.")
return self
class ApplyGuardrailResponse(LiteLLMBaseModel):
response_text: str

View file

@ -1,3 +1,4 @@
from collections.abc import Mapping
from typing import Literal
from typing_extensions import ReadOnly, TypedDict
@ -115,6 +116,7 @@ class BedrockGuardrailContextualGroundingPolicy(TypedDict, total=False):
class BedrockGuardrailCoverage(TypedDict, total=False):
textCharacters: dict[str, int]
images: ReadOnly[Mapping[str, int]]
class BedrockGuardrailInvocationMetrics(TypedDict, total=False):
@ -139,6 +141,7 @@ class BedrockGuardrailResponse(TypedDict, total=False):
output: list[BedrockGuardrailOutput] | None
outputs: list[BedrockGuardrailOutput] | None
assessments: list[BedrockGuardrailAssessment] | None
guardrailCoverage: ReadOnly[BedrockGuardrailCoverage | None]
# ---------------------------------------------------------------------------

View file

@ -28105,6 +28105,11 @@ export interface components {
entities?: components["schemas"]["PiiEntityType"][] | null;
/** Guardrail Name */
guardrail_name: string;
/**
* Images
* @description One image to scan, as a base64 data URI. Only Bedrock guardrails scan images.
*/
images?: string[] | null;
/**
* Input Type
* @default request
@ -28122,7 +28127,10 @@ export interface components {
} | null;
/** @default standard */
response_format: components["schemas"]["GuardrailResponseFormat"];
/** Text */
/**
* Text
* @default
*/
text: string;
};
/** ApplyGuardrailResponse */

View file

@ -2114,6 +2114,17 @@ def test_one_callback_opting_out_redacts_a_query_image_only_for_itself() -> None
assert SECRET in json.dumps(details, default=str)
def test_one_callback_opting_out_redacts_an_image_sent_to_apply_guardrail_only_for_itself() -> None:
details: Final[dict[str, object]] = {
"litellm_params": {"proxy_server_request": {"body": {"images": [f"data:image/png;base64,{SECRET}"]}}},
}
redacted: Final = redacted_model_call_details_for_custom_logger(details, TestCustomLogger(message_logging=False))
assert SECRET not in json.dumps(redacted, default=str)
assert SECRET in json.dumps(details, default=str)
@pytest.mark.usefixtures("_vcr_outcome_gate", "drain_logging_worker", "isolate_litellm_state", "setup_and_teardown")
@pytest.mark.asyncio
async def test_redaction_responses_api_stream():

View file

@ -6286,19 +6286,24 @@ async def test_attachment_scan_sends_image_only_turn_to_apply_guardrail_and_bloc
@pytest.mark.asyncio
async def test_attachment_scan_lets_a_passing_image_through():
@pytest.mark.parametrize(
("guarded", "logged_status"),
[(1, "success"), (0, "not_run")],
ids=["image-evaluated", "policy-without-image-modality"],
)
async def test_attachment_scan_lets_a_passing_image_through(guarded: int, logged_status: str):
guardrail = _attachment_guardrail()
data = _image_only_chat_request()
post_patch, credentials_patch, prepare_patch = _patched_bedrock_post(
guardrail, _passing_bedrock_httpx_response("ok")
)
response = _passing_bedrock_httpx_response("ok")
response.json.return_value["guardrailCoverage"] = {"images": {"guarded": guarded, "total": 1}}
post_patch, credentials_patch, prepare_patch = _patched_bedrock_post(guardrail, response)
with post_patch as mock_post, credentials_patch, prepare_patch:
await guardrail.async_scan_request_attachments(data=data, call_type=CallTypes.acompletion.value)
assert mock_post.await_count == 1
logged = data["metadata"]["standard_logging_guardrail_information"]
assert [entry["guardrail_status"] for entry in logged] == ["success"]
assert [entry["guardrail_status"] for entry in logged] == [logged_status]
@pytest.mark.asyncio
@ -8262,3 +8267,30 @@ async def test_verdict_is_recorded_on_a_request_dict_that_starts_empty(
recorded: Final = request_data["metadata"]["standard_logging_guardrail_information"]
assert [record["guardrail_status"] for record in recorded] == [recorded_status]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("guardrail_config", "bedrock"),
[
pytest.param(_APPLY_GUARDRAIL_API, _answering(503, {"message": "unavailable"}), id="apply_guardrail-error"),
pytest.param(_GUARDRAIL_CHECKS_API, _answering(500, {"message": "internal failure"}), id="guardrail_checks-error"),
],
)
async def test_a_failed_bedrock_call_keeps_reporting_as_http_exception_to_metrics_and_logs(
guardrail_config: Mapping[str, object],
bedrock: Callable[[httpx.Request], httpx.Response],
) -> None:
guardrail: Final = BedrockGuardrail(
guardrail_name="bedrock-guard",
aws_region_name="us-east-1",
aws_access_key_id="test-access-key",
aws_secret_access_key="test-secret-key",
async_handler=AsyncHTTPHandler(transport=httpx.MockTransport(bedrock)),
**guardrail_config,
)
with pytest.raises(HTTPException) as raised:
await guardrail.apply_guardrail(inputs={"texts": ["hello"]}, request_data={}, input_type="request")
assert type(raised.value) is HTTPException

View file

@ -0,0 +1,208 @@
import json
from collections.abc import Callable, Mapping
from typing import Final, Literal
import httpx
import pytest
from fastapi import HTTPException
import litellm
from litellm.integrations.custom_guardrail import CustomGuardrail
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler
from litellm.proxy.guardrails.endpoint_image_scan import scan_images
from litellm.proxy.guardrails.guardrail_hooks.bedrock_guardrails import BedrockGuardrail
from litellm.types.guardrails import GuardrailEventHooks
from litellm.types.llms.openai import AllMessageValues
from litellm.types.proxy.guardrails.guardrail_hooks.bedrock_guardrails import BedrockGuardrailResponse
_PNG_BASE64: Final = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
_PNG: Final = f"data:image/png;base64,{_PNG_BASE64}"
_RECORDS_KEY: Final = "standard_logging_guardrail_information"
_EVALUATED: Final = {
"action": "NONE",
"outputs": [],
"assessments": [],
"guardrailCoverage": {"images": {"guarded": 1, "total": 1}},
}
_NOT_EVALUATED: Final = {
"action": "NONE",
"outputs": [],
"assessments": [],
"guardrailCoverage": {"images": {"guarded": 0, "total": 1}},
}
_BLOCKED: Final = {
"action": "GUARDRAIL_INTERVENED",
"outputs": [{"text": "blocked"}],
"assessments": [{"contentPolicy": {"filters": [{"type": "VIOLENCE", "confidence": "HIGH", "action": "BLOCKED"}]}}],
"guardrailCoverage": {"images": {"guarded": 1, "total": 1}},
}
class _Bedrock:
def __init__(self, answer: Mapping[str, object], status_code: int = 200) -> None:
self.answer: Final = answer
self.status_code: Final = status_code
self.requests: Final[list[httpx.Request]] = []
def __call__(self, request: httpx.Request) -> httpx.Response:
self.requests.append(request)
return httpx.Response(self.status_code, json=self.answer)
def _apply_guardrail_api(bedrock: _Bedrock) -> BedrockGuardrail:
return BedrockGuardrail(
guardrail_name="image-guard",
guardrailIdentifier="gr-1",
guardrailVersion="1",
aws_region_name="us-east-1",
aws_access_key_id="test-access-key",
aws_secret_access_key="test-secret-key",
async_handler=AsyncHTTPHandler(transport=httpx.MockTransport(bedrock)),
)
def _guardrail_checks_api(bedrock: _Bedrock) -> BedrockGuardrail:
return BedrockGuardrail(
guardrail_name="image-guard",
checks={"contentFilter": {"categories": [{"category": "VIOLENCE"}]}},
aws_region_name="us-east-1",
aws_access_key_id="test-access-key",
aws_secret_access_key="test-secret-key",
async_handler=AsyncHTTPHandler(transport=httpx.MockTransport(bedrock)),
)
def _another_provider(bedrock: _Bedrock) -> CustomGuardrail:
return CustomGuardrail(guardrail_name="image-guard")
class _OwnBedrockCall(BedrockGuardrail):
async def make_bedrock_api_request(
self,
source: Literal["INPUT", "OUTPUT"],
messages: list[AllMessageValues] | None = None,
response: litellm.ModelResponse | None = None,
request_data: dict | None = None,
logging_event_type: GuardrailEventHooks | None = None,
) -> BedrockGuardrailResponse:
return {"action": "NONE"}
def _subclass_with_its_own_bedrock_call(bedrock: _Bedrock) -> CustomGuardrail:
return _OwnBedrockCall(
guardrail_name="image-guard",
guardrailIdentifier="gr-1",
guardrailVersion="1",
aws_region_name="us-east-1",
aws_access_key_id="test-access-key",
aws_secret_access_key="test-secret-key",
async_handler=AsyncHTTPHandler(transport=httpx.MockTransport(bedrock)),
)
@pytest.mark.asyncio
@pytest.mark.parametrize(
("input_type", "source", "mode"),
[("request", "INPUT", "pre_call"), ("response", "OUTPUT", "post_call")],
)
async def test_image_is_scanned_in_the_requested_direction_and_its_verdict_recorded(
input_type: Literal["request", "response"], source: str, mode: str
) -> None:
bedrock: Final = _Bedrock(_EVALUATED)
request_data: Final[dict[str, object]] = {}
await scan_images(_apply_guardrail_api(bedrock), (_PNG,), request_data, input_type)
assert [json.loads(request.content) for request in bedrock.requests] == [
{"source": source, "content": [{"image": {"format": "png", "source": {"bytes": _PNG_BASE64}}}]}
]
assert [
(record["guardrail_status"], record["guardrail_mode"]) for record in request_data["metadata"][_RECORDS_KEY]
] == [("success", mode)]
@pytest.mark.asyncio
async def test_blocked_image_raises_the_block_and_records_the_intervention() -> None:
request_data: Final[dict[str, object]] = {}
with pytest.raises(HTTPException) as blocked:
await scan_images(_apply_guardrail_api(_Bedrock(_BLOCKED)), (_PNG,), request_data, "request")
assert blocked.value.status_code == 400
assert [record["guardrail_status"] for record in request_data["metadata"][_RECORDS_KEY]] == ["guardrail_intervened"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
"image",
[
"https://example.com/cat.png",
_PNG_BASE64,
f"data:image/webp;base64,{_PNG_BASE64}",
"data:image/png;base64,bm90IGFuIGltYWdl",
"data:image/png;base64,@@@@",
],
ids=["remote-url", "bare-base64", "webp", "not-image-bytes", "invalid-base64"],
)
async def test_unreadable_image_is_refused_without_a_scan_or_a_recorded_block(image: str) -> None:
bedrock: Final = _Bedrock(_EVALUATED)
request_data: Final[dict[str, object]] = {}
with pytest.raises(HTTPException) as refused:
await scan_images(_apply_guardrail_api(bedrock), (image,), request_data, "request")
assert refused.value.status_code == 422
assert "base64 PNG or JPEG data URL" in refused.value.detail
assert bedrock.requests == []
assert request_data == {}
@pytest.mark.asyncio
@pytest.mark.parametrize("guardrail", [_guardrail_checks_api, _another_provider, _subclass_with_its_own_bedrock_call])
async def test_guardrail_that_cannot_scan_images_refuses_them(
guardrail: Callable[[_Bedrock], CustomGuardrail],
) -> None:
bedrock: Final = _Bedrock(_EVALUATED)
with pytest.raises(HTTPException) as refused:
await scan_images(guardrail(bedrock), (_PNG,), {}, "request")
assert refused.value.status_code == 422
assert bedrock.requests == []
@pytest.mark.asyncio
@pytest.mark.parametrize(
("answer", "status_code", "reason", "recorded_status"),
[
(_NOT_EVALUATED, 422, "Image modality", "not_run"),
({"action": "NONE", "outputs": [], "assessments": []}, 422, "Image modality", "not_run"),
({"Output": {"__type": "ServiceException"}}, 502, "ServiceException", "guardrail_failed_to_respond"),
],
ids=["reported-unguarded", "no-coverage-reported", "error-inside-a-200-body"],
)
async def test_image_bedrock_did_not_confirm_evaluating_is_not_reported_as_passed(
answer: Mapping[str, object], status_code: int, reason: str, recorded_status: str
) -> None:
request_data: Final[dict[str, object]] = {}
with pytest.raises(HTTPException) as refused:
await scan_images(_apply_guardrail_api(_Bedrock(answer)), (_PNG,), request_data, "request")
assert refused.value.status_code == status_code
assert reason in refused.value.detail
assert [record["guardrail_status"] for record in request_data["metadata"][_RECORDS_KEY]] == [recorded_status]
@pytest.mark.asyncio
@pytest.mark.parametrize(("bedrock_status", "status_code"), [(400, 422), (503, 503)])
async def test_image_bedrock_rejects_answers_unscanned_while_its_outages_pass_through(
bedrock_status: int, status_code: int
) -> None:
bedrock: Final = _Bedrock({"message": "The images contained in the request are not supported"}, bedrock_status)
with pytest.raises(HTTPException) as refused:
await scan_images(_apply_guardrail_api(bedrock), (_PNG,), {}, "request")
assert refused.value.status_code == status_code
assert "The images contained in the request are not supported" in refused.value.detail

View file

@ -11,7 +11,7 @@ import httpx
import pytest
import yaml
from fastapi import FastAPI, HTTPException
from pydantic import JsonValue
from pydantic import JsonValue, ValidationError
from pytest_mock import MockerFixture
import litellm
@ -53,6 +53,7 @@ from litellm.proxy.guardrails.guardrail_hooks.azure.prompt_shield import AzureCo
from litellm.proxy.guardrails.guardrail_hooks.bedrock_guardrails import BedrockGuardrail
from litellm.proxy.guardrails.guardrail_hooks.litellm_content_filter.content_filter import ContentFilterGuardrail
from litellm.proxy.hooks.proxy_track_cost_callback import ProxyDBLogger
from litellm.proxy.spend_tracking.spend_tracking_utils import get_logging_payload
MOCK_ADMIN_USER = UserAPIKeyAuth(user_role=LitellmUserRoles.PROXY_ADMIN)
from litellm.proxy.guardrails.guardrail_registry import (
@ -4215,7 +4216,12 @@ async def test_guardrail_success_delivers_sync_callback_once_with_overlapping_wo
_GUARDRAIL_RECORDS: Final = "standard_logging_guardrail_information"
_PNG_BASE64: Final = (
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
)
_PNG: Final = f"data:image/png;base64,{_PNG_BASE64}"
_BEDROCK_TEXT_ALLOWED: Final = {"action": "NONE", "outputs": [], "assessments": []}
_BEDROCK_IMAGE_ALLOWED: Final = {**_BEDROCK_TEXT_ALLOWED, "guardrailCoverage": {"images": {"guarded": 1, "total": 1}}}
_BEDROCK_BLOCKED: Final = {
"action": "GUARDRAIL_INTERVENED",
"outputs": [{"text": "blocked"}],
@ -4271,6 +4277,10 @@ def _bedrock_guard(handler: AsyncHTTPHandler, latest_user_message_only: bool = F
)
def _another_provider_guard(handler: AsyncHTTPHandler) -> CustomGuardrail:
return CustomGuardrail(guardrail_name="bedrock-guard", event_hook=GuardrailEventHooks.pre_call)
async def _apply_bedrock_guardrail_over_http(
mocker: MockerFixture,
bedrock_answers: Sequence[Mapping[str, object]],
@ -4336,6 +4346,10 @@ def _logged_verdicts(call: _BedrockGuardrailCall) -> list[tuple[object, object]]
return [(record["guardrail_name"], record["guardrail_status"]) for record in call.logged["guardrail_information"]]
def _scanned_content(call: _BedrockGuardrailCall) -> list[list[str]]:
return [[next(iter(item)) for item in payload["content"]] for payload in call.bedrock_payloads]
@pytest.mark.asyncio
async def test_apply_guardrail_logs_a_blocked_request_with_the_guardrail_verdict(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(mocker, (_BEDROCK_BLOCKED,), {"text": "hello"})
@ -4446,3 +4460,171 @@ async def test_one_callback_opting_out_of_message_logging_leaves_the_guardrail_d
assert "please bill my card" in str(call.logged["guardrail_information"])
assert [record["guardrail_name"] for record in opted_out.logged[0]["guardrail_information"]] == ["bedrock-guard"]
assert "please bill my card" not in str(opted_out.logged[0]["guardrail_information"])
@pytest.mark.asyncio
async def test_apply_guardrail_scans_an_image_sent_without_text(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(mocker, (_BEDROCK_IMAGE_ALLOWED,), {"images": [_PNG]})
assert call.response.status_code == 200
assert call.response.json() == {"response_text": ""}
assert _scanned_content(call) == [["image"]]
assert _logged_verdicts(call) == [("bedrock-guard", "success")]
assert _PNG_BASE64 not in json.dumps(call.logged, default=str)
assert [[part["type"] for part in message["content"]] for message in call.logged["messages"]] == [["image_url"]]
assert call.logged["messages"][0]["content"][0]["image_url"]["url"].startswith("data:image/png;base64,")
@pytest.mark.asyncio
async def test_apply_guardrail_scans_an_image_and_its_text_separately(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_IMAGE_ALLOWED, _BEDROCK_TEXT_ALLOWED), {"text": "hello", "images": [_PNG]}
)
assert call.response.json() == {"response_text": "hello"}
assert _scanned_content(call) == [["image"], ["text"]]
assert call.logged["messages"][0]["content"][0] == {"type": "text", "text": "hello"}
assert [part["type"] for part in call.logged["messages"][0]["content"]] == ["text", "image_url"]
@pytest.mark.asyncio
async def test_apply_guardrail_stops_at_a_blocked_image(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_BLOCKED, _BEDROCK_TEXT_ALLOWED), {"text": "hello", "images": [_PNG]}
)
assert call.response.status_code == 400
assert _scanned_content(call) == [["image"]]
assert _logged_verdicts(call) == [("bedrock-guard", "guardrail_intervened")]
@pytest.mark.asyncio
async def test_apply_guardrail_stores_the_image_in_the_spend_log_request(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(mocker, (_BEDROCK_BLOCKED,), {"images": [_PNG]})
mocker.patch("litellm.proxy.proxy_server.general_settings", {"store_prompts_in_spend_logs": True})
spend_row: Final = call.spend_rows[0]
stored_request: Final = get_logging_payload(
kwargs=spend_row["kwargs"],
response_obj=spend_row["completion_response"],
start_time=spend_row["start_time"],
end_time=spend_row["end_time"],
)["proxy_server_request"]
assert call.response.status_code == 400
assert json.loads(stored_request)["images"] == [_PNG]
@pytest.mark.asyncio
async def test_apply_guardrail_keeps_the_image_out_of_the_spend_log_when_message_logging_is_off(
mocker: MockerFixture,
) -> None:
mocker.patch.object(litellm, "turn_off_message_logging", True)
call: Final = await _apply_bedrock_guardrail_over_http(mocker, (_BEDROCK_BLOCKED,), {"images": [_PNG]})
mocker.patch("litellm.proxy.proxy_server.general_settings", {"store_prompts_in_spend_logs": True})
spend_row: Final = call.spend_rows[0]
stored_request: Final = get_logging_payload(
kwargs=spend_row["kwargs"],
response_obj=spend_row["completion_response"],
start_time=spend_row["start_time"],
end_time=spend_row["end_time"],
)["proxy_server_request"]
assert call.response.status_code == 400
assert _PNG_BASE64[:64] not in stored_request
@pytest.mark.parametrize(
("body", "reason"),
[
({"text": "hello", "images": [_PNG, _PNG]}, "at most 1 item"),
({}, "Provide 'text', 'images', or both"),
],
ids=["two-images", "neither-text-nor-image"],
)
def test_apply_guardrail_request_refuses_a_body_it_cannot_judge(body: Mapping[str, object], reason: str) -> None:
with pytest.raises(ValidationError, match=reason):
ApplyGuardrailRequest.model_validate({"guardrail_name": "bedrock-guard", **body})
@pytest.mark.asyncio
async def test_apply_guardrail_refuses_an_unreadable_image_without_calling_the_guardrail(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_IMAGE_ALLOWED,), {"text": "hello", "images": [f"data:image/webp;base64,{_PNG_BASE64}"]}
)
assert call.response.status_code == 422
assert call.bedrock_payloads == ()
@pytest.mark.asyncio
async def test_apply_guardrail_native_response_returns_the_image_findings(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_IMAGE_ALLOWED,), {"images": [_PNG], "response_format": "native"}
)
assert call.response.status_code == 200
assert call.response.json() == _BEDROCK_IMAGE_ALLOWED
@pytest.mark.asyncio
async def test_apply_guardrail_scans_a_response_image_as_model_output(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_IMAGE_ALLOWED,), {"images": [_PNG], "input_type": "response"}
)
assert call.response.status_code == 200
assert [payload["source"] for payload in call.bedrock_payloads] == ["OUTPUT"]
@pytest.mark.asyncio
async def test_apply_guardrail_blocks_text_sent_with_an_allowed_image(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (_BEDROCK_IMAGE_ALLOWED, _BEDROCK_BLOCKED), {"text": "hello", "images": [_PNG]}
)
assert call.response.status_code == 400
assert _logged_verdicts(call) == [("bedrock-guard", "success"), ("bedrock-guard", "guardrail_intervened")]
@pytest.mark.asyncio
async def test_apply_guardrail_still_scans_message_text_sent_with_an_image(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker,
(_BEDROCK_IMAGE_ALLOWED, _BEDROCK_BLOCKED),
{"images": [_PNG], "messages": [{"role": "user", "content": "hello"}]},
guardrail=lambda handler: _bedrock_guard(handler, latest_user_message_only=True),
)
assert call.response.status_code == 400
assert _scanned_content(call) == [["image"], ["text"]]
@pytest.mark.asyncio
async def test_apply_guardrail_refuses_message_text_a_bedrock_guardrail_would_not_scan(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (), {"images": [_PNG], "messages": [{"role": "user", "content": "hello"}]}
)
assert call.response.status_code == 422
assert "ignores 'messages'" in call.response.text
assert call.bedrock_payloads == ()
@pytest.mark.asyncio
async def test_apply_guardrail_refuses_an_image_for_a_guardrail_that_cannot_scan_it(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(
mocker, (), {"text": "hello", "images": [_PNG]}, guardrail=_another_provider_guard
)
assert call.response.status_code == 422
assert call.logged.get("guardrail_information") in (None, [])
@pytest.mark.asyncio
async def test_apply_guardrail_still_accepts_an_explicitly_empty_text(mocker: MockerFixture) -> None:
call: Final = await _apply_bedrock_guardrail_over_http(mocker, (_BEDROCK_TEXT_ALLOWED,), {"text": ""})
assert call.response.status_code == 200
assert call.response.json() == {"response_text": ""}