* fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException
The Bedrock-specific GuardrailInterventionNormalStringError predates the
unified guardrails refactor and no proxy code path handles it, so a block
with the flag set surfaced as an uncaught Exception -> HTTP 500 in pre_call
mode and was silently discarded in during_call mode (model call proceeded
in the parallel asyncio.gather; the block hook's data["mock_response"]
mutation happened after route_request had already unpacked kwargs).
Convert the block to ModifyResponseException at the raise site inside
make_bedrock_api_request. That exception is the industry-standard proxy
contract already caught in proxy_server, anthropic_endpoints, response_api
_endpoints, and pass_through_endpoints; it turns into a 200 response with
finish_reason=content_filter and the block message as content, which is
exactly what the flag was documented to yield. Post-call blocks attach
the LLM response to original_response so the synthetic reply reports the
upstream call's real token usage instead of zero.
Deletes the now-orphaned GuardrailInterventionNormalStringError class and
the dead create_guardrail_blocked_response / mock_response plumbing in the
Bedrock hooks; updates the existing tests that had locked in the buggy
contract.
Resolves LIT-4186
* chore(guardrails/bedrock): drop dead str branch in _update_messages_with_updated_bedrock_guardrail_response
Follow-up to the disable_exception_on_block fix. That method used to
receive either a BedrockGuardrailResponse or a plain string (the block
message, when the flag was set). Now that a block always raises
ModifyResponseException before this method runs, the string branch is
unreachable; tighten the type to BedrockGuardrailResponse and delete
the guard.
* fix(guardrails/bedrock): streaming post_call block yields synthetic stream instead of surfacing as SSE 500
Regression from the LIT-4186 refactor: pre-refactor, the streaming
post_call iterator caught GuardrailInterventionNormalStringError locally
and replaced the assembled response with a synthetic content-filter
message, then re-emitted it as chunks via MockResponseIterator. After
the refactor the exception was re-raised as ModifyResponseException,
which async_streaming_data_generator serializes as a proxy 500 error
frame because the SSE response headers are already flushed by the time
the block fires.
Non-streaming paths still let ModifyResponseException propagate to the
endpoint handler (which converts it into a 200). Streaming can't do
that, so keep the local synthesis: on the exception, rebind the
assembled response to a ModelResponse whose single choice carries the
block message as content and finish_reason=content_filter, and let the
downstream MockResponseIterator emit it as chunks. Same shape a
non-streaming block produces.
Adds a mapped-file regression test that mutation-kills the raise
behavior and locks in the synthetic-stream contract.
* fix(guardrails/bedrock): preserve upstream usage on streaming post_call block
Non-streaming post_call blocks report the upstream LLM call's real
token usage via ModifyResponseException.original_response, which the
endpoint handler unwraps through _blocked_response_usage. Streaming
post_call synthesizes its own ModelResponse locally (the exception
can't escape the SSE generator), and previously left .usage unset,
so the client saw accurate billing on non-streaming blocks and zero
on streaming blocks -- silent revenue leak.
Copy the assembled response's .usage onto the synthetic block
response before yielding. Pre-refactor code had the same gap
(create_guardrail_blocked_response never set usage); this is a net
improvement, not a regression fix.
* fix(bedrock_guardrails): select latest user message by original role in apply_guardrail (#23476)
* test(bedrock_guardrails): cover masking write-back through unified handler (#23476)
* fix(bedrock_guardrails): guard masked write-back on unresolved slice, not length
* chore(bedrock_guardrails): use builtin generics and extract write-back helper to satisfy strict ruff gate
* feat(guardrails): wire apply_guardrail into proxy logging callbacks
Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): fix Greptile review comments on apply_guardrail logging
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(apply_guardrail): preserve original exception and capture modified response
- Capture return value from post_call_success_hook so callback-modified
responses propagate to the caller.
- Wrap success/failure logging calls in defensive try/except so logging
infrastructure failures don't replace the user-visible response or mask
the original guardrail exception.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* Fix mypy
* fix(apply_guardrail): isolate failure logging and use post-hook response for logging
- Split async_failure_handler and post_call_failure_hook into independent
try/except blocks so a callback bug in one does not silently skip the
other.
- Build response_for_logging inside _emit_guardrail_success_logs after
post_call_success_hook runs, so logged data matches the response the
caller actually receives when the hook modifies the response.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(apply_guardrail): fix black formatting and update tests for fastapi_request param
- Run black on guardrail_endpoints.py to fix CI formatting check
- Add _mock_proxy_logging() helper to enterprise guardrail tests to patch
proxy-server globals imported at call time
- Pass fastapi_request=Mock() in all direct apply_guardrail test calls
to match updated function signature
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): use transformed exception from post_call_failure_hook in apply_guardrail
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(guardrails): isolate sync/async logging handlers in apply_guardrail
Separate each logging handler call into its own try/except so a failure
in the async handler does not silently skip the sync handler submission
(and vice versa). Matches the docstring's defensive intent.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(apply_guardrail): guard transformed_exception with isinstance check
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(guardrails): mock proxy globals in not_found test and share apply_guardrail logging fixture
- Add proxy-server global mocks to test_apply_guardrail_not_found so the
failure-path post_call_failure_hook call doesn't touch the real proxy
logging singleton.
- Extract the duplicated _mock_proxy_logging context manager out of the
two enterprise apply_guardrail test files into a shared conftest fixture
so the helper stays in one place.
* fix(guardrails): use update_messages to keep logging obj in sync
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* Fix Bedrock guardrail apply_guardrail method and test mocks
Fixed 4 failing tests in the guardrail test suite:
1. BedrockGuardrail.apply_guardrail now returns original texts when guardrail
allows content but doesn't provide output/outputs fields. Previously returned
empty list, causing test_bedrock_apply_guardrail_success to fail.
2. Updated test mocks to use correct Bedrock API response format:
- Changed from 'content' field to 'output' field
- Fixed nested structure from {'text': {'text': '...'}} to {'text': '...'}
- Added missing 'output' field in filter test
3. Fixed endpoint test mocks to return GenericGuardrailAPIInputs format:
- Changed from tuple (List[str], Optional[List[str]]) to dict {'texts': [...]}
- Updated method call assertions to use 'inputs' parameter correctly
All 12 guardrail tests now pass successfully.
* fix: remove python3-dev from Dockerfile.build_from_pip to avoid Python version conflict
The base image cgr.dev/chainguard/python:latest-dev already includes Python 3.14
and its development tools. Installing python3-dev pulls Python 3.13 packages
which conflict with the existing Python 3.14 installation, causing file
ownership errors during apk install.
* fix: disable callbacks in vertex fine-tuning tests to prevent Datadog logging interference
The test was failing because Datadog logging was making an HTTP POST request
that was being caught by the mock, causing assert_called_once() to fail.
By disabling callbacks during the test, we prevent Datadog from making any
HTTP calls, allowing the mock to only see the Vertex AI API call.
* fix: ensure test isolation in test_logging_non_streaming_request
Add proper cleanup to restore original litellm.callbacks after test execution.
This prevents test interference when running as part of a larger test suite,
where global state pollution was causing async_log_success_event to be
called multiple times instead of once.
Fixes test failure where the test expected async_log_success_event to be
called once but was being called twice due to callbacks from previous tests
not being cleaned up.
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail
* fix: add more rigorous call type checks
* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint
ensures call id + trace id are emitted to guardrail api
* feat(anthropic/chat/guardrail_translation): support streaming guardrails
sample on every 5 chunks
* fix(openai/chat/guardrail_translation): support openai streaming guardrails
* fix: initial commit fixing output guardrails for responses api
* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api
* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming
* test: update tests
* test: update tests
* fix: support multiple kinds of input to the guardrail api
* feat(guardrail_translation/handler.py): support extracting tool calls from openai chat completions for guardrail api's
* feat(generic_guardrail_api.py): support extracting + returning modified tool calls on generic_guardrails_api
allows guardrail api to analyze tool call being sent to provider - to run any analysis on it
* fix(guardrails.py): support anthropic /v1/messages tool calls
* feat(responses_api/): extract tool calls for guardrail processing
* docs(generic_guardrail_api.md): document tools param support
* docs: generic_guardrail_api.md
improve documentation
* fix(initial-commit): adding a way to get the right response type based on the api route
* feat(unified_guardrail.py): support streaming guardrails
* test: update tests
* fix: fix linting errors
* test: update tests
