Fixes#17477
Guardrails couldn't access request headers (like User-Agent) on Bedrock
pass-through endpoints because headers were only stored in
data["proxy_server_request"]["headers"] but not in data["metadata"]["headers"]
where guardrails typically look for them.
This fix adds headers to metadata in add_litellm_data_to_request() so
guardrails can access User-Agent, API keys, and other header-based checks
on all endpoints including Bedrock pass-through.
Test added to verify headers are available in metadata for guardrails.
- Add 'mask' action to SUPPORTED_ON_FLAGGED_ACTIONS
- Automatically sanitizes sensitive content using masked_session_messages
- Allows requests to proceed with masked content instead of blocking
- Add MCP call support
- Add pre_mcp_call and during_mcp_call to supported_event_hooks
- Verify mcp_call is supported in call_type Literal types
- Control exception details based on config
- Conditionally include scanners/evidence in exceptions based on
include_scanners and include_evidence settings
- Reduces payload size when detailed exception info isn't needed
- Add comprehensive test coverage
- Tests for masking functionality
- Tests for conditional exception details
- Tests for MCP call support
- Update documentation
- Add Mask section explaining masking functionality
- Clarify exception details control
All changes maintain backward compatibility.
extract model id from vertex ai passthrough routes that follow the pattern:
/vertex_ai/*/models/{model_id}:*
the model extraction now handles vertex ai routes by regex matching the model
segment from the url path, which allows proper model identification for
authentication and authorization in proxy pass-through endpoints.
adds comprehensive test coverage for vertex ai model extraction including:
- various vertex api versions (v1, v1beta1)
- different locations (us-central1, asia-southeast1)
- model names with special suffixes (gemini-1.5-pro, gemini-2.0-flash)
- precedence verification (request body model over url)
- non-vertex route isolation
* prioritized bedrock guardrail blocking by removing early return based on masking flags
When mask_request_content: true or mask_response_content: true, the method immediately returning False.
The Result: Even when Bedrock Guardrails returned action: "BLOCKED" for dangerous content, LiteLLM would not raise an exception and allowing the content through the response.
So removed that early condition which will return true for the blocked actions based on guardrails.
* Added test case for bedrock guardrail block content precedence
* fix(unified_guardrails.py): send all chunks on completion of final stream
* feat(generic_guardrail_api.py): handle tool call response on streaming LLM responses
* fix(anthropic/chat/guardrail_translation): initial commit adding anthropic tool response streaming guardrails
enables guardrail checks on tool response from llm's to work via `/v1/messages`
* feat(anthropic/): working guardrail checks on tool response from LLMs
ensures guardrail checks on anthropic /v1/messages works as expected
* feat(responses/guardrail_translation): support tool call response guardrails on streaming for /v1/responses
ensures complete coverage of tool call responses
* refactor(openai.py): refactor to use consistent pydantic model for responses api tool response on streaming
enables non-openai model tool call response to work correctly with guardrail checks on /v1/responses
* test: update tests
* fix: fix linting error
* fix: fix failing tests
* fix: fix import errors
* fix(openai/chat/guardrail_transformation): fix final chunk returned on streaming
- Add database and Redis setup to litellm_mapped_tests_proxy job in CircleCI
- Create shared test helpers in tests/test_litellm/proxy/conftest.py for proxy test setup
- Refactor health endpoint tests to use shared helpers from conftest
- Support automatic Redis cache configuration when REDIS_HOST is set
- Ensure minimal config is created when Redis/database is needed
