mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-09 22:31:41 +00:00
feat(panw-prisma-airs): PANW Prisma AIRS guardrail with apply_guardrail support (#22999)
* feat(panw-prisma-airs): PANW Prisma AIRS guardrail with apply_guardrail support * fix(panw): honor masking and fallback behavior * fix(panw): clean up apply_guardrail MCP metadata handling * fix(panw): clean up apply_guardrail MCP metadata handling * fix(panw): harden apply_guardrail edge cases * fix(panw): apply MCP masked data on allow responses * fix(panw): scan latest developer message in anthropic mode * fix(panw): restore legacy user-only pre-call scanning * fix(panw): record apply_guardrail in applied guardrails header * fix(panw): scan developer role in legacy pre-call path * fix(panw): harden SSE parsing and narrow MCP name fallback * fix(panw): harden streaming attr lookup and document dual scans * fix(panw): fail closed on permanent 4xx and cover streaming observability
This commit is contained in:
Jason Roberts
GitHub
parent
810de556bd
commit
70fca22f68
7 changed files with 5203 additions and 857 deletions
|
|
@ -11,6 +11,7 @@ This endpoint supports various guardrail types including:
|
|||
- **Presidio** - PII detection and masking
|
||||
- **Bedrock** - AWS Bedrock guardrails for content moderation
|
||||
- **Lakera** - AI safety guardrails
|
||||
- **PANW Prisma AIRS** - Threat detection, DLP, and policy enforcement
|
||||
- **Custom guardrails** - User-defined guardrails
|
||||
|
||||
## Configuration
|
||||
|
|
|
|||
|
|
@ -86,4 +86,5 @@ MCP guardrails work with all LiteLLM-supported guardrail providers:
|
|||
- **Lakera**: Content moderation
|
||||
- **Aporia**: Custom guardrails
|
||||
- **Noma**: Noma Security
|
||||
- **PANW Prisma AIRS**: Prisma AIRS guardrails
|
||||
- **Custom**: Your own guardrail implementations
|
||||
|
|
@ -1,24 +1,15 @@
|
|||
import Image from '@theme/IdealImage';
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# PANW Prisma AIRS
|
||||
|
||||
LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi//). This integration provides **Security-as-Code** for AI applications using Palo Alto Networks' AI security platform.
|
||||
LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi/). This integration provides Security-as-Code for AI applications using Palo Alto Networks' AI security platform.
|
||||
|
||||
## Features
|
||||
- **Prompt injection and malicious URL detection** — real-time scanning before or after LLM calls
|
||||
- **Data loss prevention (DLP)** — detect and block sensitive data in prompts and responses
|
||||
- **Sensitive content masking** — automatically mask PII, credit cards, SSNs instead of blocking
|
||||
- **MCP tool call scanning** — scan tool name and arguments on direct MCP tool invocations
|
||||
- **Configurable fail-open / fail-closed** — choose between maximum security or high availability
|
||||
|
||||
- ✅ **Real-time prompt injection detection**
|
||||
- ✅ **Malicious URL detection**
|
||||
- ✅ **Data loss prevention (DLP)**
|
||||
- ✅ **Sensitive content masking** - Automatically mask PII, credit cards, SSNs instead of blocking
|
||||
- ✅ **Comprehensive threat detection** for AI models and datasets
|
||||
- ✅ **Model-agnostic protection** across public and private models
|
||||
- ✅ **Synchronous scanning** with immediate response
|
||||
- ✅ **Configurable security profiles**
|
||||
- ✅ **Streaming support** - Real-time masking for streaming responses
|
||||
- ✅ **Multi-turn conversation tracking** - Automatic session grouping in Prisma AIRS SCM logs
|
||||
- ✅ **Configurable fail-open/fail-closed** - Choose between maximum security (block on API errors) or high availability (allow on transient errors)
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
|
@ -32,7 +23,14 @@ For detailed setup instructions, see the [Prisma AIRS API Overview](https://docs
|
|||
|
||||
### 2. Define Guardrails on your LiteLLM config.yaml
|
||||
|
||||
Define your guardrails under the `guardrails` section:
|
||||
Set `api_base` to the regional endpoint for your Prisma AIRS deployment profile:
|
||||
|
||||
| Region | Endpoint |
|
||||
|--------|----------|
|
||||
| US | `https://service.api.aisecurity.paloaltonetworks.com` |
|
||||
| EU (Germany) | `https://service-de.api.aisecurity.paloaltonetworks.com` |
|
||||
| India | `https://service-in.api.aisecurity.paloaltonetworks.com` |
|
||||
| Singapore | `https://service-sg.api.aisecurity.paloaltonetworks.com` |
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
|
|
@ -45,21 +43,15 @@ guardrails:
|
|||
- guardrail_name: "panw-prisma-airs-guardrail"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "pre_call" # Run before LLM call
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY # Your Prisma AIRS API key
|
||||
profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME # Security profile from Strata Cloud Manager
|
||||
api_base: "https://service.api.aisecurity.paloaltonetworks.com"
|
||||
mode: "pre_call"
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME
|
||||
api_base: "https://service.api.aisecurity.paloaltonetworks.com" # US — change to your region
|
||||
```
|
||||
|
||||
#### Supported values for `mode`
|
||||
|
||||
- `pre_call` Run **before** LLM call, on **input**
|
||||
- `post_call` Run **after** LLM call, on **input & output**
|
||||
- `during_call` Run **during** LLM call, on **input**. Same as `pre_call` but runs in parallel with LLM call
|
||||
|
||||
### 3. Start LiteLLM Gateway
|
||||
|
||||
```bash title="Set environment variables"
|
||||
```bash
|
||||
export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
|
||||
export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
|
||||
export OPENAI_API_KEY="sk-proj-..."
|
||||
|
|
@ -69,15 +61,8 @@ export OPENAI_API_KEY="sk-proj-..."
|
|||
litellm --config config.yaml --detailed_debug
|
||||
```
|
||||
|
||||
|
||||
### 4. Test Request
|
||||
|
||||
**[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys#request-format)**
|
||||
|
||||
<Tabs>
|
||||
<TabItem label="Blocked request" value="blocked">
|
||||
|
||||
Expect this to fail due to prompt injection attempt:
|
||||
|
||||
```shell
|
||||
curl -i http://localhost:4000/v1/chat/completions \
|
||||
|
|
@ -92,254 +77,57 @@ curl -i http://localhost:4000/v1/chat/completions \
|
|||
}'
|
||||
```
|
||||
|
||||
Expected response on failure:
|
||||
Expected response when the guardrail blocks:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": {
|
||||
"error": "Violated PANW Prisma AIRS guardrail policy",
|
||||
"panw_response": {
|
||||
"action": "block",
|
||||
"category": "malicious",
|
||||
"profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
|
||||
"profile_name": "dev-block-all-profile",
|
||||
"prompt_detected": {
|
||||
"dlp": false,
|
||||
"injection": true,
|
||||
"toxic_content": false,
|
||||
"url_cats": false
|
||||
},
|
||||
"report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
|
||||
"response_detected": {
|
||||
"dlp": false,
|
||||
"toxic_content": false,
|
||||
"url_cats": false
|
||||
},
|
||||
"scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
|
||||
"tr_id": "string"
|
||||
}
|
||||
},
|
||||
"type": "None",
|
||||
"param": "None",
|
||||
"code": "400"
|
||||
"message": "Prompt blocked by PANW Prisma AI Security policy (Category: malicious)",
|
||||
"type": "guardrail_violation",
|
||||
"code": "panw_prisma_airs_blocked",
|
||||
"guardrail": "panw-prisma-airs-guardrail",
|
||||
"category": "malicious"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem label="Successful Call" value="allowed">
|
||||
LiteLLM wraps this detail in an endpoint-specific HTTP error envelope. Optional fields that may also appear: `scan_id`, `report_id`, `profile_name`, `profile_id`, `tr_id`, `prompt_detected`.
|
||||
|
||||
```shell
|
||||
curl -i http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-your-api-key" \
|
||||
-d '{
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the weather like today?"}
|
||||
],
|
||||
"guardrails": ["panw-prisma-airs-guardrail"]
|
||||
}'
|
||||
```
|
||||
On success, the guardrail name appears in the `x-litellm-applied-guardrails` response header.
|
||||
|
||||
Expected successful response:
|
||||
## Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"message": {
|
||||
"content": "I don't have access to real-time weather data, but I can help you find weather information through various weather services or apps...",
|
||||
"role": "assistant",
|
||||
"tool_calls": null,
|
||||
"function_call": null,
|
||||
"annotations": []
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1736028456,
|
||||
"id": "chatcmpl-AqQj8example",
|
||||
"model": "gpt-4o",
|
||||
"object": "chat.completion",
|
||||
"usage": {
|
||||
"completion_tokens": 25,
|
||||
"prompt_tokens": 12,
|
||||
"total_tokens": 37
|
||||
},
|
||||
"x-litellm-panw-scan": {
|
||||
"action": "allow",
|
||||
"category": "benign",
|
||||
"profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
|
||||
"profile_name": "dev-block-all-profile",
|
||||
"prompt_detected": {
|
||||
"dlp": false,
|
||||
"injection": false,
|
||||
"toxic_content": false,
|
||||
"url_cats": false
|
||||
},
|
||||
"report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
|
||||
"response_detected": {
|
||||
"dlp": false,
|
||||
"toxic_content": false,
|
||||
"url_cats": false
|
||||
},
|
||||
"scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
|
||||
"tr_id": "string"
|
||||
}
|
||||
}
|
||||
```
|
||||
### Supported Modes
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
| Mode | Timing | What is scanned |
|
||||
|------|--------|-----------------|
|
||||
| `pre_call` | Before LLM call | Request input |
|
||||
| `during_call` | Parallel with LLM call | Request input |
|
||||
| `post_call` | After LLM call | Response output |
|
||||
| `pre_mcp_call` | Before MCP tool execution | MCP tool input |
|
||||
| `during_mcp_call` | Parallel with MCP tool execution | MCP tool input |
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
### Configuration Parameters
|
||||
|
||||
| Parameter | Required | Description | Default |
|
||||
|-----------|----------|-------------|---------|
|
||||
| `api_key` | Yes | Your PANW Prisma AIRS API key from Strata Cloud Manager | - |
|
||||
| `profile_name` | No | Security profile name configured in Strata Cloud Manager. Optional if API key has linked profile | - |
|
||||
| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (will be prefixed with "LiteLLM-") | `LiteLLM` |
|
||||
| `api_base` | No | Regional API endpoint (see [Regional Endpoints](#regional-endpoints) below) | `https://service.api.aisecurity.paloaltonetworks.com` (US) |
|
||||
| `mode` | No | When to run the guardrail | `pre_call` |
|
||||
| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed, default) or `"allow"` (fail-open). Config errors always block. | `block` |
|
||||
| `timeout` | No | PANW API call timeout in seconds (1-60) | `10.0` |
|
||||
| `violation_message_template` | No | Custom template for error message when request is blocked. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
|
||||
| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (prefixed with "LiteLLM-") | `LiteLLM` |
|
||||
| `api_base` | No | Regional API endpoint. US: `https://service.api.aisecurity.paloaltonetworks.com`, EU: `https://service-de.api.aisecurity.paloaltonetworks.com`, India: `https://service-in.api.aisecurity.paloaltonetworks.com`, Singapore: `https://service-sg.api.aisecurity.paloaltonetworks.com` | US |
|
||||
| `mode` | No | When to run the guardrail (see mode table above) | `pre_call` |
|
||||
| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed) or `"allow"` (fail-open). Config errors always block. | `block` |
|
||||
| `timeout` | No | PANW API call timeout in seconds (recommended: 1-60) | `10.0` |
|
||||
| `violation_message_template` | No | Custom template for blocked requests. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
|
||||
| `mask_request_content` | No | Mask sensitive data in prompts instead of blocking | `false` |
|
||||
| `mask_response_content` | No | Mask sensitive data in responses instead of blocking | `false` |
|
||||
| `mask_on_block` | No | Backwards-compatible flag that enables both request and response masking | `false` |
|
||||
| `experimental_use_latest_role_message_only` | No | Anthropic `/v1/messages` only. When unset: scans only latest user message on request side. Set `false` to scan all user/system/developer messages. Non-Anthropic unaffected. | Unset (true for Anthropic) |
|
||||
|
||||
### Regional Endpoints
|
||||
Use the regional `api_base` that matches your Prisma AIRS deployment profile region for lower latency and data residency compliance.
|
||||
|
||||
PANW Prisma AIRS supports multiple regional endpoints based on your deployment profile region:
|
||||
|
||||
| Region | API Base URL |
|
||||
|--------|--------------|
|
||||
| **US** (default) | `https://service.api.aisecurity.paloaltonetworks.com` |
|
||||
| **EU (Germany)** | `https://service-de.api.aisecurity.paloaltonetworks.com` |
|
||||
| **India** | `https://service-in.api.aisecurity.paloaltonetworks.com` |
|
||||
|
||||
**Example configuration for EU region:**
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-eu"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
api_base: "https://service-de.api.aisecurity.paloaltonetworks.com"
|
||||
profile_name: "production"
|
||||
```
|
||||
|
||||
:::tip Region Selection
|
||||
Use the regional endpoint that matches your Prisma AIRS deployment profile region configured in Strata Cloud Manager. Using the correct region ensures:
|
||||
- Lower latency (requests stay in-region)
|
||||
- Compliance with data residency requirements
|
||||
- Optimal performance
|
||||
:::
|
||||
|
||||
## Per-Request Metadata Overrides
|
||||
|
||||
You can override guardrail settings on a per-request basis using the `metadata` field:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-4",
|
||||
"messages": [...],
|
||||
"metadata": {
|
||||
"profile_name": "dev-allow-all", // Override profile name
|
||||
"profile_id": "uuid-here", // Override profile ID (takes precedence)
|
||||
"user_ip": "192.168.1.100", // Track user IP
|
||||
"app_name": "MyApp" // Custom app name (becomes "LiteLLM-MyApp")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Supported Metadata Fields:**
|
||||
|
||||
| Field | Description | Priority |
|
||||
|-------|-------------|----------|
|
||||
| `profile_name` | PANW AI security profile name | Per-request > config |
|
||||
| `profile_id` | PANW AI security profile ID (takes precedence over profile_name) | Per-request only |
|
||||
| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
|
||||
| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
|
||||
| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
|
||||
|
||||
:::info Profile Resolution
|
||||
- If both `profile_id` and `profile_name` are provided, PANW API uses `profile_id` (it takes precedence)
|
||||
- If no profile is specified in metadata, uses the config `profile_name`
|
||||
- If no profile is specified at all, PANW API will use the profile linked to your API key in Strata Cloud Manager
|
||||
- **Note:** If your API key is not linked to a profile, you must provide `profile_name` or `profile_id`
|
||||
:::
|
||||
|
||||
## Multi-Turn Conversation Tracking
|
||||
|
||||
PANW Prisma AIRS automatically tracks multi-turn conversations using LiteLLM's `litellm_trace_id`. This enables you to:
|
||||
|
||||
- **Group related requests** - All requests in a conversation share the same AI Session ID in Prisma AIRS SCM logs
|
||||
- **Track conversation context** - See the full history of prompts and responses for a user session
|
||||
- **Analyze attack patterns** - Identify sophisticated multi-turn attacks across conversation history
|
||||
|
||||
### How It Works
|
||||
|
||||
LiteLLM automatically generates a unique `litellm_trace_id` for each conversation session. The PANW guardrail uses this as the PANW transaction ID (which maps to "AI Session ID" in Strata Cloud Manager):
|
||||
|
||||
```
|
||||
Conversation Session: litellm_trace_id = "abc-123-def-456"
|
||||
|
||||
Turn 1 (User): "What's the capital of France?"
|
||||
→ Scan ID: scan_001 | Prisma AIRS AI Session ID: abc-123-def-456
|
||||
|
||||
Turn 2 (Assistant): "Paris is the capital of France."
|
||||
→ Scan ID: scan_002 | Prisma AIRS AI Session ID: abc-123-def-456
|
||||
|
||||
Turn 3 (User): "What's the population?"
|
||||
→ Scan ID: scan_003 | Prisma AIRS AI Session ID: abc-123-def-456
|
||||
|
||||
Turn 4 (Assistant): "Paris has approximately 2.1 million residents."
|
||||
→ Scan ID: scan_004 | Prisma AIRS AI Session ID: abc-123-def-456
|
||||
```
|
||||
|
||||
All scans appear under the same AI Session ID in Prisma AIRS logs, making it easy to:
|
||||
- Review complete conversation history (all 4 turns grouped together)
|
||||
- Identify patterns across multiple turns
|
||||
- Correlate security events within a session
|
||||
- Track the flow of user prompts and AI responses
|
||||
|
||||
### Session Tracking
|
||||
|
||||
LiteLLM automatically generates a unique `litellm_trace_id` for each request, which the PANW guardrail uses as the AI Session ID in Strata Cloud Manager. All prompt and response scans for a request are automatically grouped under the same session.
|
||||
|
||||
#### Custom Session IDs (Per-App Tracking)
|
||||
|
||||
You can provide your own `litellm_trace_id` to track sessions on a per-app or per-conversation basis:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"messages": [{"role": "user", "content": "capital of France"}],
|
||||
"litellm_trace_id": "my-app-session-123", # Custom AI Session ID
|
||||
"metadata": {
|
||||
"profile_name": "dev-allow-all-profile", # Override security profile
|
||||
"user_ip": "192.168.1.1", # Track user IP
|
||||
"app_name": "eng" # Custom app identifier
|
||||
},
|
||||
"guardrails": ["panw-prisma-airs-pre-guard", "panw-prisma-airs-post-guard"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Result in PANW SCM:**
|
||||
- AI Session ID: `my-app-session-123`
|
||||
- All prompt and response scans will be grouped under this custom session ID
|
||||
- Perfect for tracking multi-turn conversations or per-application sessions
|
||||
|
||||
:::tip Viewing Sessions in Prisma AIRS SCM Logs
|
||||
In Strata Cloud Manager, navigate to **AI Runtime > Sessions** to view all AI Session IDs and their associated scans. Click on a session to see the complete conversation history with security analysis.
|
||||
:::
|
||||
|
||||
## Environment Variables
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
|
||||
|
|
@ -348,12 +136,31 @@ export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
|
|||
export PANW_PRISMA_AIRS_API_BASE="https://custom-endpoint.com"
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
### Per-Request Metadata Overrides
|
||||
|
||||
| Field | Description | Priority |
|
||||
|-------|-------------|----------|
|
||||
| `profile_name` | PANW AI security profile name | Per-request > config |
|
||||
| `profile_id` | PANW AI security profile ID (takes precedence over `profile_name`) | Per-request only |
|
||||
| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
|
||||
| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
|
||||
| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-4",
|
||||
"messages": [...],
|
||||
"metadata": {
|
||||
"profile_name": "dev-allow-all",
|
||||
"profile_id": "uuid-here",
|
||||
"user_ip": "192.168.1.100",
|
||||
"app_name": "MyApp"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Multiple Security Profiles
|
||||
|
||||
You can configure different security profiles for different use cases:
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-strict-security"
|
||||
|
|
@ -361,126 +168,40 @@ guardrails:
|
|||
guardrail: panw_prisma_airs
|
||||
mode: "pre_call"
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: "strict-policy" # High security profile
|
||||
|
||||
- guardrail_name: "panw-permissive-security"
|
||||
profile_name: "strict-policy"
|
||||
|
||||
- guardrail_name: "panw-permissive-security"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "post_call"
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: "permissive-policy" # Lower security profile
|
||||
profile_name: "permissive-policy"
|
||||
```
|
||||
|
||||
### Multiple API Keys (Multi-Tenant)
|
||||
|
||||
For multi-tenant deployments where different customers need different PANW API keys, create separate guardrail instances:
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-customer-a"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "pre_call"
|
||||
api_key: os.environ/PANW_CUSTOMER_A_KEY # Linked to Customer A profile in SCM
|
||||
|
||||
- guardrail_name: "panw-customer-b"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "pre_call"
|
||||
api_key: os.environ/PANW_CUSTOMER_B_KEY # Linked to Customer B profile in SCM
|
||||
```
|
||||
|
||||
Then route requests to the appropriate guardrail:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"guardrails": ["panw-customer-a"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- **Multi-tenant deployments**: Different customers with different security policies
|
||||
- **Environment-specific policies**: Dev/staging/prod with different API keys and profiles
|
||||
- **A/B testing**: Compare different security profiles side-by-side
|
||||
|
||||
### Content Masking
|
||||
|
||||
PANW Prisma AIRS can automatically mask sensitive content (PII, credit cards, SSNs, etc.) instead of blocking requests. This allows your application to continue functioning while protecting sensitive data.
|
||||
|
||||
#### How It Works
|
||||
|
||||
1. **Detection**: PANW scans content and identifies sensitive data
|
||||
2. **Masking**: Sensitive data is replaced with placeholders (e.g., `XXXXXXXXXX` or `{PHONE}`)
|
||||
3. **Pass-through**: Masked content is sent to the LLM or returned to the user
|
||||
|
||||
#### Configuration Options
|
||||
:::warning Important: Masking is Controlled by PANW Security Profile
|
||||
The actual masking behavior (what content gets masked and how) is controlled by your PANW Prisma AIRS security profile in Strata Cloud Manager. The LiteLLM flags (`mask_request_content`, `mask_response_content`) only control whether to apply the masked content and allow the request to continue, or block entirely.
|
||||
:::
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-with-masking"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "post_call" # Scan response output
|
||||
mode: "post_call"
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: "default"
|
||||
mask_request_content: true # Mask sensitive data in prompts
|
||||
mask_response_content: true # Mask sensitive data in responses
|
||||
mask_request_content: true
|
||||
mask_response_content: true
|
||||
```
|
||||
|
||||
**Masking Parameters:**
|
||||
|
||||
- `mask_request_content: true` - When PANW detects sensitive data in prompts, mask it instead of blocking
|
||||
- `mask_response_content: true` - When PANW detects sensitive data in responses, mask it instead of blocking
|
||||
- `mask_on_block: true` - Backwards compatible flag that enables both request and response masking
|
||||
|
||||
:::warning Important: Masking is Controlled by PANW Security Profile
|
||||
The **actual masking behavior** (what content gets masked and how) is controlled by your **PANW Prisma AIRS security profile** configured in Strata Cloud Manager. The LiteLLM config settings (`mask_request_content`, `mask_response_content`) only control whether to:
|
||||
- **Apply the masked content** returned by PANW and allow the request to continue, OR
|
||||
- **Block the request** entirely when sensitive data is detected
|
||||
|
||||
LiteLLM does not alter or configure your PANW security profile. To change what content gets masked, update your profile settings in Strata Cloud Manager.
|
||||
:::
|
||||
|
||||
:::info Security Posture
|
||||
The guardrail is **fail-closed** by default - if the PANW API is unavailable, requests are blocked to ensure no unscanned content reaches your LLM. This provides maximum security.
|
||||
:::
|
||||
|
||||
### Custom Violation Messages
|
||||
|
||||
You can customize the error message returned to the user when a request is blocked by configuring the `violation_message_template` parameter. This is useful for providing user-friendly feedback instead of technical details.
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-custom-message"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
# Simple message
|
||||
violation_message_template: "Your request was blocked by our AI Security Policy."
|
||||
|
||||
- guardrail_name: "panw-detailed-message"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
# Message with placeholders
|
||||
violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
|
||||
```
|
||||
|
||||
**Supported Placeholders:**
|
||||
- `{guardrail_name}`: Name of the guardrail (e.g. "panw-custom-message")
|
||||
- `{category}`: Violation category (e.g. "malicious", "injection", "dlp")
|
||||
- `{action_type}`: "Prompt" or "Response"
|
||||
- `{default_message}`: The original technical error message
|
||||
- `mask_request_content: true` — mask sensitive data in prompts instead of blocking
|
||||
- `mask_response_content: true` — mask sensitive data in responses instead of blocking
|
||||
- `mask_on_block: true` — backwards-compatible flag that enables both request and response masking
|
||||
|
||||
### Fail-Open Configuration
|
||||
|
||||
By default, the PANW guardrail operates in **fail-closed** mode for maximum security. If the PANW API is unavailable (timeout, rate limit, network error), requests are blocked. You can configure **fail-open** mode for high-availability scenarios where service continuity is critical.
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-high-availability"
|
||||
|
|
@ -488,135 +209,86 @@ guardrails:
|
|||
guardrail: panw_prisma_airs
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: "production"
|
||||
fallback_on_error: "allow" # Enable fail-open mode
|
||||
timeout: 5.0 # Shorter timeout for fail-open
|
||||
fallback_on_error: "allow"
|
||||
timeout: 5.0
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Parameter | Value | Behavior |
|
||||
|-----------|-------|----------|
|
||||
| `fallback_on_error` | `"block"` (default) | **Fail-closed**: Block requests when API unavailable (maximum security) |
|
||||
| `fallback_on_error` | `"allow"` | **Fail-open**: Allow requests when API unavailable (high availability) |
|
||||
| `timeout` | `1.0` - `60.0` | API call timeout in seconds (default: `10.0`) |
|
||||
|
||||
**Error Handling Matrix:**
|
||||
|
||||
| Error Type | `fallback_on_error="block"` | `fallback_on_error="allow"` |
|
||||
|------------|----------------------------|----------------------------|
|
||||
| 401 Unauthorized | Block (500) | Block (500) ⚠️ |
|
||||
| 403 Forbidden | Block (500) | Block (500) ⚠️ |
|
||||
| Profile Error | Block (500) | Block (500) ⚠️ |
|
||||
| 401 Unauthorized | Block (500) | Block (500) |
|
||||
| 403 Forbidden | Block (500) | Block (500) |
|
||||
| Profile Error | Block (500) | Block (500) |
|
||||
| 429 Rate Limit | Block (500) | Allow (`:unscanned`) |
|
||||
| Timeout | Block (500) | Allow (`:unscanned`) |
|
||||
| Network Error | Block (500) | Allow (`:unscanned`) |
|
||||
| 5xx Server Error | Block (500) | Allow (`:unscanned`) |
|
||||
| Content Blocked | Block (400) | Block (400) |
|
||||
|
||||
⚠️ = Always blocks regardless of fail-open setting
|
||||
Authentication and configuration errors (401, 403, invalid profile) always block. Only transient errors (429, timeout, network) trigger fail-open.
|
||||
|
||||
:::warning Security Trade-Off
|
||||
Enabling `fallback_on_error="allow"` reduces security in exchange for availability. Requests may proceed **without scanning** when the PANW API is unavailable. Use only when:
|
||||
- Service availability is more critical than security scanning
|
||||
- You have other security controls in place
|
||||
- You monitor the `:unscanned` header for audit trails
|
||||
When fail-open is triggered, the response includes a tracking header: `X-LiteLLM-Applied-Guardrails: panw-airs:unscanned`
|
||||
|
||||
**Authentication and configuration errors (401, 403, invalid profile) always block** - only transient errors (429, timeout, network) trigger fail-open behavior.
|
||||
:::
|
||||
|
||||
**Observability:**
|
||||
|
||||
When fail-open is triggered, the response includes a special header for tracking:
|
||||
|
||||
```
|
||||
X-LiteLLM-Applied-Guardrails: panw-airs:unscanned
|
||||
```
|
||||
|
||||
This allows you to:
|
||||
- Track which requests bypassed scanning
|
||||
- Alert on unscanned request volumes
|
||||
- Audit compliance requirements
|
||||
|
||||
#### Example: Masking Credit Card Numbers
|
||||
|
||||
<Tabs>
|
||||
<TabItem label="Without Masking" value="no-mask">
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:** ❌ **Blocked with 400 error**
|
||||
|
||||
</TabItem>
|
||||
<TabItem label="With Masking" value="with-mask">
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Masked prompt sent to LLM:**
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "My credit card is XXXXXXXXXXXXXXXXXX"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:** ✅ **Allowed with masked content**
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
#### Masking Capabilities
|
||||
|
||||
The guardrail masks sensitive content in:
|
||||
|
||||
- ✅ **Chat messages** - User prompts and assistant responses
|
||||
- ✅ **Streaming responses** - Real-time masking of streamed content
|
||||
- ✅ **Multi-choice responses** - All choices in the response
|
||||
- ✅ **Tool/function calls** - Arguments passed to tools and functions
|
||||
- ✅ **Content lists** - Mixed content types (text, images, etc.)
|
||||
|
||||
#### Complete Example
|
||||
### Custom Violation Messages
|
||||
|
||||
```yaml
|
||||
guardrails:
|
||||
- guardrail_name: "panw-production-security"
|
||||
- guardrail_name: "panw-custom-message"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
mode: "post_call" # Scan input and output
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
profile_name: "production-profile"
|
||||
mask_request_content: true # Mask sensitive prompts
|
||||
mask_response_content: true # Mask sensitive responses
|
||||
violation_message_template: "Your request was blocked by our AI Security Policy."
|
||||
|
||||
- guardrail_name: "panw-detailed-message"
|
||||
litellm_params:
|
||||
guardrail: panw_prisma_airs
|
||||
api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
|
||||
violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
|
||||
```
|
||||
|
||||
## Use Cases
|
||||
**Supported Placeholders:** `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}`
|
||||
|
||||
From [official Prisma AIRS documentation](https://docs.paloaltonetworks.com/ai-runtime-security/activation-and-onboarding/ai-runtime-security-api-intercept-overview):
|
||||
## Behavior and Limitations
|
||||
|
||||
- **Secure AI models in production**: Validate prompt requests and responses to protect deployed AI models
|
||||
- **Detect data poisoning**: Identify contaminated training data before fine-tuning
|
||||
- **Protect against adversarial input**: Safeguard AI agents from malicious inputs and outputs
|
||||
- **Prevent sensitive data leakage**: Use API-based threat detection to block sensitive data leaks
|
||||
### Transaction Tracking
|
||||
|
||||
For standard request/response scans, `tr_id` maps to `litellm_call_id`. MCP tool scans use the parent `litellm_call_id` when available; if missing, PANW synthesizes a fallback MCP transaction ID. The real limitation is correlation loss — synthesized MCP `tr_id` values are not grouped with the parent request's prompt/response scans in AIRS dashboards.
|
||||
|
||||
By default, LiteLLM generates a UUID for `litellm_call_id`. To provide your own:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-H "x-litellm-call-id: my-custom-call-id-789" \
|
||||
-d '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"messages": [{"role": "user", "content": "capital of France"}],
|
||||
"guardrails": ["panw-prisma-airs-guardrail"]
|
||||
}'
|
||||
```
|
||||
|
||||
The `x-litellm-call-id` is also returned in response headers. If you pass `litellm_trace_id` in request metadata (or via the `x-litellm-trace-id` header), it is included in the PANW API payload metadata but does not affect `tr_id` or appear in Prisma AIRS.
|
||||
|
||||
### Streaming
|
||||
|
||||
- Response masking works on OpenAI chat streaming (`mask_response_content: true`)
|
||||
- `/v1/messages` and `/v1/responses` raw streaming blocks instead of masking when violations are detected
|
||||
- Request-side masking (`mask_request_content`) is unaffected by endpoint type
|
||||
- When `fallback_on_error: "allow"` is set, streaming responses fail open on transient PANW API errors (timeout, 5xx, network) — original chunks are yielded unchanged
|
||||
|
||||
## MCP Tool Security
|
||||
|
||||
Tool invocations are sent to AIRS as structured `tool_event` payloads containing tool name, ecosystem, and serialized arguments. Tool-event scans always use request mode.
|
||||
|
||||
**What is scanned:** LLM-driven `tool_calls` (name + arguments) and MCP request-side invocations when `mcp_tool_name` (or fallback `name`) is present. Response-side OpenAI-compatible `tool_calls` are also scanned when surfaced into `apply_guardrail()`.
|
||||
|
||||
**What is not scanned:** Tool definitions in `inputs["tools"]` and post-MCP tool results (no `post_mcp_call` hook exists yet).
|
||||
|
||||
|
||||
## Next Steps
|
||||
### Current Limitations
|
||||
|
||||
- Configure your security policies in [Strata Cloud Manager](https://apps.paloaltonetworks.com/)
|
||||
- Review the [Prisma AIRS API documentation](https://pan.dev/airs/) for advanced features
|
||||
- Set up monitoring and alerting for threat detections in your PANW dashboard
|
||||
- Consider implementing both pre_call and post_call guardrails for comprehensive protection
|
||||
- Monitor detection events and tune your security profiles based on your application needs
|
||||
- **No post-MCP response scanning.** Actual post-MCP tool-result scanning is not supported because there is no `post_mcp_call` hook in the framework. Response-side MCP events are only scanned when they appear as regular `tool_calls` in the LLM response.
|
||||
- **Guardrail selection not inherited by MCP sub-calls.** With `default_on: false`, MCP request-side child-call scans can be skipped because the parent request's guardrail selection is not propagated to the synthetic MCP payload. Workaround: use a dedicated guardrail with `mode: pre_mcp_call` and `default_on: true`.
|
||||
- **MCP transaction correlation.** MCP tool scans use the parent `litellm_call_id` when available; otherwise a fallback ID is synthesized and will not be grouped with the parent request in AIRS dashboards.
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ def initialize_guardrail(litellm_params: "LitellmParams", guardrail: "Guardrail"
|
|||
|
||||
_panw_callback = PanwPrismaAirsHandler(
|
||||
**{
|
||||
**litellm_params.model_dump(),
|
||||
**litellm_params.model_dump(exclude_unset=True),
|
||||
"guardrail_name": guardrail_name,
|
||||
"event_hook": litellm_params.mode,
|
||||
"default_on": litellm_params.default_on or False,
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -52,6 +52,13 @@ class PanwPrismaAirsGuardrailConfigModel(GuardrailConfigModel):
|
|||
description="PANW API call timeout in seconds (1-60).",
|
||||
)
|
||||
|
||||
experimental_use_latest_role_message_only: Optional[bool] = Field(
|
||||
default=None,
|
||||
description="Anthropic /v1/messages only. When unset: scans only latest user/developer "
|
||||
"message on request side. Set false to scan all user/system/developer messages. "
|
||||
"Non-Anthropic unaffected.",
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def ui_friendly_name() -> str:
|
||||
return "PANW Prisma AIRS"
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Reference in a new issue