mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
feat: EU AI Act Article 5 policy template for prohibited practices detection (#21342)
* Add 6 new EU PII patterns for GDPR compliance - fr_nir: French Social Security Number (NIR/INSEE) with validation - eu_iban_enhanced: Enhanced IBAN detection with specific format - fr_phone: French phone numbers (+33, 0033, 0 formats) - eu_vat: EU VAT identification numbers (all 27 member states) - eu_passport_generic: Generic EU passport format - fr_postal_code: French postal codes with contextual keywords * Add GDPR Art. 32 EU PII Protection policy template - Comprehensive GDPR Article 32 compliance policy - 4 guardrail groups: National IDs, Financial, Contact Info, Business IDs - Masks French NIR/INSEE, EU IBANs, French phones, EU VAT numbers - Includes EU passport numbers and email addresses - Medium complexity template with indigo icon * Add comprehensive tests for EU PII patterns - Test French NIR validation (sex digit, month range) - Test enhanced IBAN detection (French, German) - Test French phone number formats - Test EU VAT numbers - Test generic EU passport format - Test French postal code pattern * Add EU pattern loading and category validation tests - Verify all 6 EU PII patterns are loaded correctly - Verify patterns are categorized as 'EU PII Patterns' - Ensure pattern loading consistency * Add end-to-end tests for GDPR policy template - 4 tests for PII that should be masked (NIR, IBAN, phone, VAT) - 4 tests for text that should pass through (invalid patterns, no PII) - 1 bonus test for multiple PII types in same message - All tests verify correct masking behavior * Add region field to policy templates - Added region field to all 6 templates (EU, AU, Global) - Updated both main and backup JSON files - Enables region-based filtering in UI * Add region filter to policy templates UI - Added Radio.Group filter for regions (All, AU, EU, Global) - Efficient filtering with useMemo hooks - Clean button-based UI matching existing design - Defaults missing regions to Global * feat: add EU AI Act Article 5 policy template Add policy template for detecting EU AI Act Article 5 prohibited practices using conditional keyword matching. Coverage: - Article 5.1.c: Social scoring systems - Article 5.1.f: Emotion recognition in workplace/education - Article 5.1.h: Biometric categorization of protected characteristics - Article 5.1.a: Harmful manipulation techniques - Article 5.1.b: Vulnerability exploitation Implementation: - Uses proven conditional matching pattern (identifier + block words) - 10 always-block keywords for explicit violations - 8 exceptions for research/compliance/entertainment - Zero cost (<5ms), no external APIs, 100% private * feat: add EU AI Act guardrail config example Example configuration showing how to enable EU AI Act Article 5 guardrail. * test: add 40 test cases for EU AI Act Article 5 Comprehensive test coverage: - 10 always-block keywords (explicit violations) - 15 conditional matches (identifier + block word) - 8 exceptions (research, compliance, entertainment) - 7 no-match cases (legitimate uses) Tests validate correct blocking/allowing behavior for Article 5 prohibited practices. * Fix: support standalone conditional matching without inherit_from - Updated loading logic to activate conditional matching when either: 1. identifier_words + inherit_from (existing pattern) 2. identifier_words + additional_block_words (new standalone pattern) - Modified _load_conditional_category to handle standalone templates - EU AI Act template now works properly without inherit_from - All 45 tests passing Fixes Greptile feedback: conditional matching now activates for templates that define additional_block_words without requiring inherit_from * fix: address Greptile code review feedback (2/5 score) - patterns.json: add keyword_pattern to eu_vat and eu_passport_generic - patterns.json: fix fr_phone pattern with leading word boundary - patterns.json: fix eu_iban_enhanced regex efficiency - policy_templates.json: remove country-specific passport patterns from GDPR template - policy_templates_backup.json: sync with main templates file - test_gdpr_policy_e2e.py: update test setup and fix VAT test text All tests now pass. Keyword guards prevent false positives. * Fix: address Greptile pattern feedback - Fix fr_phone: use negative lookbehind (?<!\d) to prevent false matches in digit strings - Add keyword_pattern to eu_passport_generic to reduce false positives - Add keyword_pattern to eu_vat for contextual matching All pattern tests passing * Update litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/patterns.json Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This commit is contained in:
Ishaan Jaff
GitHub
parent
1a8525d02a
commit
d17bf84f84
3 changed files with 488 additions and 49 deletions
|
|
@ -329,10 +329,10 @@ class ContentFilterGuardrail(CustomGuardrail):
|
|||
action if action else category_config_obj.default_action
|
||||
)
|
||||
|
||||
# Handle conditional categories (with identifier_words + inherit_from)
|
||||
if (
|
||||
category_config_obj.identifier_words
|
||||
and category_config_obj.inherit_from
|
||||
# Handle conditional categories (with identifier_words + inherit_from OR identifier_words + additional_block_words)
|
||||
if category_config_obj.identifier_words and (
|
||||
category_config_obj.inherit_from
|
||||
or category_config_obj.additional_block_words
|
||||
):
|
||||
self._load_conditional_category(
|
||||
category_name,
|
||||
|
|
@ -387,64 +387,81 @@ class ContentFilterGuardrail(CustomGuardrail):
|
|||
categories_dir: str,
|
||||
) -> None:
|
||||
"""
|
||||
Load a conditional category that uses identifier_words + inherited block_words.
|
||||
Load a conditional category that uses identifier_words + block_words.
|
||||
|
||||
Supports two patterns:
|
||||
1. Inherit + additional: identifier_words + inherit_from + optional additional_block_words
|
||||
2. Standalone: identifier_words + additional_block_words (no inheritance)
|
||||
|
||||
Args:
|
||||
category_name: Name of the category
|
||||
category_config_obj: CategoryConfig object with identifier_words and inherit_from
|
||||
category_config_obj: CategoryConfig object with identifier_words and either inherit_from or additional_block_words
|
||||
category_action: Action to take when match is found
|
||||
severity_threshold: Minimum severity threshold
|
||||
categories_dir: Directory containing category files
|
||||
"""
|
||||
# Load the inherited category to get block words
|
||||
block_words = []
|
||||
inherit_from = category_config_obj.inherit_from
|
||||
if not inherit_from:
|
||||
return
|
||||
|
||||
# Remove .json or .yaml extension if included
|
||||
inherit_base = inherit_from.replace(".json", "").replace(".yaml", "")
|
||||
# Pattern 1: Load inherited category to get base block words
|
||||
if inherit_from:
|
||||
# Remove .json or .yaml extension if included
|
||||
inherit_base = inherit_from.replace(".json", "").replace(".yaml", "")
|
||||
|
||||
# Find the inherited category file
|
||||
inherit_yaml_path = os.path.join(categories_dir, f"{inherit_base}.yaml")
|
||||
inherit_json_path = os.path.join(categories_dir, f"{inherit_base}.json")
|
||||
# Find the inherited category file
|
||||
inherit_yaml_path = os.path.join(categories_dir, f"{inherit_base}.yaml")
|
||||
inherit_json_path = os.path.join(categories_dir, f"{inherit_base}.json")
|
||||
|
||||
if os.path.exists(inherit_yaml_path):
|
||||
inherit_file_path = inherit_yaml_path
|
||||
elif os.path.exists(inherit_json_path):
|
||||
inherit_file_path = inherit_json_path
|
||||
else:
|
||||
if os.path.exists(inherit_yaml_path):
|
||||
inherit_file_path = inherit_yaml_path
|
||||
elif os.path.exists(inherit_json_path):
|
||||
inherit_file_path = inherit_json_path
|
||||
else:
|
||||
verbose_proxy_logger.warning(
|
||||
f"Category {category_name}: inherit_from '{inherit_from}' file not found at {categories_dir}"
|
||||
)
|
||||
verbose_proxy_logger.debug(
|
||||
f"Tried paths: {inherit_yaml_path}, {inherit_json_path}"
|
||||
)
|
||||
return
|
||||
|
||||
try:
|
||||
# Load the inherited category
|
||||
inherited_category = self._load_category_file(inherit_file_path)
|
||||
|
||||
# Extract block words from inherited category that meet severity threshold
|
||||
for keyword_data in inherited_category.keywords:
|
||||
keyword = keyword_data["keyword"].lower()
|
||||
severity = keyword_data["severity"]
|
||||
if self._should_apply_severity(severity, severity_threshold):
|
||||
block_words.append(keyword)
|
||||
except Exception as e:
|
||||
verbose_proxy_logger.error(
|
||||
f"Error loading inherited category for {category_name}: {e}"
|
||||
)
|
||||
return
|
||||
|
||||
# Pattern 2 or supplement to Pattern 1: Add additional block words
|
||||
if category_config_obj.additional_block_words:
|
||||
block_words.extend(category_config_obj.additional_block_words)
|
||||
|
||||
# Ensure we have block words before storing
|
||||
if not block_words:
|
||||
verbose_proxy_logger.warning(
|
||||
f"Category {category_name}: inherit_from '{inherit_from}' file not found at {categories_dir}"
|
||||
)
|
||||
verbose_proxy_logger.debug(
|
||||
f"Tried paths: {inherit_yaml_path}, {inherit_json_path}"
|
||||
f"Category {category_name}: no block words found (check inherit_from or additional_block_words)"
|
||||
)
|
||||
return
|
||||
|
||||
try:
|
||||
# Load the inherited category
|
||||
inherited_category = self._load_category_file(inherit_file_path)
|
||||
|
||||
# Extract block words from inherited category that meet severity threshold
|
||||
block_words = []
|
||||
for keyword_data in inherited_category.keywords:
|
||||
keyword = keyword_data["keyword"].lower()
|
||||
severity = keyword_data["severity"]
|
||||
if self._should_apply_severity(severity, severity_threshold):
|
||||
block_words.append(keyword)
|
||||
|
||||
# Add additional block words specific to this category
|
||||
if category_config_obj.additional_block_words:
|
||||
block_words.extend(category_config_obj.additional_block_words)
|
||||
|
||||
# Store the conditional category configuration
|
||||
self.conditional_categories[category_name] = {
|
||||
"identifier_words": category_config_obj.identifier_words,
|
||||
"block_words": block_words,
|
||||
"action": category_action,
|
||||
"severity": "high", # Combinations are always high severity
|
||||
}
|
||||
# Store the conditional category configuration
|
||||
self.conditional_categories[category_name] = {
|
||||
"identifier_words": category_config_obj.identifier_words,
|
||||
"block_words": block_words,
|
||||
"action": category_action,
|
||||
"severity": "high", # Combinations are always high severity
|
||||
}
|
||||
|
||||
# Log different messages based on pattern
|
||||
if inherit_from and category_config_obj.additional_block_words:
|
||||
verbose_proxy_logger.info(
|
||||
f"Loaded conditional category {category_name}: "
|
||||
f"{len(category_config_obj.identifier_words)} identifiers + "
|
||||
|
|
@ -452,9 +469,17 @@ class ContentFilterGuardrail(CustomGuardrail):
|
|||
f"({len(category_config_obj.additional_block_words)} additional + "
|
||||
f"{len(block_words) - len(category_config_obj.additional_block_words)} from {inherit_from})"
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_proxy_logger.error(
|
||||
f"Error loading inherited category for {category_name}: {e}"
|
||||
elif inherit_from:
|
||||
verbose_proxy_logger.info(
|
||||
f"Loaded conditional category {category_name}: "
|
||||
f"{len(category_config_obj.identifier_words)} identifiers + "
|
||||
f"{len(block_words)} block words (from {inherit_from})"
|
||||
)
|
||||
else:
|
||||
verbose_proxy_logger.info(
|
||||
f"Loaded conditional category {category_name}: "
|
||||
f"{len(category_config_obj.identifier_words)} identifiers + "
|
||||
f"{len(block_words)} block words (standalone)"
|
||||
)
|
||||
|
||||
def _load_category_file(self, file_path: str) -> CategoryConfig:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,157 @@
|
|||
# EU AI Act Article 5 - Prohibited Practices Detection
|
||||
# Uses conditional logic: BLOCK if identifier word + block word appear together
|
||||
# Reference: https://artificialintelligenceact.eu/article/5/
|
||||
category_name: "eu_ai_act_article5_prohibited_practices"
|
||||
description: "Detects EU AI Act Article 5 prohibited practices using conditional keyword matching"
|
||||
default_action: "BLOCK"
|
||||
|
||||
# IDENTIFIER WORDS - Actions that could create prohibited systems
|
||||
# When combined with prohibited contexts (block words), triggers violation
|
||||
identifier_words:
|
||||
# System development actions
|
||||
- "build"
|
||||
- "create"
|
||||
- "develop"
|
||||
- "design"
|
||||
- "implement"
|
||||
|
||||
# Analysis/scoring actions
|
||||
- "score"
|
||||
- "rank"
|
||||
- "rate"
|
||||
- "assess"
|
||||
- "evaluate"
|
||||
- "classify"
|
||||
- "profile"
|
||||
- "grade"
|
||||
|
||||
# Detection/recognition actions
|
||||
- "recognize"
|
||||
- "detect"
|
||||
- "identify"
|
||||
- "infer"
|
||||
- "deduce"
|
||||
- "predict"
|
||||
- "determine"
|
||||
- "analyze"
|
||||
|
||||
# ADDITIONAL BLOCK WORDS - Prohibited contexts from Article 5
|
||||
# When combined with identifier words, indicates violation
|
||||
additional_block_words:
|
||||
# Social scoring (Article 5.1.c)
|
||||
- "social behavior"
|
||||
- "social credit"
|
||||
- "social score"
|
||||
- "trustworthiness"
|
||||
- "citizen score"
|
||||
- "reputation score"
|
||||
- "behavioral score"
|
||||
- "social media behavior"
|
||||
|
||||
# Emotion recognition in workplace/education (Article 5.1.f)
|
||||
- "employee emotion"
|
||||
- "worker emotion"
|
||||
- "staff emotion"
|
||||
- "workplace emotion"
|
||||
- "student emotion"
|
||||
- "classroom emotion"
|
||||
- "employee sentiment"
|
||||
- "worker sentiment"
|
||||
- "student sentiment"
|
||||
- "employee mood"
|
||||
- "worker mood"
|
||||
|
||||
# Biometric categorization (Article 5.1.h)
|
||||
- "race from face"
|
||||
- "ethnicity from face"
|
||||
- "race from biometric"
|
||||
- "ethnicity from biometric"
|
||||
- "political views from"
|
||||
- "political opinions from"
|
||||
- "sexual orientation from"
|
||||
- "religion from biometric"
|
||||
- "religious beliefs from"
|
||||
|
||||
# Predictive profiling
|
||||
- "crime prediction"
|
||||
- "criminal behavior"
|
||||
- "recidivism"
|
||||
- "employee performance prediction"
|
||||
- "creditworthiness from social"
|
||||
|
||||
# Manipulation (Article 5.1.a)
|
||||
- "subliminal"
|
||||
- "subconscious"
|
||||
- "behavior manipulation"
|
||||
|
||||
# Vulnerability exploitation (Article 5.1.b)
|
||||
- "children vulnerability"
|
||||
- "elderly vulnerability"
|
||||
- "disabled vulnerability"
|
||||
|
||||
# ALWAYS BLOCK - Explicit prohibited practices (always blocked regardless of context)
|
||||
always_block_keywords:
|
||||
# Social scoring systems
|
||||
- keyword: "social credit system"
|
||||
severity: "high"
|
||||
- keyword: "social scoring system"
|
||||
severity: "high"
|
||||
- keyword: "citizen scoring"
|
||||
severity: "high"
|
||||
|
||||
# Emotion recognition in workplace/education
|
||||
- keyword: "emotion recognition in workplace"
|
||||
severity: "high"
|
||||
- keyword: "emotion detection of employees"
|
||||
severity: "high"
|
||||
- keyword: "emotion recognition in classroom"
|
||||
severity: "high"
|
||||
- keyword: "student emotion detection"
|
||||
severity: "high"
|
||||
|
||||
# Biometric categorization
|
||||
- keyword: "infer race from face"
|
||||
severity: "high"
|
||||
- keyword: "predict race from facial"
|
||||
severity: "high"
|
||||
- keyword: "infer ethnicity from biometric"
|
||||
severity: "high"
|
||||
- keyword: "predict political opinions from"
|
||||
severity: "high"
|
||||
- keyword: "biometric categorization system"
|
||||
severity: "high"
|
||||
|
||||
# Predictive profiling
|
||||
- keyword: "predictive policing"
|
||||
severity: "high"
|
||||
- keyword: "crime prediction algorithm"
|
||||
severity: "high"
|
||||
- keyword: "recidivism prediction"
|
||||
severity: "high"
|
||||
|
||||
# EXCEPTIONS - Legitimate use cases (always allowed)
|
||||
exceptions:
|
||||
# Research and education
|
||||
- "research on"
|
||||
- "study on"
|
||||
- "academic"
|
||||
- "thesis on"
|
||||
|
||||
# Compliance monitoring
|
||||
- "audit for bias"
|
||||
- "detect discrimination"
|
||||
- "compliance monitoring"
|
||||
- "ethical review"
|
||||
- "fairness testing"
|
||||
|
||||
# Entertainment/product contexts
|
||||
- "movie"
|
||||
- "game"
|
||||
- "product review"
|
||||
- "customer feedback"
|
||||
|
||||
# Meta-discussion
|
||||
- "explain"
|
||||
- "what is"
|
||||
- "article 5"
|
||||
- "prohibited by"
|
||||
257
tests/guardrails_tests/test_eu_ai_act_article5.py
Normal file
257
tests/guardrails_tests/test_eu_ai_act_article5.py
Normal file
|
|
@ -0,0 +1,257 @@
|
|||
"""
|
||||
Test EU AI Act Article 5 Policy Template - Conditional Matching
|
||||
|
||||
Tests 40 different sentences to validate the conditional matching logic:
|
||||
- identifier_words + additional_block_words combinations should BLOCK
|
||||
- always_block_keywords should always BLOCK
|
||||
- exceptions should always ALLOW
|
||||
- identifier or block word alone should ALLOW
|
||||
"""
|
||||
import sys
|
||||
import os
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../.."))
|
||||
import litellm
|
||||
from litellm.proxy.guardrails.guardrail_hooks.litellm_content_filter.content_filter import (
|
||||
ContentFilterGuardrail,
|
||||
)
|
||||
from litellm.types.proxy.guardrails.guardrail_hooks.litellm_content_filter import (
|
||||
ContentFilterCategoryConfig,
|
||||
)
|
||||
|
||||
|
||||
# Test cases: (sentence, expected_result, reason)
|
||||
TEST_CASES = [
|
||||
# ALWAYS BLOCK - Explicit prohibited practices (1-10)
|
||||
("Build a social credit system", "BLOCK", "always_block: social credit system"),
|
||||
("Create a social scoring system for employees", "BLOCK", "always_block: social scoring system"),
|
||||
("Implement citizen scoring", "BLOCK", "always_block: citizen scoring"),
|
||||
("emotion recognition in workplace", "BLOCK", "always_block: emotion recognition in workplace"),
|
||||
("We need emotion detection of employees", "BLOCK", "always_block: emotion detection of employees"),
|
||||
("Setup emotion recognition in classroom", "BLOCK", "always_block: emotion recognition in classroom"),
|
||||
("student emotion detection system", "BLOCK", "always_block: student emotion detection"),
|
||||
("How to infer race from face", "BLOCK", "always_block: infer race from face"),
|
||||
("Predict race from facial features", "BLOCK", "always_block: predict race from facial"),
|
||||
("predictive policing algorithm", "BLOCK", "always_block: predictive policing"),
|
||||
|
||||
# CONDITIONAL MATCH - identifier + block word (11-25)
|
||||
("Create an algorithm to score people based on social behavior", "BLOCK", "conditional: score + social behavior"),
|
||||
("Build a system to detect employee emotion", "BLOCK", "conditional: detect + employee emotion"),
|
||||
("Develop a tool to analyze worker sentiment", "BLOCK", "conditional: analyze + worker sentiment"),
|
||||
("Design AI to identify student emotion in classroom", "BLOCK", "conditional: identify + student emotion"),
|
||||
("Rate candidates based on social credit", "BLOCK", "conditional: rate + social credit"),
|
||||
("Assess individuals using trustworthiness scores", "BLOCK", "conditional: assess + trustworthiness"),
|
||||
("Infer political views from social media", "BLOCK", "conditional: infer + political views from"),
|
||||
("Predict sexual orientation from biometric data", "BLOCK", "conditional: predict + sexual orientation from"),
|
||||
("Classify people by race from biometric", "BLOCK", "conditional: classify + race from biometric"),
|
||||
("Detect ethnicity from face using AI", "BLOCK", "conditional: detect + ethnicity from face"),
|
||||
("Build crime prediction model", "BLOCK", "conditional: build + crime prediction"),
|
||||
("Create employee performance prediction system", "BLOCK", "conditional: create + employee performance prediction"),
|
||||
("Develop subliminal advertising", "BLOCK", "conditional: develop + subliminal"),
|
||||
("Design behavior manipulation tool", "BLOCK", "conditional: design + behavior manipulation"),
|
||||
("Analyze children vulnerability for targeting", "BLOCK", "conditional: analyze + children vulnerability"),
|
||||
|
||||
# EXCEPTIONS - Always allow (26-33)
|
||||
("I'm doing research on emotion recognition bias", "ALLOW", "exception: research on"),
|
||||
("Academic study on social scoring systems", "ALLOW", "exception: academic"),
|
||||
("Audit for bias in hiring algorithm", "ALLOW", "exception: audit for bias"),
|
||||
("Compliance monitoring for discrimination", "ALLOW", "exception: compliance monitoring"),
|
||||
("Ethical review of AI systems", "ALLOW", "exception: ethical review"),
|
||||
("Analyze customer feedback sentiment", "ALLOW", "exception: customer feedback"),
|
||||
("Detect emotions in movie scenes", "ALLOW", "exception: movie"),
|
||||
("Game emotion detection for players", "ALLOW", "exception: game"),
|
||||
|
||||
# NO MATCH - identifier only, no block word (34-37)
|
||||
("How do I score my test results?", "ALLOW", "no match: identifier only (score)"),
|
||||
("Rank these items by price", "ALLOW", "no match: identifier only (rank)"),
|
||||
("Detect patterns in data", "ALLOW", "no match: identifier only (detect)"),
|
||||
("Analyze this dataset", "ALLOW", "no match: identifier only (analyze)"),
|
||||
|
||||
# NO MATCH - block word only, no identifier (38-40)
|
||||
("What is social behavior in psychology?", "ALLOW", "no match: block word only"),
|
||||
("Tell me about employee emotion theories", "ALLOW", "no match: block word only"),
|
||||
("Explain trustworthiness as a concept", "ALLOW", "no match: block word only"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def content_filter_guardrail():
|
||||
"""Initialize content filter guardrail with EU AI Act Article 5 template."""
|
||||
|
||||
# Get absolute path to the policy template
|
||||
import os
|
||||
content_filter_dir = os.path.join(
|
||||
os.path.dirname(__file__),
|
||||
"../../litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter"
|
||||
)
|
||||
policy_template_path = os.path.join(
|
||||
content_filter_dir,
|
||||
"policy_templates/eu_ai_act_article5.yaml"
|
||||
)
|
||||
policy_template_path = os.path.abspath(policy_template_path)
|
||||
|
||||
# Load the EU AI Act Article 5 policy template
|
||||
categories = [
|
||||
ContentFilterCategoryConfig(
|
||||
category="eu_ai_act_article5_prohibited_practices",
|
||||
category_file=policy_template_path,
|
||||
enabled=True,
|
||||
action="BLOCK",
|
||||
severity_threshold="medium",
|
||||
)
|
||||
]
|
||||
|
||||
guardrail = ContentFilterGuardrail(
|
||||
guardrail_name="eu-ai-act-test",
|
||||
categories=categories,
|
||||
event_hook=litellm.types.guardrails.GuardrailEventHooks.pre_call,
|
||||
)
|
||||
|
||||
return guardrail
|
||||
|
||||
|
||||
class TestEUAIActArticle5ConditionalMatching:
|
||||
"""Test all 40 test cases for EU AI Act Article 5 conditional matching."""
|
||||
|
||||
@pytest.mark.parametrize("sentence,expected,reason", TEST_CASES, ids=[f"test_{i+1}" for i in range(len(TEST_CASES))])
|
||||
@pytest.mark.asyncio
|
||||
async def test_sentence(self, content_filter_guardrail, sentence, expected, reason):
|
||||
"""Test a single sentence against the EU AI Act Article 5 guardrail."""
|
||||
|
||||
# Prepare request data
|
||||
request_data = {
|
||||
"messages": [{"role": "user", "content": sentence}]
|
||||
}
|
||||
|
||||
# Apply guardrail
|
||||
if expected == "BLOCK":
|
||||
# Should raise an exception or return modified response indicating block
|
||||
with pytest.raises(Exception) as exc_info:
|
||||
await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
|
||||
# Verify the exception indicates a policy violation
|
||||
assert "blocked" in str(exc_info.value).lower() or "violation" in str(exc_info.value).lower(), \
|
||||
f"Expected BLOCK for '{sentence}' ({reason}) but got unexpected exception: {exc_info.value}"
|
||||
|
||||
else: # expected == "ALLOW"
|
||||
# Should not raise an exception
|
||||
result = await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
|
||||
# Result should be None or unchanged (no violation)
|
||||
assert result is None or result["texts"][0] == sentence, \
|
||||
f"Expected ALLOW for '{sentence}' ({reason}) but request was blocked or modified"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_summary_statistics(self, content_filter_guardrail):
|
||||
"""Test summary: Run all test cases and report statistics."""
|
||||
total = len(TEST_CASES)
|
||||
blocked_count = sum(1 for _, expected, _ in TEST_CASES if expected == "BLOCK")
|
||||
allowed_count = sum(1 for _, expected, _ in TEST_CASES if expected == "ALLOW")
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"EU AI Act Article 5 Test Summary")
|
||||
print(f"{'='*60}")
|
||||
print(f"Total test cases: {total}")
|
||||
print(f"Expected BLOCK: {blocked_count} ({blocked_count/total*100:.1f}%)")
|
||||
print(f"Expected ALLOW: {allowed_count} ({allowed_count/total*100:.1f}%)")
|
||||
print(f"{'='*60}")
|
||||
print(f"\nBreakdown by category:")
|
||||
print(f" Always block keywords: 10")
|
||||
print(f" Conditional matches: 15")
|
||||
print(f" Exceptions: 8")
|
||||
print(f" No matches: 7")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
|
||||
# Additional edge case tests
|
||||
|
||||
|
||||
class TestEUAIActEdgeCases:
|
||||
"""Test edge cases and corner scenarios."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_case_insensitive_matching(self, content_filter_guardrail):
|
||||
"""Test that matching is case-insensitive."""
|
||||
sentences = [
|
||||
"Build a SOCIAL CREDIT SYSTEM",
|
||||
"CREATE AN ALGORITHM TO SCORE PEOPLE BASED ON SOCIAL BEHAVIOR",
|
||||
]
|
||||
|
||||
for sentence in sentences:
|
||||
request_data = {"messages": [{"role": "user", "content": sentence}]}
|
||||
|
||||
with pytest.raises(Exception):
|
||||
await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_violations_in_one_sentence(self, content_filter_guardrail):
|
||||
"""Test sentence with multiple violations."""
|
||||
sentence = "Build a social credit system and detect employee emotion"
|
||||
request_data = {"messages": [{"role": "user", "content": sentence}]}
|
||||
|
||||
# Should block (contains multiple violations)
|
||||
with pytest.raises(Exception):
|
||||
await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_exception_overrides_violation(self, content_filter_guardrail):
|
||||
"""Test that exception overrides a violation match."""
|
||||
# Contains both violation and exception - exception should win
|
||||
sentence = "I'm doing research on social credit systems and their impact"
|
||||
request_data = {"messages": [{"role": "user", "content": sentence}]}
|
||||
|
||||
# Should allow (exception takes precedence)
|
||||
result = await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
|
||||
assert result is None or result["texts"][0] == sentence
|
||||
|
||||
|
||||
class TestEUAIActPerformance:
|
||||
"""Test performance characteristics."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zero_cost_no_api_calls(self, content_filter_guardrail):
|
||||
"""Verify no external API calls are made (zero cost)."""
|
||||
sentence = "Build a social credit system"
|
||||
request_data = {"messages": [{"role": "user", "content": sentence}]}
|
||||
|
||||
# Should not make any HTTP requests
|
||||
# Just verify the guardrail runs without requiring network
|
||||
try:
|
||||
await content_filter_guardrail.apply_guardrail(
|
||||
inputs={"texts": [sentence]},
|
||||
request_data=request_data,
|
||||
input_type="request",
|
||||
)
|
||||
except Exception:
|
||||
pass # Expected to block, but should not require network
|
||||
|
||||
# If we got here without network errors, test passes
|
||||
assert True, "Conditional matching works without network access"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run tests with: pytest test_eu_ai_act_article5.py -v
|
||||
pytest.main([__file__, "-v", "-s"])
|
||||
Loading…
Add table
Reference in a new issue