litellm/tests/guardrails_tests
ryan-crabbe-berri 680bcfd8aa
test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731)
* test(lint): ban blind pytest.raises(Exception) with ruff B017

A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.

All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.

Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.

* fix(test): widen the openai batch-dispatch assertion to OpenAIError

The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
2026-08-20 18:09:42 -07:00
..
conftest.py fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
test_akto_guardrails.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_bedrock_guardrails.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_custom_guardrail.py [Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook (#10840) 2025-05-14 17:41:33 -07:00
test_deepkeep_guardrails.py feat: add deepkeep as custom guardrail (#33844) 2026-07-20 19:27:40 -07:00
test_dynamoai_guardrails.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_eu_ai_act_article5.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_eu_ai_act_french_3_scenarios.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_guardrail_load_balancing.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_guardrails_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_javelin_guardrails.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lakera_v2.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_lasso_guardrails.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_presidio_pii.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_semantic_guard.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_sg_mas_ai_guardrails.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_sg_pdpa_guardrails.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_tracing_guardrails.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_zscaler_ai_guard.py fix(guardrails): honor configured timeout in Zscaler AI Guard (#36110) 2026-08-07 00:25:52 +00:00