litellm/tests/test_litellm/proxy/guardrails
yuneng-jiang 39a580aa91
test(guardrails): stop five guardrail test files leaking env vars on failure (#37828)
Onyx, prompt security, hiddenlayer, repelloai and deepkeep all write straight to
os.environ and unset again at the bottom of each test. None of the five has a
try/finally, so the moment a test fails it returns to the runner with the keys
still set and whatever runs next in that worker inherits them.

Raising inside test_onyx_guard_with_custom_timeout_from_kwargs on the current
files leaves ONYX_API_BASE and ONYX_API_KEY behind; doing the same in
test_hiddenlayer_config_saas leaves HIDDENLAYER_API_BASE. Both come back clean
after this.

89 raw writes and the hand-rolled deletes become monkeypatch calls. The
class-level setup_method and teardown_method pair in the onyx file, sweeping the
same three keys twice, becomes one autouse fixture. The sys.path.insert lines
and their now-unused imports go too, and litellm.set_verbose = True, which only
turned global debug logging on for whatever ran next, is dropped rather than
restored.

test_onyx_guard_config and test_prompt_security_guard_config asserted nothing at
all, so they could only fail by raising. Each now pins what init_guardrails_v2
produces: exactly one guardrail of the right class on litellm.callbacks,
carrying the configured name, default_on and hook. The zero-assert tests in the
other three are left alone; those are a judgement about each guardrail rather
than a mechanical sweep.

tests/test_litellm/proxy/guardrails passes at 2873.
2026-08-21 21:29:31 -07:00
..
guardrail_hooks test(guardrails): stop five guardrail test files leaking env vars on failure (#37828) 2026-08-21 21:29:31 -07:00
test_content_filter_path_traversal.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_content_utils.py fix(guardrails): walk custom_tool_call_output items in _content_utils (#32969) 2026-07-13 09:27:57 -07:00
test_custom_code_security.py fix(guardrails): return HTTP 400 for litellm content filter blocks (#28418) 2026-05-30 14:28:46 -07:00
test_deferred_guardrail_logging.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_guardrail_coverage.py fix(tests): use canonical litellm_enterprise import path (#27699) 2026-05-12 12:32:57 -07:00
test_guardrail_endpoints.py fix(bedrock_guardrails): skip ApplyGuardrail when there is no content to scan (#36441) 2026-08-11 17:30:54 -07:00
test_guardrail_registry.py fix(guardrails): refuse scan_only_tool_results combos that scan nothing 2026-08-06 01:58:06 -07:00
test_init_guardrails.py fix(guardrails): chunk oversized Bedrock ApplyGuardrail requests instead of failing (#36119) 2026-08-07 19:44:24 -07:00
test_llm_as_a_judge.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_mcp_jwt_signer.py fix(mcp): JWT on tools/list and REST tools/call server resolution (#28227) 2026-05-20 13:31:44 -07:00
test_pillar_guardrails.py ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
test_prompt_security_guardrails.py test(guardrails): stop five guardrail test files leaking env vars on failure (#37828) 2026-08-21 21:29:31 -07:00
test_qostodian_nexus_guardrail.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_usage_endpoints.py fix(guardrails): reject non-canonical date formats in usage windows 2026-08-18 15:46:09 -07:00
test_usage_tracking.py fix(guardrails): requeue usage rollup rows dropped after retry exhaustion 2026-08-18 15:51:56 -07:00