mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
220bbfc0ce
* feat(guardrails): add Airia guardrail integration Adds `airia` as a built-in guardrail provider. The hook sends the prompt under `pre_call` and the model's response under `post_call` to the Airia AI Gateway, which answers allow, block, or redacted content; the only configuration is `api_base`, `api_key`, and an optional `timeout`. `during_call` is not offered: it runs concurrently with the model call, so a block could land after the prompt has already reached the provider. A BLOCKED verdict, or any action this version does not recognise, raises with `blocked_content=True`; a transport error, a non-2xx, or a response body that is not a JSON object raises with `blocked_content=False`, so callers can tell "could not evaluate" from "evaluated and blocked" while both still fail closed. On GUARDRAIL_INTERVENED a copy of the inputs is returned with every rewritten field substituted; an intervention carrying no applicable rewrite, a rewrite of the wrong shape, or a rewrite whose item count does not match what was sent (which would misalign the positional write-back downstream) blocks instead of letting the original through. Streamed responses are moderated whole and then emitted redacted: the hook opts into the unified hook's `incremental_diff` mode (the default `block_only` drops rewrites) with `streaming_end_of_stream_only`, so a redaction can never span transform rounds and underflow. Covered by a test that drives the real streaming iterator hook end to end and checks the assembled client output, not just the guardrail's own config attributes. The HTTP client is dependency-injected (an `async_handler` constructor parameter, defaulting to a real client) rather than mutated post-construction, so tests inject a fake client instead of monkeypatching an instance method. Known limitation, not fixable from this guardrail alone: LiteLLM's unified guardrail framework yields a tool-call chunk to the client immediately during `incremental_diff` streaming, inspecting the assembled tool calls against the guardrail's verdict only once the stream ends (`_run_incremental_transform_stream` in `unified_guardrail.py`). A blocked tool call can therefore reach an executing client before the terminal error arrives. Switching to `block_only` with `streaming_buffer_until_moderated` does not close this without a regression: that flag has no effect under `incremental_diff` (the mode branches away before it is read), and `block_only` itself replays the original, unredacted chunks on release, silently dropping any REDACT verdict on streamed text. Relatedly, LiteLLM's OpenAI chat completions guardrail translation skips `post_call` guardrails entirely for a response with no text content, so a tool-call-only response bypasses this guardrail (and every other post-call guardrail) even outside streaming. Both are shared-framework behavior, not specific to Airia; fixing either is out of scope for an isolated single-guardrail contribution. * fix(guardrails): fail closed on an Airia rewrite for a field that was not sent Bugbot finding on #40784. An INTERVENED verdict carrying `structured_messages: []` with `texts` omitted passed `_rewritten` unchanged: an empty list has the same length as an omitted field, so it counted as an applied rewrite while the `texts` that actually carried the content went through untouched. post_call sends only `texts`, so this was reachable. A rewrite for a field the request did not send is now a mismatch, same as a wrong length, and the verdict blocks. Regression test covers both the empty and the non-empty case. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(guardrails): fail closed on streamed non-chat routes the Airia rewrite cannot reach Veria finding on #40784. incremental_diff, the mode that carries a rewrite into the stream, exists only for OpenAI chat completions; UnifiedLLMGuardrails sends every other streamed route (/v1/responses, /v1/messages) to block_only, which streams the original text live and drops rewrites. So with post_call on, a streamed Responses or Anthropic call could receive text that Airia then blocked or redacted. Two changes, both confined to this guardrail: - streaming_buffer_until_moderated is set, so on the block_only fallback the framework holds every chunk until the end-of-stream verdict instead of releasing it live. The flag is never read on the incremental_diff path, so streamed chat completions are unchanged. - A REDACT verdict on a streamed response whose route cannot carry a rewrite is enforced as a block, since block_only would otherwise replay the original. The route check mirrors _resolve_transform_call_type so the two cannot disagree; an unresolvable route fails closed. Request-side rewrites and non-streamed calls are untouched. The end-to-end test drives the real UnifiedLLMGuardrails iterator over an Anthropic /v1/messages SSE stream and asserts not one original chunk is released for either verdict. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(guardrails): keep the Airia stream test's pytest.raises block to one statement CI's test-tree ruff run flags PT012 on the non-chat streaming test: the `async for` that collected released chunks sat inside `pytest.raises`, so a raise from the loop body would be indistinguishable from the one the test is about. The loop now lives in a `_drain_into` helper and the `with` block awaits it alone. Same assertion, same partial-leak capture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(guardrails): keep the Airia guardrail within the per-rule basedpyright ceilings scripts/type_check_gate.py now gates every rule on its merge-base count (#40193), and the guardrail added sixteen strict-rule errors it previously absorbed under the global budget. - The parsed gateway response goes through `_as_object` once, so `action`, `blocked_reason` and the rewrite fields are read from `dict[str, object]` rather than `dict[Unknown, Unknown]`; `blocked_reason` is only used as the message when it is a non-empty string. - `get_config_model` returns the concrete `type[AiriaGuardrailConfigModel]`, and the config model carries the same pyright ignore as its siblings for the unparameterised base. - The remaining untyped upstream members (the decorator, the client factory, `super().__init__`'s kwargs, the handler's `post`) are ignored per line with the reason, as the Wingback guardrail does. The registry reads `guardrail["guardrail_name"]`, a required TypedDict key. - Tests reach the injected mock and the handler timeout through typed helpers instead of the real handler's partially typed attributes. No behaviour change: 46/46 tests, ruff default and strict, and both lint gates pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(guardrails): clear the Airia guardrail's type-discipline gate violations Drop the mutable-ok comments that suppress nothing (LIT013), give the two deferred-import noqa lines a reason (LIT003), and annotate the parsed JSON object as Mapping instead of dict (LIT001). --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| guardrail_hooks | ||
| __init__.py | ||
| test_auto_router_compression.py | ||
| test_content_filter_path_traversal.py | ||
| test_content_utils.py | ||
| test_custom_code_security.py | ||
| test_deferred_guardrail_logging.py | ||
| test_guardrail_coverage.py | ||
| test_guardrail_endpoints.py | ||
| test_guardrail_initializers.py | ||
| test_guardrail_load_balancing.py | ||
| test_guardrail_registry.py | ||
| test_init_guardrails.py | ||
| test_llm_as_a_judge.py | ||
| test_mcp_jwt_signer.py | ||
| test_pillar_guardrails.py | ||
| test_prompt_security_guardrails.py | ||
| test_qostodian_nexus_guardrail.py | ||
| test_usage_endpoints.py | ||
| test_usage_tracking.py | ||