mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-15 04:19:08 +00:00
220bbfc0ce
* feat(guardrails): add Airia guardrail integration Adds `airia` as a built-in guardrail provider. The hook sends the prompt under `pre_call` and the model's response under `post_call` to the Airia AI Gateway, which answers allow, block, or redacted content; the only configuration is `api_base`, `api_key`, and an optional `timeout`. `during_call` is not offered: it runs concurrently with the model call, so a block could land after the prompt has already reached the provider. A BLOCKED verdict, or any action this version does not recognise, raises with `blocked_content=True`; a transport error, a non-2xx, or a response body that is not a JSON object raises with `blocked_content=False`, so callers can tell "could not evaluate" from "evaluated and blocked" while both still fail closed. On GUARDRAIL_INTERVENED a copy of the inputs is returned with every rewritten field substituted; an intervention carrying no applicable rewrite, a rewrite of the wrong shape, or a rewrite whose item count does not match what was sent (which would misalign the positional write-back downstream) blocks instead of letting the original through. Streamed responses are moderated whole and then emitted redacted: the hook opts into the unified hook's `incremental_diff` mode (the default `block_only` drops rewrites) with `streaming_end_of_stream_only`, so a redaction can never span transform rounds and underflow. Covered by a test that drives the real streaming iterator hook end to end and checks the assembled client output, not just the guardrail's own config attributes. The HTTP client is dependency-injected (an `async_handler` constructor parameter, defaulting to a real client) rather than mutated post-construction, so tests inject a fake client instead of monkeypatching an instance method. Known limitation, not fixable from this guardrail alone: LiteLLM's unified guardrail framework yields a tool-call chunk to the client immediately during `incremental_diff` streaming, inspecting the assembled tool calls against the guardrail's verdict only once the stream ends (`_run_incremental_transform_stream` in `unified_guardrail.py`). A blocked tool call can therefore reach an executing client before the terminal error arrives. Switching to `block_only` with `streaming_buffer_until_moderated` does not close this without a regression: that flag has no effect under `incremental_diff` (the mode branches away before it is read), and `block_only` itself replays the original, unredacted chunks on release, silently dropping any REDACT verdict on streamed text. Relatedly, LiteLLM's OpenAI chat completions guardrail translation skips `post_call` guardrails entirely for a response with no text content, so a tool-call-only response bypasses this guardrail (and every other post-call guardrail) even outside streaming. Both are shared-framework behavior, not specific to Airia; fixing either is out of scope for an isolated single-guardrail contribution. * fix(guardrails): fail closed on an Airia rewrite for a field that was not sent Bugbot finding on #40784. An INTERVENED verdict carrying `structured_messages: []` with `texts` omitted passed `_rewritten` unchanged: an empty list has the same length as an omitted field, so it counted as an applied rewrite while the `texts` that actually carried the content went through untouched. post_call sends only `texts`, so this was reachable. A rewrite for a field the request did not send is now a mismatch, same as a wrong length, and the verdict blocks. Regression test covers both the empty and the non-empty case. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(guardrails): fail closed on streamed non-chat routes the Airia rewrite cannot reach Veria finding on #40784. incremental_diff, the mode that carries a rewrite into the stream, exists only for OpenAI chat completions; UnifiedLLMGuardrails sends every other streamed route (/v1/responses, /v1/messages) to block_only, which streams the original text live and drops rewrites. So with post_call on, a streamed Responses or Anthropic call could receive text that Airia then blocked or redacted. Two changes, both confined to this guardrail: - streaming_buffer_until_moderated is set, so on the block_only fallback the framework holds every chunk until the end-of-stream verdict instead of releasing it live. The flag is never read on the incremental_diff path, so streamed chat completions are unchanged. - A REDACT verdict on a streamed response whose route cannot carry a rewrite is enforced as a block, since block_only would otherwise replay the original. The route check mirrors _resolve_transform_call_type so the two cannot disagree; an unresolvable route fails closed. Request-side rewrites and non-streamed calls are untouched. The end-to-end test drives the real UnifiedLLMGuardrails iterator over an Anthropic /v1/messages SSE stream and asserts not one original chunk is released for either verdict. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(guardrails): keep the Airia stream test's pytest.raises block to one statement CI's test-tree ruff run flags PT012 on the non-chat streaming test: the `async for` that collected released chunks sat inside `pytest.raises`, so a raise from the loop body would be indistinguishable from the one the test is about. The loop now lives in a `_drain_into` helper and the `with` block awaits it alone. Same assertion, same partial-leak capture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(guardrails): keep the Airia guardrail within the per-rule basedpyright ceilings scripts/type_check_gate.py now gates every rule on its merge-base count (#40193), and the guardrail added sixteen strict-rule errors it previously absorbed under the global budget. - The parsed gateway response goes through `_as_object` once, so `action`, `blocked_reason` and the rewrite fields are read from `dict[str, object]` rather than `dict[Unknown, Unknown]`; `blocked_reason` is only used as the message when it is a non-empty string. - `get_config_model` returns the concrete `type[AiriaGuardrailConfigModel]`, and the config model carries the same pyright ignore as its siblings for the unparameterised base. - The remaining untyped upstream members (the decorator, the client factory, `super().__init__`'s kwargs, the handler's `post`) are ignored per line with the reason, as the Wingback guardrail does. The registry reads `guardrail["guardrail_name"]`, a required TypedDict key. - Tests reach the injected mock and the handler timeout through typed helpers instead of the real handler's partially typed attributes. No behaviour change: 46/46 tests, ruff default and strict, and both lint gates pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(guardrails): clear the Airia guardrail's type-discipline gate violations Drop the mutable-ok comments that suppress nothing (LIT013), give the two deferred-import noqa lines a reason (LIT003), and annotate the parsed JSON object as Mapping instead of dict (LIT001). --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
74 lines
5.9 KiB
TOML
74 lines
5.9 KiB
TOML
extend = "ruff.toml"
|
|
|
|
[lint]
|
|
preview = true
|
|
select = ["ANN", "ASYNC230", "B004", "B006", "B008", "B009", "B010", "B018", "B019", "B021", "B026", "B033", "BLE", "C401", "C404", "C405", "C408", "C414", "C419", "C901", "D419", "DTZ001", "DTZ003", "DTZ005", "DTZ006", "DTZ007", "DTZ011", "EXE001", "EXE002", "F401", "FURB136", "FURB168", "FURB188", "I001", "LOG015", "N999", "PERF102", "PERF401", "PERF402", "PERF403", "PIE790", "PIE800", "PIE804", "PIE810", "PLC0206", "PLC0208", "PLC0414", "PLR0124", "PLR0206", "PLR0402", "PLR1704", "PLR1711", "PLR1714", "PLR1730", "PLR2044", "PLW0127", "PLW0133", "PLW0602", "PLW0603", "PLW1508", "PLW1510", "PYI030", "PYI036", "PYI041", "PYI064", "RET501", "RET504", "RUF010", "RUF012", "RUF015", "RUF019", "RUF022", "RUF023", "RUF046", "RUF051", "RUF059", "RUF100", "S110", "S112", "SIM101", "SIM102", "SIM103", "SIM113", "SIM114", "SIM115", "SIM117", "SIM118", "SIM201", "SIM210", "SIM211", "SIM222", "SIM401", "TC004", "TC005", "TID251", "TRY002", "TRY004", "TRY201", "TRY203", "TRY300", "UP006", "UP007", "UP008", "UP012", "UP018", "UP024", "UP028", "UP031", "UP032", "UP034", "UP035", "UP036", "UP037", "UP045"]
|
|
extend-select = []
|
|
# Overrides the inherited list: rules this gate enforces itself must NOT be external here,
|
|
# so this config's RUF100 flags their stale `# noqa` directives. What remains external is
|
|
# only what other tooling enforces: every base ruff.toml rule this select list doesn't
|
|
# re-enable (all of the default E/F families plus T20/PGH004/RUF008/RUF009, minus the
|
|
# strict-selected F401 and RUF100; F4 is split out so stale F401 noqas stay detectable),
|
|
# plus upstream litellm's ruff config.
|
|
external = [
|
|
"T20", "PGH004", "RUF008", "RUF009", "E4", "E7", "E9",
|
|
"F402", "F404", "F406", "F407", "F5", "F6", "F7", "F8", "F9",
|
|
"PLC0415", "E402", "BLE001", "ARG002", "S102", "S324", "S606", "D401", "F403", "F405",
|
|
]
|
|
|
|
[lint.per-file-ignores]
|
|
# ANN401 (explicit `Any` disallowed) has no per-line/function-level ignore mechanism
|
|
# in ruff, only file-level. These two files each have a handful of parameters that
|
|
# are genuinely heterogeneous with no fitting concrete type: a response object that
|
|
# varies across every LLM call type (completion/embedding/transcription/etc. each
|
|
# return a different shape), and *args/**kwargs forwarded verbatim with no fixed
|
|
# shape. Tried the closest existing union (CostResponseTypes) first; basedpyright
|
|
# caught a real mismatch, confirming Any is correct here, not a shortcut.
|
|
"litellm/litellm_core_utils/litellm_logging.py" = ["ANN401"]
|
|
"litellm/utils.py" = ["ANN401"]
|
|
# `**kwargs` forwards verbatim to CustomGuardrail.__init__, whose param list is wide and
|
|
# grows over time; typing it concretely (`object`) broke that forwarding call outright —
|
|
# basedpyright turned every named param into a reportArgumentType error. Any is correct here.
|
|
"litellm/proxy/guardrails/guardrail_hooks/alice/alice.py" = ["ANN401"]
|
|
"litellm/proxy/guardrails/guardrail_hooks/airia/airia.py" = ["ANN401"]
|
|
# Same reason: `**kwargs` forwards verbatim to CustomGuardrail.__init__, and the lifecycle
|
|
# hook signatures inherit `Any` for `response` from CustomLogger, so narrowing them here
|
|
# would break the override rather than describe it.
|
|
"litellm/proxy/guardrails/guardrail_hooks/llm_shield_proxy/llm_shield_proxy.py" = ["ANN401"]
|
|
|
|
[lint.mccabe]
|
|
max-complexity = 15
|
|
|
|
[lint.pylint]
|
|
max-args = 5
|
|
|
|
[lint.flake8-tidy-imports.banned-api]
|
|
"typing.Any".msg = "Use a concrete type. Frozen slots=True dataclass (preferred) / NamedTuple / ReadOnly TypedDict for payloads."
|
|
"typing_extensions.Any".msg = "Same as typing.Any."
|
|
"typing.List".msg = "tuple[X, ...] for state, Sequence[X] for params."
|
|
"typing.Dict".msg = "Frozen dataclass / NamedTuple / ReadOnly TypedDict; if truly dynamic, use MappingProxyType."
|
|
"typing.Set".msg = "frozenset[X] or AbstractSet[X]."
|
|
"typing.MutableSequence".msg = "Sequence[X]."
|
|
"typing.MutableMapping".msg = "See typing.Dict."
|
|
# Unchecked casts: cast() lies to the type checker with no runtime guarantee.
|
|
# Validate into a concrete frozen type at the boundary (pydantic) instead.
|
|
# Per-call-site coverage lives in check_type_discipline.py (LIT006); this freezes
|
|
# new cast imports. Suppress (with a reason) via `# noqa: TID251 # <reason>`.
|
|
"typing.cast".msg = "No unchecked casts: validate into a frozen dataclass/NamedTuple/ReadOnly TypedDict at the boundary (pydantic)."
|
|
"typing_extensions.cast".msg = "Same as typing.cast."
|
|
# Unverified narrowing predicates: the checker never validates the guard body, so a
|
|
# wrong guard silently corrupts types. Banned outright (there are none today).
|
|
"typing.TypeGuard".msg = "Unverified narrowing. Parse into a concrete type, or use isinstance for a runtime-checked narrowing."
|
|
"typing_extensions.TypeGuard".msg = "Same as typing.TypeGuard."
|
|
"typing.TypeIs".msg = "Unverified narrowing (the body is trusted). Parse into a concrete type instead."
|
|
"typing_extensions.TypeIs".msg = "Same as typing.TypeIs."
|
|
# Dispatched public entry points: import them from their dispatch module so every
|
|
# supported call path selects Rust or Python in one place. Only the dispatch
|
|
# modules and internal recursive calls may reach the Python implementation
|
|
# directly, each with a `# noqa: TID251 # <reason>`.
|
|
"litellm.responses.main.responses".msg = "Import litellm.responses.dispatch.responses so the call routes through dispatch."
|
|
"litellm.responses.main.aresponses".msg = "Import litellm.responses.dispatch.aresponses so the call routes through dispatch."
|
|
"litellm.llms.anthropic.pass_through.messages.handler.anthropic_messages".msg = "Import litellm.messages.anthropic_messages so the call routes through dispatch."
|
|
"litellm.llms.anthropic.pass_through.messages.handler.anthropic_messages_handler".msg = "Import litellm.messages.anthropic_messages_handler so the call routes through dispatch."
|
|
"litellm.main.completion".msg = "Import litellm.completion so the call routes through dispatch."
|
|
"litellm.main.acompletion".msg = "Import litellm.acompletion so the call routes through dispatch."
|