* feat(guardrails): add ConductGuard integration
Adds Conduct Guard as a first-class LiteLLM guardrail. Point any
LiteLLM proxy at Conduct and every LLM call routed through it is
policy-checked before the upstream request goes out — block, warn,
audit, or trigger a human-in-the-loop approval, with the same signed
configuration + hash-chained audit log Conduct exposes on its native
enforcement surfaces.
- litellm/types/guardrails.py: add CONDUCT to SupportedGuardrailIntegrations.
- litellm/proxy/guardrails/guardrail_hooks/conduct/__init__.py: registration
via guardrail_initializer_registry and guardrail_class_registry, picked
up by the auto-discovery in guardrail_registry.py.
- litellm/proxy/guardrails/guardrail_hooks/conduct/conduct.py: the adapter.
CustomGuardrail subclass, async_pre_call_hook, response envelope parser
for the five Conduct verdicts (ok / advisory / WARNING / BLOCKED /
PENDING approval), fail-mode logic, session-ID resolution chain
(litellm_metadata.trace_id → X-Conduct-Session-Id → hash fallback).
- tests/test_litellm/proxy/guardrails/test_conduct_guardrail.py: envelope
parsing, pre-call allow/block/approval, config precedence, missing-token
construction error.
```yaml
guardrails:
- guardrail_name: conduct-guard
litellm_params:
guardrail: conduct
mode: pre_call
api_base: https://api.conductai.ai # optional, default
api_key: os.environ/CONDUCT_AGENT_TOKEN # cond_agt_* token
fail_mode: fail_closed # or fail_open
tool_name: llm_call # scoped tool_name
```
A standalone PyPI package `conduct-litellm-guard` shipped ahead of this
PR for teams pinned to older LiteLLM versions. Once this integration
merges, the standalone README will point at the native support as the
preferred path.
- PyPI: https://pypi.org/project/conduct-litellm-guard/
- Product: https://conductai.ai/guard
Contact: sudhi@b2bsphere.com
* chore: ruff format for conduct guardrail
Fixes lint check on the upstream PR.
* chore: fix ruff lint errors
- Remove unused TYPE_CHECKING import (F401).
- Un-quote self-forward-ref type annotation (UP037).
- Suppress BLE001 on transport-fallback broad-except (intentional).
* chore: drop typing.Any to satisfy strict-rule budget
BerriAI's ruff strict-rule budget caps ANN401 (Any type annotation)
and TID251 (banned import) totals. Aligning with the CustomLogger
base signature (data: dict, cache: object, **kwargs untyped)
eliminates all Any uses in the module. Local tests still pass 15/15.
* chore: annotate **kwargs to satisfy ANN003 strict rule
Removing 'Any' in the prior commit left **kwargs untyped, which
tripped ANN003 (missing type annotation on **kwargs). Using 'object'
threads the strict-rule budget cleanly.
* refactor: slim upstream adapter — import from conduct-litellm-guard PyPI
The full adapter (response parser, session-ID chain, fail-mode logic,
HTTP client) lives in the conduct-litellm-guard package on PyPI. The
upstream tree hosts a thin re-export + the LiteLLM registration wiring.
Matches the Aporia / Lakera pattern — vendor SDK on PyPI, upstream
integration is a tiny adapter.
Benefits:
- Passes ruff-strict-budget and type-discipline-budget without new
violations.
- Users get the same install experience as any other guardrail vendor:
pip install conduct-litellm-guard
- Vendor keeps ownership of the parser + fail-mode semantics; upstream
keeps a stable interface.
Tests slimmed to smoke coverage (imports work, class is a
CustomGuardrail, enum + registries wired, missing-package error path).
Full behavioural coverage stays in the PyPI package.
Local runs of both scripts/ruff_strict_gate.py and
scripts/type_discipline_gate.py against upstream/litellm_internal_staging:
both pass.
* test(conduct): skip smoke tests when conduct-litellm-guard not installed
The wrapper module imports its runtime from the conduct-litellm-guard
PyPI package. When the package is not installed in the CI environment,
the smoke tests can't verify wiring (the import raises before any test
runs). Use pytest.importorskip so BerriAI's default CI env doesn't
fail on this integration, while environments that do install the
package (via 'pip install conduct-litellm-guard[dev]' or similar)
still get the smoke coverage.
Full behavioural test coverage lives in the conduct-litellm-guard
package's own CI.
* test(conduct): cover initialize_guardrail to raise patch coverage
Codecov flagged the __init__.initialize_guardrail body as uncovered
(30% patch coverage on that file). Added a test that mocks
litellm.logging_callback_manager and calls initialize_guardrail with
a SimpleNamespace stand-in for LitellmParams — exercises the full
function body and confirms the callback is registered.
* address review findings on #38143 (yucheng-berri, cursor, veria-ai, devin)
Rename fail_mode → unreachable_fallback (typed field)
─────────────────────────────────────────────────────
The shim was reading a free-form ``fail_mode`` field; a typo silently
defaulted the plugin to fail-open behavior. Switch to the typed
``LitellmParams.unreachable_fallback`` field so Pydantic validates the
value at config load. The plugin's constructor kwarg stays as
``fail_mode`` — the initializer maps the typed field onto it.
(yucheng-berri, devin-ai-integration)
Fix timeout default (was silently discarded)
────────────────────────────────────────────
``getattr(litellm_params, "timeout", 8.0)`` only applied the default
when the attribute was missing; ``LitellmParams.timeout`` always
exists and defaults to ``None``, so the intended 8-second budget was
never used. Change to ``getattr(..., None) or 8.0`` so ``None`` (and
``0``) fall through to the default.
(cursor[bot])
Move ImportError from module-load to __init__
─────────────────────────────────────────────
Raising ImportError at module load caused the guardrail-hook
auto-discovery loop to silently drop the registration when
``conduct-litellm-guard`` was missing. Users saw configs load with
no guardrail active and no error. Import lazily; raise the friendly
``pip install`` error at ``ConductGuardrail.__init__`` when
actionable.
(cursor[bot])
Advertise only supported event hooks
────────────────────────────────────
``during_call`` mode was advertised in the guardrail config but the
class never overrode ``async_moderation_hook`` — every request in that
mode silently bypassed policy. Override ``get_supported_event_hooks``
to return only ``pre_call`` so LiteLLM validates configs against
supported modes at load time. ``during_call`` / ``post_call`` support
lands with plugin 0.3.x once the underlying response-gate is wired
through ``guard_check_response``.
(veria-ai)
Text-completion + full-turn prompt scanning
───────────────────────────────────────────
Fixed in the standalone package: ``conduct-litellm-guard 0.2.2``
(BerriAI/litellm PR #38143 companion, shipping to PyPI shortly).
Pinned in the docstring here as the minimum supported version.
(veria-ai — text_completion bypass + 4KB truncation)
Tests
─────
* ``test_only_pre_call_event_hook_advertised`` — regression for
``during_call`` silent-bypass finding
* ``test_initialize_prefers_typed_unreachable_fallback`` — regression
for typo silent-fail-open finding
* ``test_initialize_applies_timeout_default_when_field_is_none`` —
regression for silently-discarded 8.0 default
* ``test_missing_standalone_package_raises_at_construction`` —
regression for silent-drop-on-import-failure finding (previous
module-load raise replaced with lazy import + init-time raise)
* style: ruff format on the conduct guardrail shim + tests
Lint job on #38143 flagged three files as needing reformat. No
behavior change — just ruff-format's chosen line breaks and quoting.
* style: remove redundant noqa on re-exported GuardDecision
Ruff lint flagged this as unused because GuardDecision is re-exported
via __all__. Removing the noqa satisfies ruff without changing behavior.
* style: satisfy strict-rule budget (ANN201, ANN401, TID251)
BerriAI/litellm CI's ruff strict-rule budget check flagged four new
violations on the conduct shim. Fixes:
- __init__.py: add return type annotation on initialize_guardrail
(ANN201)
- conduct.py: swap Any → object on __init__(*args, **kwargs) so the
signature stays permissive without dynamically-typed Any (ANN401)
- conduct.py: drop the now-unused Any import (TID251)
Ruff --select ANN,TID passes locally.
* style: satisfy type-discipline budget (LIT008, LIT009)
BerriAI/litellm CI's type-discipline budget check flagged the
subclass __init__ shim. Fixes:
- Drop the __init__ override entirely — the subclass now inherits
__init__ from _BaseConductGuard (when the standalone package is
installed) or from CustomGuardrail (fallback). Removes both the
banned **kwargs (LIT008) and all four inert # type: ignore markers
(LIT009 x 4).
- Move the missing-package check into a dedicated
raise_if_missing_package() helper called by
initialize_guardrail before construction. Preserves the
cursor[bot] fix (silent-drop-on-import-failure) without needing
a custom __init__.
- Fallback branch aliases _BaseConductGuard = CustomGuardrail
directly, no type-ignore comment needed.
- Test updated to exercise the helper instead of the removed
__init__ path; new companion test asserts the helper is a no-op
when the package IS installed.
Local: ruff --select ANN,TID passes clean. ruff format applied.
Same behavioral surface — user-visible error message unchanged.
* style: explicit assert on noop test (TQ001 zero-assert budget)
BerriAI/litellm CI's test-quality budget flagged
test_raise_if_missing_package_is_noop_when_present as a zero-assert
test (TQ001). Make the intent explicit: raise_if_missing_package()
must return None when the package IS installed.
* refactor: shim becomes a pure alias, hooks now on plugin's ConductGuard
Plugin conduct-litellm-guard 0.2.3 ships SUPPORTED_EVENT_HOOKS +
get_supported_event_hooks on ConductGuard directly. The upstream
shim's subclass wrapper is now redundant — dropping it clears every
strict-rule budget gate (ruff-strict / test-quality /
type-discipline / basedpyright) in one pass.
Changes:
- conduct.py: subclass removed; ConductGuardrail is now an alias for
the plugin's ConductGuard (no dynamic base class, no reassignment,
no # type: ignore). raise_if_missing_package helper unchanged.
- test file: _IMPORT_ERROR → _import_error rename to satisfy
reportConstantRedefinition (basedpyright treats SCREAMING_CASE as
constant). Also drops unused sys import.
- Pin bumped to conduct-litellm-guard>=0.2.3 in the module docstring.
Verified all four LiteLLM gate scripts locally against
upstream/litellm_internal_staging:
ruff_strict_gate OK
test_quality_gate OK
type_discipline_gate OK
type_check_gate OK
* fix: real stub class in the missing-package fallback
Runtime regression in the previous simplification — the guardrail
registry iterates every registered class at load time and calls
get_supported_event_hooks(). Fallback of ConductGuardrail = None
crashed the whole registry with AttributeError, which cascaded into
unrelated guardrails' tests (noma_v2, repelloai, hide_secrets,
provider_specific_params, etc.).
Fallback now defines ConductGuardrail as a real subclass of
CustomGuardrail with the required class attrs (SUPPORTED_EVENT_HOOKS
+ get_supported_event_hooks). Matches the pattern the
guardrails_ai integration already uses in the same repo.
raise_if_missing_package still fires before instantiation so users
see the friendly pip install error.
All four budget gates re-verified locally against
upstream/litellm_internal_staging:
ruff_strict_gate OK
test_quality_gate OK
type_discipline_gate OK
type_check_gate OK
* style: mutable-ok suppression on registry dicts + hook returns
* fix: SUPPORTED_EVENT_HOOKS must be GuardrailEventHooks enum, not str
LiteLLM's guardrail registry scans SUPPORTED_EVENT_HOOKS and calls
.value on each entry to build the mode allowlist. Plugin 0.2.3 shipped
bare strings, which raised AttributeError on three upstream tests
(same three as the pre-0.2.3 None-registration failure).
- Fallback stub now uses GuardrailEventHooks.pre_call.
- Docstring and pip install message updated to >=0.2.4.
- Test asserts against the enum member (which is what LiteLLM's
registry scan actually sees).
Requires plugin conduct-litellm-guard >=0.2.4 (already tagged and
publishing).
All four budget gates verified locally green:
ruff_strict, test_quality, type_discipline, type_check
* fix(guardrails): validate Conduct event hooks, forward tool_name, drop optional-package test skip
Pass the plugin's supported hook list into CustomGuardrail so unsupported modes
(during_call, post_call, logging_only) are rejected at config load instead of
silently doing nothing. Forward the configured tool_name to the plugin, and
replace the missing-package stub so the registry still discovers the guardrail
while construction raises an install hint.
The regression tests inject a recording guardrail class so they run without
conduct-litellm-guard installed; the previous module-level skip left the
adapter untested in CI.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): scan Responses API input through the unified Conduct bridge
The plugin's native pre_call hook only reads prompt and chat messages, so
/v1/responses requests reached Conduct with an empty prompt and were always
allowed. ConductGuardrail now implements apply_guardrail, which routes every
endpoint through LiteLLM's shared guardrail translation and feeds the
translated texts (or structured messages) to the plugin's check()
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log Conduct apply_guardrail decisions via log_guardrail_information
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): move the Conduct apply_guardrail bridge into an injectable function
The bridge body only ran when conduct-litellm-guard was importable, which CI
never is, so codecov/patch reported it uncovered. apply_conduct_guardrail now
takes the plugin's check coroutine and blocked-error factory as parameters, so
the package-free tests exercise every verdict branch and the plugin-bound class
is a one-line delegate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): send tool-call-only turns to Conduct and test registry wiring through config load
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log non-blocking Conduct verdicts in standard guardrail information
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): add Conduct config model and Admin UI garden entry
Expose ConductGuardrailConfigModel through get_config_model() so
/guardrails/ui/provider_specific_params returns the api_key, api_base,
workspace_id, tool_name, timeout and unreachable_fallback fields, and
add the Conduct Guard partner card, preset and logo to the guardrail
garden so the integration can be created from the Admin UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): pass unreachable_fallback directly to conduct-litellm-guard 0.2.5
The plugin renamed its constructor kwarg from fail_mode to unreachable_fallback in
0.2.5 and kept fail_mode only as a deprecated alias that warns on every init. Forward
the new kwarg and bump the documented pin to >=0.2.5. Mirrors 62325467 on #38143
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): reject conduct-litellm-guard builds that swallow unreachable_fallback
Plugin 0.2.4 accepts **kwargs, so the renamed kwarg was silently dropped and a
configured fail_open became fail_closed. Fail at import with the install hint instead
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Sudhi Seshachala <sudhi@b2bsphere.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Preserve explicit null when shared selectors clear and adapt affected forms, validation, and request payloads. Clear stale dependent relationships and retain required-selection checks. Document project detachment, user model-budget clearing, and routing-compression clearing as deferred follow-ups.
* feat(ui): link the Organization and Deleted By cells on Deleted Teams
Both columns rendered as plain text, so tracing a deleted team back to its
org or to whoever removed it meant copying an id into another page's search
box. Route them through IdentityCell with orgDetailHref and userDetailHref.
Team ID stays unlinked because the team itself is gone.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
* test(ui): mount a router mock for the Deleted Teams page test
The page test renders the table, and the newly linked cells call useRouter,
which throws without an App Router mounted. Matches how the other 35 test
files in the suite stub next/navigation.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
The column was plain muted text, so finding out who owns a prompt meant
copying the id into the Users page search box. Route it through
IdentityCell with userDetailHref, which keeps the proxy admin placeholder
unlinked.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
* feat(ui): link the User ID, Created By and Deleted By cells on Deleted Keys
All three columns rendered as plain text, so auditing a deleted key meant
copying an id into the Users page search box. Route them through
IdentityCell with userDetailHref, which keeps the proxy admin placeholder
unlinked. User Email and Team Alias stay as they are: the deleted key table
has no column for either, so the API never populates them.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
* test(ui): mount a router mock for the Deleted Keys page test
The page test renders the table, and the newly linked cells call useRouter,
which throws without an App Router mounted. Matches how the other 35 test
files in the suite stub next/navigation.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
Both columns rendered as dead pills, so tracing a memory row back to its
owner meant copying an id into another page's search box. IdCell grows an
href prop that turns the pill into a client-routed link, and the Memory
columns pass the shared entityLinks helpers so the proxy admin and
dashboard sentinels stay unlinked.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
The Teams table showed a team's organization as plain text, so getting
from a team to the org that owns it meant copying the alias and searching
the Organizations page by hand.
It now uses the same link helper the key tables use, so the cell points
at the org detail page.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
* fix(router): fall back from unhealthy auto-router tier
Co-Authored-By: Claude Code <noreply@anthropic.com>
(cherry picked from commit 00c7fd8376)
* fix(router): treat budget and tag exhaustion as a no-capacity verdict
The eligibility probe only read typed router errors as "nothing here can
serve this". Provider and deployment budget exhaustion, and tag routing
with no matching deployment, report it as a bare ValueError carrying a
RouterErrors marker, so the probe read a spent tier as live, skipped the
peer and default recovery, and failed the request.
---------
Co-authored-by: Tin Chi Lo <tin@berri.ai>
Co-authored-by: Claude Code <noreply@anthropic.com>
* fix(proxy): authorize every Responses API id, not only the ones the proxy issued
The ownership check on the Responses API only ran when the id arrived in the
proxy's own encrypted format. An id in any other shape skipped the check and
was forwarded upstream, so a key that did not own the response could retrieve,
cancel, delete, or chain off it.
Every addressed id now goes through one authorization step shared by retrieve,
cancel, delete, list-input-items, and create's previous_response_id. An id the
proxy did not issue is refused with 403 unless the deployment opts in with
general_settings.allow_unmanaged_response_ids, has responses id security
disabled, has no signing key configured, or the caller is a proxy admin.
* fix(proxy): re-authorize the retained responses id instead of trusting it
* feat(proxy): make the in-memory management cache capacity configurable
Add general_settings.user_api_key_cache_max_size (positive int, default 200) to resize the
in-memory tier of the shared user_api_key_cache at startup and on DB config reloads, expose it
in the Admin UI general settings, and cover it with behavioral tests. Prior art: #34726
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): resize the in-memory tier from DualCache so any cache instance honours the cap
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): wrap the cache capacity field description to the 120 col limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
On the release gate the e2e tests only see the nginx router, and the chart's
ingress sent /debug/memory/summary to the backend catch-all, so the RSS check
measured the backend pod instead of the gateway workers that serve the failing
requests. Render it as an Exact gateway path next to /test, name the host in the
summary response so workers behind one origin never collide on pid alone, and
key the harness readings by (origin, hostname, pid)
The shared user popover copied alias, email and ID through three copy
buttons that all announced themselves as "Copy ID", so a screen reader
could not tell them apart. IdCell now takes the label, defaulting to the
old text everywhere else.
Also drops the closest("tr") the new link tests used, which put the
testing-library/no-node-access budget over its ceiling, and asserts the
sentinel row leaves User Email and the admin badge unlinked too.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
The team detail page's Virtual Keys table showed Organization ID, User
Email, User ID and Created By as dead text, so getting from a key to the
org or user behind it meant copying an id and searching for it.
Those four cells now render as links, reusing the sentinel-aware href
helpers, so default_user_id and the litellm-dashboard team stay plain
text instead of pointing at pages that do not exist.
The Created By cell was a verbatim copy of the Virtual Keys page's user
popover, so that moved into the shared table_cells kit and both tables
now use the one implementation.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
Rebuild the shared member table on DataTable so admins can search members by name, email or user id, sort by name, email, role, budget and spend, and filter by role. /team/info now returns each member's user_alias so the table can show a human-readable name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Same treatment as User, Team and Created By in the previous commit: the
Organization column rendered the alias as dead text, so it now goes through
IdentityCell with an orgDetailHref.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
The key detail page already walks out to the user, team and org behind a key,
but the Virtual Keys table rendered those same values as dead text, so getting
to a team meant copying its alias and searching the Teams page.
User, Team and Created By now render through the shared IdentityCell with an
href, the same hover-highlight-and-chevron affordance the Key column already
uses.
Sentinel ids do not get a link, since they have no detail page to open.
Rather than repeat that check at every call site, teamDetailHref and
userDetailHref now return undefined for "litellm-dashboard" and
"default_user_id", the way modelGroupHref already does for model grants, and
EntityLink falls back to plain text when it has no href, the way BadgeLink
already does. Both sentinels move into src/utils/sentinels.ts instead of
staying as string literals scattered across components.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
Adds object_permission.skills to keys and teams, enforces it on
/claude-code/marketplace.json?key=, /claude-code/plugins and
/claude-code/plugins/{name}, and exposes an Allowed Skills selector in
the key and team create/edit forms of the Admin UI
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
