* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
* feat(guardrails): honor Lakera v2 skip-message flags and add advisory (inject_system_message) mode
Squashed rebase of bugfix/lakera-v2-skip-system-tool-messages onto latest
litellm_internal_staging (900+ commits ahead; a commit-by-commit rebase hit
repeated conflicts against the same files across earlier review-round
commits, so the branch's cumulative diff was reapplied in one pass instead).
Adds skip_system_message_in_guardrail/skip_tool_message_in_guardrail support
to Lakera v2, a third on_flagged: "inject_system_message" advisory mode, and
the associated masking-safety-guard hardening (multimodal content, non-
maskable message fields, combined messages+input, and structured Responses-
API input in advisory delivery) found across this PR's review rounds.
* fix(guardrails): don't let one invalid guardrail config crash proxy boot
init_guardrails_v2 had no try/except around initialize_guardrail, so a
guardrail whose litellm_params fail validation at construction time (for
example Lakera's on_flagged=inject_system_message combined with
mode=during_call, or a malformed advisory_system_message template) raised
uncaught and crashed the entire proxy at startup, taking down every other,
correctly-configured guardrail in the list. Catch ValueError/TypeError per
guardrail, log a warning, and skip it, matching the same pattern already
used for the DB-driven guardrail-creation path in guardrail_endpoints.py.
* fix(guardrails): preserve message fields and mask PII before advising in Lakera v2
Mask-in-place degraded to a hard block for any message carrying a field
beyond role/content (tool_call_id, tool_calls, name, cache_control), for a
message excluded by skip_system_message_in_guardrail/skip_tool_message_in_guardrail,
or for a message with no inspectable text, since it rewrote data["messages"]
wholesale from a synthetic role/content-only list built for the Lakera API
call. That made masking effectively unusable for any real tool-calling
conversation and made the skip flags flip every PII-only violation to a hard
block instead of masking just the in-scope text.
Replace the wholesale rewrite with a scope-index merge, reusing the same
merge_guardrailed_scoped_messages helper the OpenAI/Anthropic guardrail
translation handlers already use for this: patch content in place on a copy
of each original message actually sent to Lakera, and leave every
skipped/no-text/out-of-scope message untouched at its original position.
This also fixes on_flagged="inject_system_message" (advisory mode) shipping
raw unmasked PII to the model: a PII-only violation is now masked the same
way regardless of on_flagged, and the advisory note is reserved for flags
masking can't resolve on its own.
Addresses maintainer-reported regressions on BerriAI/litellm#34940.
* fix(guardrails): satisfy new lint gates for the masking/advisory fix
Parameterize the write-back helper's dict param and suppress the two new
lint rules that landed on the base while this branch was in flight: TQ008
(patching an internal collaborator) for two pre-existing tests unrelated to
this change, and LIT001 for a param that genuinely needs to mutate the
caller's request dict in place.
* fix(guardrails): normalize role casing in Lakera v2 masking scope, log skipped guardrails louder
Greptile finding: the masking scope helper compared roles case-sensitively
while filter_messages_by_skip_flags (used to build what's actually sent to
Lakera) normalizes casing, so an uppercase-cased "System"/"TOOL" role
survived the scope filter but was excluded from the inspected list. The
resulting length mismatch raised inside the strict positional zip, turning
a maskable PII-only violation into an unhandled request failure. Lowercase
the role comparison to match.
Also, per veria-ai's finding that a skipped invalid guardrail now fails
open: log it at error level with an explicit note that the proxy is
starting without that guardrail, so it's not mistaken for routine info.
* fix(guardrails): mask maskable PII in mixed violations before advising in Lakera v2
on_flagged="inject_system_message" only masked when a violation was
PII-only; a mixed violation (PII plus a non-PII flag like prompt injection)
fell straight through to the advisory branch with the raw PII still in
place, in both async_pre_call_hook and async_moderation_hook. Mask whatever
Lakera returned location data for before appending or logging the advisory,
so a mixed violation never ships raw PII just because something else was
also flagged.
Also degrade to blocking, same as block mode already does, when nothing
can be safely masked at all (multimodal content, or messages combined with
a Responses API input field) instead of showing an advisory note next to
raw, unredacted content.
Widened call_v2_guard/_mask_pii_in_messages/the write-back helper's message
parameters from list to Sequence to match what's actually passed through
from _filter_skipped_messages, instead of duplicating list(...) casts at
every call site.
* fix(guardrails): don't hard-block advisory mode for non-PII flags on non-maskable input
Bugbot finding: gating the entire inject_system_message branch on
is_multimodal_input hard-blocked every flagged request on Responses
instructions, combined messages+input, or multimodal content, including
a prompt-injection-only violation with no PII at all. Masking safety only
matters when there's actual PII to mask; a violation with no PII needs no
masking, so the advisory should still be delivered normally.
Only degrade to blocking when the breakdown actually contains a PII
detection and masking isn't safely possible. Otherwise, mask whatever's
maskable (if any) and deliver the advisory as before.
* fix(guardrails): require payload and breakdown for Lakera v2 advisory mode
Advisory mode's mixed-violation masking safety net can only redact
detected PII when Lakera's response carries both the breakdown (to
detect a PII hit at all) and payload (the location data to mask by).
payload=False or breakdown=False alongside on_flagged='inject_system_message'
silently forwarded raw PII next to the advisory note. Reject that
combination at construction and hot-reload time instead.
* fix(guardrails): skip_system_message_in_guardrail must not force-block Lakera masking
_has_responses_instructions treated any non-empty data["instructions"]
as unsafe to mask regardless of skip_system_message_in_guardrail, even
though that flag excludes the instructions-derived synthetic system
message from what Lakera ever inspects. PII detected purely in the
maskable non-system content was force-blocked instead of masked.
Also fixes pre-existing LIT010 (missing Final) violations in
_has_responses_instructions, _breakdown_has_pii_violation, and
async_post_call_success_hook that the rebase's lowered budget ceiling
now flags.
* chore: retrigger CI (GitHub Actions runner-acquisition failure on prior push)
* fix(guardrails): address maintainer review findings on Lakera v2 advisory mode
- Gate advisory_system_message template validation on on_flagged=
'inject_system_message', since block/monitor mode never reads it.
- Allow on_flagged='inject_system_message' with mode='during_call' at
construction/hot-reload instead of rejecting it; async_moderation_hook
already degrades gracefully (masks if possible, else logs a warning).
- reinitialize_guardrail now restores the previous live instance when the
new config fails to initialize, instead of leaving the guardrail deleted
entirely with nothing enforcing it.
- PATCH /guardrails/{id} rolls back the DB write and returns 422 when the
in-memory sync rejects the new config, instead of persisting a config
that never actually took effect and returning 200.
- Qualifire now rejects on_flagged values it doesn't implement (only
Lakera should accept 'inject_system_message'; LitellmParams flattens
the field across every guardrail config mixin).
* fix(tests): satisfy lint gates and update collateral test for advisory-mode fixes
- Add match= to a too-broad pytest.raises(ValueError), and suppress the
new TQ008 mocker.patch findings (same pattern already used by sibling
scenarios in this test).
- test_init_guardrails_v2_skips_invalid_guardrail_instead_of_crashing_boot
used mode='during_call' + on_flagged='inject_system_message' as its
invalid-config example; that combination is now accepted, so swap in
the payload/breakdown-missing case and add a test confirming during_call
advisory mode constructs successfully.
* docs(CLAUDE.md): auto-capture review learnings without being asked
This session found three real bugs a human maintainer caught after eight
rounds of bot review and live-proxy verification all missed them. Add a
standing instruction to write learnings.md entries the moment a root
cause is understood, in both the repo-wide file and any relevant skill's
own file, instead of relying on being asked.
* feat(guardrails): add scan_raw_request flag so YAML order can't change enforcement
Maintainer finding on BerriAI/litellm#34940: guardrails for the same hook
run sequentially over one shared, progressively-mutated request dict, so
declaring a masking guardrail before a blocking one hides the violation
from it (200 vs 400 depending purely on YAML order).
scan_raw_request opts a guardrail into always evaluating a snapshot taken
before any guardrail in the hook ran, regardless of its declared position.
Same contract as run_in_parallel: block-only, its own mutations discarded.
Verified live: real proxy, real Gemini call, two custom guardrails (a
redactor then a blocker). Same request, same declared order -- without the
flag the blocker never sees the raw secret (200); with it, the blocker
correctly rejects before any provider call (400).
* fix(guardrails): harden scan_raw_request against review findings
- Use safe_deep_copy instead of a bare deepcopy for the raw-request
snapshot; request payloads commonly carry unpicklable objects (e.g. an
otel span in metadata), which previously raised on every guarded
request when tracing was enabled (Bugbot, High).
- Only compute the snapshot when a guardrail actually opted in, and take
it before _maybe_execute_pipelines runs, so a pipeline-mutated payload
can't hide a violation from a scan_raw_request guardrail outside the
pipeline (veria-ai).
- Log a warning when a scan_raw_request guardrail returns a modified
payload, since that mutation is discarded and the combination is
otherwise silently exploitable for a masking-capable integration
misconfigured this way (veria-ai).
* chore(openapi): regenerate lazy snapshot and dashboard schema types
The lazy OpenAPI snapshot (litellm/proxy/_lazy_openapi_snapshot.json) and
the derived dashboard schema.d.ts had drifted stale relative to the
guardrail config model changes across this PR's rounds (advisory mode,
scan_raw_request, and upstream additions picked up by rebasing).
Regenerated via the CI's own documented fix:
uv run python -m litellm.proxy._lazy_openapi_snapshot
npm run gen:api (via make check)
* chore(openapi): pick up cache_hit_filter field after rebase
* fix(guardrails): stop scan_raw_request warning from firing on every call
_process_guardrail_callback always returns a dict once a guardrail runs
(mark_pre_call_hook_ran unconditionally stamps bookkeeping metadata), so
comparing the result to non-None warned on every request even when the
guardrail never touched the payload. Compare against a bookkeeping-only
baseline instead, so only an actual content mutation triggers the warning.
* fix(guardrails): make scan_raw_request snapshots independent of safe_memory_mode
safe_deep_copy can return the original object under litellm.safe_memory_mode,
or alias a per-key reference on copy failure. Under that mode, the
scan_raw_request comparison baseline aliased raw_request_snapshot (and
therefore the live request), letting mark_pre_call_hook_ran write a
premature execution marker that a deployment-level guardrail sharing the
same name would read as "already ran" and skip. Also affected the feature's
core isolation guarantee: input_data itself could alias the live request
under the same mode. Replace every scan_raw_request snapshot with
_independent_snapshot, which never returns an alias, only a genuine copy
or None.
* fix(guardrails): gate during_call mixed-violation masking behind an actual PII check
The during_call branch for a mixed violation under on_flagged=inject_system_message
unconditionally masked and reassigned data["messages"], even for a pure
prompt-injection violation with zero PII, unlike async_pre_call_hook which
already gates the same call behind _breakdown_has_pii_violation. The
unconditional reassignment touched shared request state during a hook
documented as racing with the concurrent LLM dispatch, for no reason when
there was nothing to mask.
* fix(guardrails): stop scan_raw_request from silently no-op'ing on real requests
_independent_snapshot did one whole-dict copy.deepcopy and returned None on
any failure. Every real proxy request carries data["litellm_logging_obj"]
(a Logging instance nesting a live OTel span with a real lock) by the time
pre_call_hook runs, which can never be deep-copied, so the snapshot failed
on every real request and silently fell back to the live, unisolated data
with no warning -- defeating the entire feature in production while every
existing test (none of which set litellm_logging_obj) kept passing.
Rework the helper to deep-copy each top-level key independently, falling
back to the original reference only for the specific key that fails, same
crash tolerance as safe_deep_copy's own per-key fallback. It never returns
None now; only the keys scan_raw_request actually depends on (messages/
input, metadata/litellm_metadata) need to be genuinely independent.
* fix(guardrails): block during_call when PII can't be safely masked
Greptile finding (P1, security): async_moderation_hook's inject_system_message
branch had no equivalent to async_pre_call_hook's degrade-to-blocking case for
a PII violation on input that can't be safely masked (e.g. combined
messages+input). It fell through to the advisory no-op branch and let raw,
unredacted PII reach the model with no protection at all. Raising still
blocks the response from reaching the caller even though during_call races
with the LLM dispatch, the same mechanism on_flagged="block" already relies
on for this hook, so add the same block-instead-of-advisory branch pre_call
already has.
* chore(lint): fix LIT002 ceiling after rebase merge conflict resolution
* fix(lint): suppress genuine LIT002 hits instead of padding the ceiling
My earlier rebase conflict resolution for type-discipline-budget.json's
LIT002 limit was too low, then overcorrected by padding it well above the
actual measured count. Root-caused instead: _independent_snapshot and the
PATCH-endpoint rollback path legitimately construct plain, mutable
request-payload/config dicts (matching this file's existing precedent for
the same shape), so suppress those four sites with `# mutable-ok:` rather
than reshaping code that must stay a plain dict by contract. Set the limit
to the exact current measured total; the small remaining gap vs upstream's
own committed ceiling is pre-existing drift in litellm_internal_staging
itself (its own tree already measures over its committed limit), not
attributable to this PR.
* fix(guardrails): stamp live request when a scan_raw_request guardrail runs
_run_sequential_guardrail_callback and _run_parallel_pre_call_guardrails only
called mark_pre_call_hook_ran on throwaway snapshot copies for a
scan_raw_request guardrail, never on the live request returned to the
caller. A later async_pre_call_deployment_hook (router-level guardrail
re-check) reads that marker on live kwargs to decide whether to skip
re-running the same guardrail; since it was never stamped there, the
guardrail ran a second time on live data, doubling the external call and
re-applying whatever scan_raw_request's contract says should be discarded.
* fix(guardrails): revalidate Qualifire's on_flagged on live config reload
on_flagged was validated only in __init__. The base
CustomGuardrail.update_in_memory_litellm_params is a generic setattr loop
with no revalidation, so a live config update (PUT /guardrails/{id}, no
restart) could setattr on_flagged="inject_system_message" onto a running
instance, bypassing the constructor's rejection -- silently blocking every
flagged request under an "advisory" label. Mirrors LakeraAIGuardrail's own
update_in_memory_litellm_params override added earlier in this PR.
* fix(guardrails): honor scan_raw_request for pipeline-managed guardrails
A scan_raw_request=True guardrail that is itself a pipeline step never saw
raw_request_snapshot: PipelineExecutor.execute_steps had no way to receive
it, and pipeline-managed guardrails are fully excluded from the normal
sequential/parallel loops that implement the flag. Such a guardrail silently
evaluated whatever an earlier pass_data step in the same pipeline had
already rewritten, defeating the flag for pipeline-managed guardrails.
Moves the snapshot helper (renamed independent_snapshot) from proxy/utils.py
to litellm_core_utils/core_helpers.py so pipeline_executor.py can use the
same independent-copy logic without a circular import, threads
raw_request_snapshot through _maybe_execute_pipelines and
PipelineExecutor.execute_steps/_run_step, and discards a scan_raw_request
step's returned data the same way the sequential/parallel loops already do.
* chore(openapi): pick up upstream drift after rebase onto litellm_internal_staging
* fix(guardrails): stop attempting PII masking during during_call in Lakera v2
Greptile finding (P1, security): during_call runs concurrently with the LLM
dispatch. In the common path, the provider call already binds its messages
kwarg before this guardrail's coroutine gets a chance to run, let alone
before its own network round trip to Lakera completes -- masking here can
never reliably reach the outgoing request, and _apply_redacted_messages_back_
preserving_fields reassigns to a new list object rather than mutating in
place, so even winning the race wouldn't help. This affected both the
PII-only and mixed-violation masking branches, all added in this same PR.
Remove masking from async_moderation_hook entirely and let PII violations
fall through to the normal on_flagged branching: block under "block" or
"inject_system_message" (extending the existing multimodal-only block to
cover every PII case, since masking is proven non-functional regardless of
input shape), log-and-allow under "monitor" -- consistent with how every
other violation type in this hook is already handled.
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* fix(proxy): give proxy_admin_viewer read parity with proxy_admin
Route-level checks already default-allow management GETs for the viewer
role, but ~15 handlers compared user_role to PROXY_ADMIN only, dropping
viewers into regular-user scoping (/key/list, /user/info, /model/info,
guardrails, prompts, agents, memory, workflows, MCP catalog, coordination
redis settings, credential migration check, enterprise projects). Swap
those read paths to user_api_key_has_admin_view; write gates unchanged.
The dashboard now presents the viewer session as Admin for all gating
(effectiveSessionRole) so every page fetches with admin visibility, with
userRoleLabel/isViewOnly preserving the account-menu label and the
playground cost guard. The server remains the write authority.
* refactor(agents): remove side-effectful health_check param from GET /v1/agents
Addresses a security review finding on the admin viewer read parity change:
listing agents with health_check=true made the proxy issue a server-side GET
to every agent URL, so a read-scoped caller could trigger request fan-out
beyond their object permissions. The list endpoint is now a pure read for
every role.
Removes the query param, the URL probing helper and its timeouts, the
AgentHealthCheck httpx provider tag, and the dashboard's Health Check
toggle. Requests still passing health_check=true get the full list back
with the param ignored.
* fix(proxy): keep credential encryption check proxy_admin only
The residual scan behind GET /credentials/migrate-encryption/check loads
every model, credential, MCP, team, and verification-token row and runs a
decryption attempt on each stored value. Extending it to proxy_admin_viewer
let a read-only account repeatedly trigger deployment-wide scans, so the
route keeps its original full-admin gate.
* fix(agents): restore health_check, keep list fast path proxy_admin only
Restores the agent health_check feature exactly as before this PR: the
query param, the URL probing helper, the httpx provider tag, and the
dashboard toggle all return, so existing callers keep the filtering
contract. The viewer expansion is instead reverted at its source: the
GET /v1/agents admin fast path stays PROXY_ADMIN only, so a
proxy_admin_viewer goes through the object-permission scoped branch as
before and cannot fan out health checks beyond their allowlist. The
viewer read of a single agent stays viewer-inclusive since it has no
side effects.
ruff.toml has excluded litellm/types/* since 2024, so no lint rule ever ran
on the types tree. Remove the exclusion, apply ruff --fix and ruff format
across litellm/types, and hand-fix what autofix cannot reach so the
pyupgrade budgets stay at zero: implicit type aliases converted to PEP 604
unions, RootModel[Union[...]] bases, duplicate imports, and a stray print.
Load-bearing import X as X re-exports deleted by preview-mode F401 are
restored, and the six star-imported hub modules keep their re-export
surface via per-file F401 ignores. Star-import consumers that silently
relied on typing names leaking from those hubs are modernized to builtin
generics and PEP 604 unions.
Runtime annotation introspection that only recognized typing.Union is
taught types.UnionType (guardrail UI field schemas, volcengine response
fill), with regression tests for both. Strict budget limits for the rules
the types tree now trips are raised to exact measured totals, so any
net-new violation still fails the gate
Clients calling the standalone apply_guardrail endpoint had no way to pass
per-request configuration to custom guardrail implementations. This adds an
optional metadata field to ApplyGuardrailRequest and forwards it to
CustomGuardrail.apply_guardrail via request_data, only when the client sends
it. The messages guard is aligned to the same is-not-None semantics so an
explicitly-sent empty list is forwarded instead of silently dropped.
The Admin UI's Guardrail Test Playground gains an optional Metadata JSON
input (validated client-side) wired through applyGuardrail in networking.tsx,
so parameterized guardrails can be exercised from the dashboard.
Tests cover metadata alone, metadata with messages, explicit empty values,
the omitted-field passthrough, and the UI panel's parse/error behavior
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* feat(guardrails): add pre_mcp_call support to Content Filter
* test(guardrails): cover canonical MCP key gate under pre_mcp_call mode
* fix(guardrails): scan MCP arguments per value and gate mixed-mode scans by call type
* fix(guardrails): cap MCP argument scan depth and register the walker with the recursion detector
* test(guardrails): update LIT-4226 UI settings tests for content filter pre_mcp_call support
* fix(guardrails): use builtin generics in MCP scan annotations to satisfy strict-rule budget
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(guardrails): filter Add-Guardrail mode dropdown per provider
The GET /guardrails/ui/add_guardrail_settings endpoint returned every
GuardrailEventHooks value in one flat supported_modes list, so the Admin
UI rendered pre_mcp_call as a selectable Mode for every guardrail. Saving
Content Filter or Tool Permission with pre_mcp_call then failed with a
400 because those guardrails' server-side supported_event_hooks list
excludes it.
Expose each guardrail's supported hooks as a get_supported_event_hooks
classmethod on CustomGuardrail (mirrors the existing get_config_model
pattern) and have the endpoint iterate guardrail_class_registry to build
a supported_modes_by_provider map. The UI Mode dropdown filters by that
map when the selected provider is known and falls back to the global
list otherwise. __init__ now sources its own supported_event_hooks list
from the classmethod so the two sides can't drift.
Also register BedrockGuardrail, ToolPermissionGuardrail, lakera,
lakera_v2, and presidio in guardrail_class_registry so they participate
in the map (they were previously only in guardrail_initializer_registry
and had no class-registry entry).
Behavior change: guardrails that previously had no supported_event_hooks
declared (aim, javelin, azure/text_moderation, cato_networks,
crowdstrike_aidr, headroom, hiddenlayer, lasso, noma, onyx,
prompt_security, qualifire, repelloai, zscaler_ai_guard, aporia_ai,
lakera_ai, lakera_ai_v2, mcp_jwt_signer, model_armor, presidio) now
validate the configured mode at instantiation. Existing configs where
the mode was silently a no-op will fail at proxy startup with a clear
validation error rather than running as a broken guardrail.
Resolves LIT-4226
* fix(guardrails): add LITELLM_STRICT_GUARDRAIL_MODES escape hatch, preserve current mode in edit form
Address Greptile P1 (startup break) and P2 (edit form UX):
LITELLM_STRICT_GUARDRAIL_MODES defaults to true (raise on unsupported
event_hook, unchanged behavior for the guardrails validated pre-PR).
Setting it to false logs a warning and continues, giving deployments an
opt-out while they fix configs that now surface as errors instead of
silently no-op'ing. Regression test covers both modes.
Edit form now surfaces the currently-saved mode even when it is not in
the filtered per-provider list, so a legacy row (e.g. content_filter
saved with pre_mcp_call before this fix) no longer disappears from the
dropdown; the option renders with a 'not supported by <provider>' note
so the user knows to pick another.
* fix(guardrails): correct audited hook lists, prune stale modes on provider switch, clean form lint
Audited every get_supported_event_hooks classmethod against the hooks
each guardrail's own tests exercise and its handler methods. Five were
too narrow and their tests caught it in CI: rubrik gains pre_call,
presidio gains during_call and pre_mcp_call, prompt_security, onyx and
qualifire gain during_call. The remaining classes match either their
original __init__ declarations or their exercised modes exactly.
Cursor review fixes: the Add form now drops selected modes the new
provider does not support when the user switches providers, so a
pre_mcp_call selection cannot ride along into a provider that rejects
it at save; the edit form handles list-shaped stored modes instead of
treating mode as always a string.
Extracted shared toModeArray and getSupportedModesForProvider helpers
into guardrail_info_helpers so both forms use one implementation, typed
the remaining any usages in both forms, removed nested ternaries, and
committed the ratcheted-down eslint metrics and pruned suppressions
* feat(guardrails): wire apply_guardrail into proxy logging callbacks
Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): fix Greptile review comments on apply_guardrail logging
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(apply_guardrail): preserve original exception and capture modified response
- Capture return value from post_call_success_hook so callback-modified
responses propagate to the caller.
- Wrap success/failure logging calls in defensive try/except so logging
infrastructure failures don't replace the user-visible response or mask
the original guardrail exception.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* Fix mypy
* fix(apply_guardrail): isolate failure logging and use post-hook response for logging
- Split async_failure_handler and post_call_failure_hook into independent
try/except blocks so a callback bug in one does not silently skip the
other.
- Build response_for_logging inside _emit_guardrail_success_logs after
post_call_success_hook runs, so logged data matches the response the
caller actually receives when the hook modifies the response.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(apply_guardrail): fix black formatting and update tests for fastapi_request param
- Run black on guardrail_endpoints.py to fix CI formatting check
- Add _mock_proxy_logging() helper to enterprise guardrail tests to patch
proxy-server globals imported at call time
- Pass fastapi_request=Mock() in all direct apply_guardrail test calls
to match updated function signature
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): use transformed exception from post_call_failure_hook in apply_guardrail
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(guardrails): isolate sync/async logging handlers in apply_guardrail
Separate each logging handler call into its own try/except so a failure
in the async handler does not silently skip the sync handler submission
(and vice versa). Matches the docstring's defensive intent.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(apply_guardrail): guard transformed_exception with isinstance check
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(guardrails): mock proxy globals in not_found test and share apply_guardrail logging fixture
- Add proxy-server global mocks to test_apply_guardrail_not_found so the
failure-path post_call_failure_hook call doesn't touch the real proxy
logging singleton.
- Extract the duplicated _mock_proxy_logging context manager out of the
two enterprise apply_guardrail test files into a shared conftest fixture
so the helper stays in one place.
* fix(guardrails): use update_messages to keep logging obj in sync
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
- _get_masked_values now recurses into nested dict values and covers
additional field name patterns (credentials, password, passwd)
- _row_to_submission_item applies masking before returning litellm_params
- list_guardrails_v2 filters DB and in-memory guardrails to the caller's
team memberships for non-admin users; admins still see all guardrails
- approve_guardrail_submission propagates team_id into the in-memory
guardrail dict so ownership is preserved after approval
Bug 1: internal users hit route-level 403 on /guardrails/submissions.
The route wasn't in self_managed_routes, so the route allowlist rejected
non-admin callers before our endpoint's team-scoping ran. Added
/guardrails/submissions and /guardrails/submissions/{guardrail_id} to
self_managed_routes.
Bug 2: register_guardrail 403'd non-admins registering for teams in
their user.teams list. It used get_team_membership() which reads the
litellm_teammembership join table, but that row is only created when
the team has a budget (management_helpers/utils.py:225). Switched to
the _get_user_team_ids helper (reads user_obj.teams), making it
consistent with list_guardrail_submissions.
UI: moved the Test Playground tab inside the isAdmin conditional in
guardrails.tsx. Internal users now see only the Submitted Guardrails
tab; admins still see all four.
Tests: added coverage for non-admin register paths (cross-team allowed
and cross-team forbidden).
Backend:
- list_guardrail_submissions no longer 403s non-admins; it returns only
submissions whose team_id matches one of the caller's teams (via
get_user_object.teams). Admins still see all.
- Filtering by a team the caller is not in returns 403.
- Users with no team memberships get an empty list (no DB query).
- get_guardrail_submission applies the same scoping to single-item GETs.
Frontend:
- Remove admin-only bail-out in TeamGuardrailsTab.fetchSubmissions so
internal users actually load their team's submissions.
- Finish antd migration in guardrails.tsx: drop the last Tremor Button.
- Remove guardrailsList.length === 0 gate on the Test Playground tab;
the playground already renders a "No guardrails available" inline
empty state, which is more discoverable than a disabled tab.
Tests:
- Cover non-admin scoped access, empty teams, cross-team filter 403,
and per-submission GET scoping.
* style(ui/): distinguish agent calls from llm calls on ui
* feat: initial grouping working
* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls
* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call
allows stable trace id usage
* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict
_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: propagate trace/session id from headers in MCP server calls
Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(guardrails): team-based guardrail registration and approval workflow
Add team-based guardrail submission system where teams can register
Generic Guardrail API guardrails for admin review. Includes:
- POST /guardrails/register endpoint for team-scoped submissions
- Admin review endpoints (list/get/approve/reject submissions)
- Team Guardrails tab in the UI dashboard
- extra_headers support for forwarding client headers to guardrail APIs
- Prisma schema migration for status, submitted_at, reviewed_at fields
- Documentation for team-based guardrails and static/dynamic headers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(guardrails): address review feedback - SSRF, silent failure, redundant query
- Validate api_base URL scheme (http/https only) and hostname in
register_guardrail to prevent SSRF via team submissions
- Return warning field in approve response when in-memory initialization
fails so admins know the guardrail won't work until next sync cycle
- Eliminate redundant DB query in list_guardrail_submissions by fetching
all team guardrails once and deriving both filtered list and summary
counts from the single result set
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(guardrails): add pending_review status guard to reject endpoint
Prevent rejecting already-active or already-rejected guardrails, which
would create a DB/memory inconsistency (active in memory but rejected
in DB). Now mirrors the approve endpoint's status check.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* UI Config Guardrails should not be editable and guardrails info definition location fix
* Remove unused import
* Added literals for guardrail definition location
* chore: Consistency in install-test-deps using poetry run
* feat: update in-memory guardrails after database CRUD operations
* test: add parameterized tests for guardrail CRUD with memory sync
* refactor(aim.py): refactor to support adding aim guardrails on UI
* fix(base.py): add ui_friendly_name to config model
* feat(ui/): support loading new guardrails from backend api call
removes need to onboard each guardrail to ui
* fix: don't show optional params if not set and don't show ui_friendly_name (internal param0
* fix(ui/add_guardrail_form.tsx): ensure dynamic provider value is used
* fix(ui/): just one-time update the provider map dictionary
* fix(ui/): show masked api base / api key on guardrail update
* refactor(aporia_ai/): refactor to show on UI
* feat(aporia_ai/): add aporia ai guardrail to UI
* refactor(guardrails_ai/): refactor to add via UI
* refactor(lasso.py): refactor to enable adding lasso guardrails via UI
* feat(pangea.py): add pangea guardrail on UI
* feat(panw): add panw prisma airs through UI
* test: update tests
* fix: fix ruff linting error
* test: update tests
* fix: add missing docs
* fix: fix guardrail init
* fix: suppress linting errors
* fix(proxy_server.py): fix linting error
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* feat(text_moderation.py): handle str to int conversion
* fix(guardrail_info.tsx): only render pii settings if guardrail is presidio
* fix(guardrail_info.tsx): add guardrail specific fields to update settings
allows updating guardrail fields (e.g. severity threshold) post-create
* fix(guardrail_endpoints.py): set guardrail_id in guardrail object
ensures duplicate objects not created on guardrail update
* fix(guardrail_endpoints.py): allow provider specific fields to be updated on patch update
* refactor(guardrail_endpoints.py): remove duplicate info endpoint
* fix(guardrail_endpoints.py): mask sensitive keys on returning via guardrail `/info`
Prevent leaking keys
* fix(guardrail_optional_params.tsx): return numerical input when numerical component used
fixes issue where output was a str
* fix(guardrail_optional_params.tsx): render dict keys correctly
* fix(text_moderation.py): fix severity by category check
* fix(proxy/utils.py): check if guardrail should run for post call streaming hook
Prevents invalid guardrails from running if not requested
* test: fix import
* fix: fix linting error
* test: update test
* fix: fix tests
* fix: fix code qa errors
* fix(guardrail_endpoints.py): set max depth for function
* test: update recursive_detector.py
* test: update list
* build: merge main
* fix: fix ruff check errors
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* test: update tests
* fix mapped tests (#12320)
* fix - use flush llm client cache
* faster mapped tests
* test_async_multiple_response_ids_routing
* fix tests
* test_ateam_member_update_admin_requires_premium
* regular mapped tests
* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"
This reverts commit 2c60c316ec.
* reset num workers
* check_for_litellm_module_deletion
* add test
* add code_qa_check_tests
* ci/cd new release
* docs(opentelemetry_integration.md): add otel debug flag to docs
help people debug their traces easily
* [Feat] Add failure logging support for s3 logger (#12299)
* add async_log_failure_event
* test_basic_s3_v2_logging_failure
* Comma separated spend and budget (#12317)
* all_keys_table, key_info_view, organizations
* teams
* the whole usage page
* teams page
* users table
* user info
* more commas
* some more commas
* Segregate MCP tools on connections using headers (#12296)
* Add get tools segregation
* add ui changes (#12302)
* resolve comments
* add mapped tests
* remove advanced settings (#12323)
* [Bug Fix] /generateContent API - Only pass supported params when using OpenAI models (#12297)
* fix - only pass GenericLiteLLMParams
* test_google_generate_content_with_openai
* Fix: Fix custom ca bundle support in aiohttp transport (#12281)
* Unify usage of get_ssl_configuration
* Fix doc
* [Feat] Add github co-pilot as a new LLM API provider (#12325)
* Litellm dev 03 05 2025 contributor prs (#9079)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
* Fix#7629 - Add tzdata package to Dockerfile (#8915)
* Add tzdata package to Dockerfile
* Move tzdata to python requirement.txt
* feat: add support for copilot provider (#8577)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat: add model information for copilot models
* fix: fix linting errors
* test: remove integration test for github_copilot + fix misisng mock
* fix: use print to make sure the logger message shown
* test: remove debug print
* fix lint (#11112)
* Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119)
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
---------
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* refactor github copilot
* test_github_copilot_transformation.py
* test_github_copilot_authenticator.py
* add GitHub Copilot
* fix order
* doc fix
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
* add new env vars to docs
* ssl_cert_error (#12327)
* feat - add new banner on startup (#12328)
* [Security Bug Fix] Ensure only LLM API route fails get logged on Langfuse (and other loggers) (#12308)
* _is_proxy_only_llm_api_error
* test_proxy_only_error_true_for_llm_route
* add not on change
* Update tests/test_litellm/proxy/test_proxy_utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add test_post_call_failure_hook_auth_error_key_info_route
* test fix _is_proxy_only_llm_api_error
* test_chat_completion_request_with_redaction
* test_post_call_failure_hook_auth_error_llm_api_route
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Ensure message redaction works for responses API logging (#12291)
* add fixes to choice implementation redaction
* add isInstance check on responses API
* change datadog to revert back
* change datadog to revert back
* fix type errors
* Redaction test changes
* Redaction test changes
* Redaction test changes - remove changes
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* fix type error
* test fix - import loc
* [Feat] QA - Arize Team based logging (#12331)
* add _get_tracer_with_dynamic_headers
* fix construct_dynamic_arize_headers
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* add construct_dynamic_otel_headers for arize
* refactor common code
* test_construct_dynamic_arize_headers
* otel unit tests
* test_arize_dynamic_params
* test_arize_dynamic_headers_in_grpc_requests
* [Feat] UI - Allow setting Logging Callback Setting per Key (#12333)
* allow adding logging settings on a key
* add ability to edit key logging settings
* dont leak api keys on ui
* fix order
* add langsmith settings
* ui new build
* test_default_api_base
* fix new utils tests
* bump: version 1.73.7 → 1.74.0
* Add mcp server segregation comma separated support (#12326)
* add mcp comma separated support
* dont support legacy json array
* lint
* add live tail state to session storage (#12335)
* add aiml
* fix: fix unused imports
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
Co-authored-by: Jugal D. Bhatt <55304795+jugaldb@users.noreply.github.com>
Co-authored-by: Joost van Doorn <joost.van.doorn@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
