Commit graph

5 commits

Author SHA1 Message Date
Yassin Kortam
4faeabc254
fix(guardrails): run pre_call hook once for model-level guardrails (#30543)
* fix(guardrails): run pre_call hook once for model-level guardrails

A CustomGuardrail attached to a deployment via litellm_params.guardrails
gets its async_pre_call_hook invoked twice per request: once by the proxy
pre-call loop and again by async_pre_call_deployment_hook after the router
spreads the model-level guardrails into the top-level request kwargs.

Record in request metadata that the proxy pre-call loop already ran a given
guardrail, and have the deployment hook skip it when the marker is present.
Direct-SDK usage never runs the proxy loop, so the deployment hook stays the
sole invocation there and still fires exactly once.

The marker key is stripped from untrusted caller metadata so a request body
cannot suppress a model-only guardrail by pre-seeding it.

* fix(guardrails): mark pre_call dedup on the post-hook request data

Record the exactly-once marker after async_pre_call_hook runs, on the data
object that flows downstream, rather than before it. A guardrail whose hook
returns a brand-new request dict (instead of mutating or spreading the one it
received) would otherwise discard the marker, letting the deployment hook
re-run the guardrail a second time.
2026-06-16 11:17:03 -07:00
michelligabriele
9f1b41d206
fix(proxy): run model-level post_call guardrails on streaming requests (#26922) 2026-05-07 11:53:03 -07:00
Ishaan Jaffer
e8461b5b97
style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
michelligabriele
b0c317e6d0 test: use __file__-relative sys.path for stable test imports
Address Greptile review — resolve sys.path relative to the test file
location instead of the process working directory.
2026-03-16 21:07:07 +01:00
michelligabriele
23b69eea62 fix(proxy): model-level guardrails not executing for non-streaming post_call
Model-level guardrails (litellm_params.guardrails on a deployment) were
only merged into request metadata in the streaming post_call path
(async_post_call_streaming_hook) but not in the non-streaming path
(post_call_success_hook). This caused should_run_guardrail to skip the
guardrail because the guardrail name was never added to metadata.guardrails.

Add the same _check_and_merge_model_level_guardrails call to
post_call_success_hook before the guardrail callback loop, mirroring the
streaming path.

Fixes model-level guardrails silently not firing for non-streaming
post_call requests.
2026-03-16 20:55:22 +01:00