litellm/tests/unit/proxy/hooks
Agung Besti c482013a8a
fix(guardrails): log requests that a guardrail blocks on /apply_guardrail (#51014)
* fix(guardrails): record the guardrail verdict on a request dict that starts empty

The Bedrock guardrail and the shared `log_guardrail_information` wrapper both picked the request dict with `or {}`. An empty dict is falsy, so a caller that passed one had the verdict written to a throwaway dict. /apply_guardrail passes an empty dict for a request that carries no messages and no metadata, so nothing downstream could see what the guardrail decided. The verdict now goes to the dict the caller passed, and only a missing dict falls back to a new one.

* fix(guardrails): log requests that a guardrail blocks on /apply_guardrail

A request a guardrail blocked on /apply_guardrail was missing from the logs. The endpoint skipped the failure handlers for an HTTPException, the spend log failure hook wrote no row because the route is not an LLM API route, and the guardrail's verdict stayed on a request dict no logger reads.

The endpoint now runs the failure handlers for every exception and copies the verdict onto the logged request, so allowed and blocked requests both carry it along with the guardrail's cost. It marks the failure of a request it had set up with a sentinel, and the spend log failure hook writes its row for that mark. A request rejected before setup, such as one with a bad key or an unknown guardrail, is still not logged. Guardrail records a caller sends in metadata are dropped for both response formats so a verdict cannot be forged in the logs.

* fix(guardrails): keep Presidio and Gray Swan verdicts on empty request data

Presidio's text-only path and Gray Swan's API-failure path replaced an empty request dict with a throwaway one, so the verdict never reached the logged request. The apply_guardrail logging tests now run the real request setup and read what a logging callback receives.

* test(guardrails): cover the endpoint failure row, the sync decorator, and an unreachable Gray Swan without real sockets

* fix(logging): redact prompt text in guardrail records when message logging is off

* fix(logging): redact the failure payload when message logging is off

* fix(logging): keep guardrail details for other callbacks when one opts out of message logging

* fix(logging): redact failure events per callback and keep guardrail compression stats

A callback that turns off message logging now gets the same redacted copy on failure events as on success events, so a blocked /apply_guardrail call no longer hands it the raw prompt and verdict

Guardrail redaction keeps the numeric compression stats (tokens saved, compression ratio) that spend tracking reads, through one helper shared with the spend-log sanitizer

* fix(policy): test each batch input on its own copy of the request data

The guardrail logging decorator now records a verdict on an empty request dict instead of dropping it. The policy test endpoint shared one request dict across every input and merged it into the agent call, so each later input and agent request carried the earlier inputs' verdicts. Each input now gets its own copy, which keeps the caller's request data unchanged

* test(policy): look up the counting guardrail through the real registry

The batch isolation test now registers its guardrail as a litellm callback, so the endpoint finds it through the real guardrail registry instead of a mocked one, and its apply_guardrail override carries the same concrete types as the base class

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-16 15:18:08 -08:00
..
litellm_skills ci(unit): cut unit check time with cheaper per-test resets and fewer non-unit tests (#51864) 2026-11-14 15:41:30 -08:00
__init__.py ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903) 2026-09-24 22:59:11 +00:00
test_async_post_call_streaming_iterator_hook.py fix(guardrails): run the end-of-stream post_call scan when the client disconnects mid-stream (#43839) 2026-10-05 01:44:43 -07:00
test_autorouter_baseline_cache.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_banned_keyword_list.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_batch_enqueued_tokens.py feat(proxy): add per-model project batch enqueued token limits (#51608) 2026-11-14 14:36:53 -05:00
test_batch_file_validation.py feat(proxy): add per-model project batch enqueued token limits (#51608) 2026-11-14 14:36:53 -05:00
test_batch_rate_limiter.py fix(proxy): rewrite batch JSONL alias rows to the deployment model on model-routed uploads (#51843) 2026-11-16 15:12:52 -08:00
test_dynamic_rate_limiter.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_dynamic_rate_limiter_v3.py fix(dynamic_rate_limiter): keep counting and reporting priority requests after a priority pool's tokens run out (#52081) 2026-11-16 22:00:36 +00:00
test_image_generation_guardrails.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_key_management_event_hooks.py feat(control_plane): dataplane registry, credential strategies and client (#49655) 2026-11-16 17:36:23 +00:00
test_max_budget_per_session_limiter.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_max_iterations_limiter.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_message_thread_ownership.py feat(anthropic): forward message threads on native /v1/messages (#50976) 2026-11-10 18:33:54 -08:00
test_model_budget_shared_windows.py feat(proxy)!: model family and access group entries in model_max_budget (#47722) 2026-10-28 15:40:57 -07:00
test_model_max_budget_limiter.py feat: add OpenRouter-style -latest model aliases for first-party providers (#50315) 2026-11-06 18:22:38 +08:00
test_parallel_request_limiter.py fix(proxy): count native Responses WebSocket turn tokens toward key and tag TPM limits (#49257) 2026-10-31 02:53:27 -07:00
test_parallel_request_limiter_v3.py feat(dynamic_rate_limiter): customer organizations and keys borrow idle model capacity and split class shares (#51705) 2026-11-15 12:05:58 -08:00
test_post_call_failure_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_response_headers_hook.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_streaming_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_success_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_priority_class_share.py feat(dynamic_rate_limiter): customer organizations and keys borrow idle model capacity and split class shares (#51705) 2026-11-15 12:05:58 -08:00
test_prompt_cache_observer.py fix(proxy): keep a retried /v1/messages stream's spend log by logging the superseded attempt as a failure (#45136) 2026-10-16 21:19:58 +00:00
test_prompt_injection_detection.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_proxy_hooks_init.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_proxy_rate_limit_provider_field.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_proxy_track_cost_callback.py fix(guardrails): log requests that a guardrail blocks on /apply_guardrail (#51014) 2026-11-16 15:18:08 -08:00
test_rate_limiter_toctou.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_search_free_tier.py fix(proxy): scope the search free-tier counter to the request's provider (#47646) 2026-10-20 22:01:53 +00:00
test_send_invite_email.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_sensitive_data_routing.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_tpm_concurrent.py perf(token_counter): memoize token counts of long texts per tokenizer (#51904) 2026-11-16 19:21:55 +00:00
test_unit_test_max_model_budget_limiter.py fix(router): inspect image content without translating routed requests (#49371) 2026-10-31 10:44:24 -07:00
test_user_management_event_hooks.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00