litellm/tests/unit/proxy/guardrails
paveln83 06cdee91da
fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails (#49035)
* feat(guardrails): scan and rewrite streamed /v1/completions in post_call pipelines

Gives OpenAITextCompletionHandler a process_output_streaming_response that
assembles the buffered TextCompletionResponse chunks per choice index at end
of stream, runs the guardrail over the assembled text, and writes text
rewrites back into the first text-carrying chunk of each choice. Tests and
live proof still pending.

* fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails

Assemble each choice's streamed text once the stream ends, run the guardrail on it, write a mask back into the buffered chunks, and end the stream with the guardrail's error on a block. User key metadata now goes through the shared metadata bucket so applied guardrails reach spend logs.

* test(guardrails): cover streamed /v1/completions under post_call guardrails

* fix(guardrails): keep Gray Swan key metadata, scan a cut-short text stream, and pass non-chunk frames through

* fix(guardrails): replace blocked text completion streams like chat, keep Gray Swan's vendor payload, scan text streams in the content filter

* fix(guardrails): release an unfinished choice's held text and omit usage the upstream never sent

The content filter's streaming hook held back the end of a choice until its
finish_reason, which litellm sends only for the first choice of an n>1
stream, so choice 1 lost its last characters on text and chat streams. A
closing chunk now releases every unfinished choice when the stream ends.

A blocked text stream's closing chunk reported 0/0 usage when the upstream
sent none; it now carries usage only when the upstream reported it.

* fix(guardrails): send no usage on the content filter's end-of-stream chunk

* fix(guardrails): keep the usage chunk last and log the masked text of streamed text completions

* fix(guardrails): stream Responses API events through the content filter as they arrive

* fix(guardrails): treat a null text completion choice index as choice zero when assembling streams

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 08:46:49 -07:00
..
guardrail_hooks fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails (#49035) 2026-10-29 08:46:49 -07:00
__init__.py feat(guardrails): send a configured gateway_name from noma_v2 to Noma (#43678) 2026-09-29 10:20:40 -07:00
test_anthropic_sse.py fix(anthropic): carry safeguard_results and stop_details in synthesized message_delta events (#46096) 2026-10-17 16:48:37 +00:00
test_auto_router_compression.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_content_filter_path_traversal.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_content_utils.py fix(guardrails): skip embeddings input in google_text_moderation (#48934) 2026-10-28 20:05:03 -07:00
test_custom_code_security.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_deferred_guardrail_logging.py fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails (#49035) 2026-10-29 08:46:49 -07:00
test_guardrail_coverage.py fix(guardrails): skip embeddings input in google_text_moderation (#48934) 2026-10-28 20:05:03 -07:00
test_guardrail_endpoints.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
test_guardrail_initializers.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_guardrail_load_balancing.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_guardrail_registry.py feat(guardrails): split logging_only_scope direction from input-failure policy (#45709) 2026-10-10 01:46:39 -07:00
test_init_guardrails.py fix(config): treat blank env values as unset so the proxy neither crash-loops nor silently drops a guardrail (#46341) 2026-10-14 12:49:31 -07:00
test_llm_as_a_judge.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_mcp_jwt_signer.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_pillar_guardrails.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_prompt_security_guardrails.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_qostodian_nexus_guardrail.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_usage_endpoints.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00
test_usage_tracking.py test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy (#44003) 2026-10-01 10:52:03 -07:00