mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
627bcab42b
* fix(guardrails): honor streaming settings on the Noma v2 guardrail The Noma v2 initializer now passes streaming_buffer_until_moderated, streaming_end_of_stream_only, streaming_sampling_rate and streaming_buffer_release_on_scan from the guardrail config to NomaV2Guardrail, so the unified streaming hook applies them. The constructor validates them, so a custom guardrail class built on NomaV2Guardrail gets the same checks. Unset keys keep the hook defaults, and an invalid value is ignored with a warning naming the guardrail, so the guardrail still loads. Resolves LIT-9633 * test(guardrails): cover Noma v2 streaming settings across clients, failures and workers Adds integration cells for the custom-subclass loader, string and invalid values, unset and false keys, a Noma outage with and without block_failures, an upstream rejection, n=2 choices, pre_call and non-streaming guardrails, the OpenAI and Anthropic SDKs (sync and async), stored-guardrail info, a PATCH while traffic flows, a burst while Noma is down, a slow Noma, and replaced workers. Resolves LIT-9633 * test(guardrails): pin the PATCH probes to one worker at a time Each probe suspends the other worker and opens a fresh connection, so the test checks that every worker applies the patched setting. Resolves LIT-9633 * test(guardrails): check sampled scans by count, not window position /v1/messages hands the streaming hook raw network reads, so where the default sampling windows fall depends on timing; the base leg caught one shifted window. Check the scan count and the final full-text scan instead. Resolves LIT-9633 --------- Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| faked_responses_case.py | ||
| test_assistants_evals_stream.py | ||
| test_azure_streaming_function_calls.py | ||
| test_decision_model_buffering.py | ||
| test_decision_model_pipeline_buffering.py | ||
| test_faked_responses_stream_bridges.py | ||
| test_faked_responses_stream_cache_replay.py | ||
| test_faked_responses_stream_chaos.py | ||
| test_faked_responses_stream_guardrail_blocks.py | ||
| test_faked_responses_stream_items.py | ||
| test_file_content_streaming.py | ||
| test_google_genai_stream_usage_settings_wire.py | ||
| test_interception_converted_stream_options.py | ||
| test_noma_v2_streaming_settings.py | ||
| test_openai_like_stream_usage_wire.py | ||
| test_openai_stream_passthrough_fields.py | ||
| test_responses_bridge_stream_chaos.py | ||
| test_responses_bridge_stream_errors.py | ||
| test_responses_top_level_error_events.py | ||
| test_stream_contracts.py | ||
| test_stream_open_error_provider_headers_proxy.py | ||
| test_stream_parallel_slot_release.py | ||
| test_ttft_keepalive.py | ||