mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
4f7b20ec10
* fix(guardrails): skip streaming guardrail rounds that re-scan cleared output Streaming guardrails scanned the finished answer twice at end of stream whenever the chunk count landed on a multiple of the sampling rate, ran sampled rounds whose payload was identical to the previous one, and on /v1/messages could scan an empty text before the first content chunk. Every redundant round is a paid guardrail provider call. Each endpoint handler now exposes a scan key describing what a round would hand to apply_guardrail (the text so far, plus tool calls once the stream has ended), and the unified streaming hook skips a sampled or end-of-stream round whose key equals the last scanned one or carries nothing to scan yet. Rounds that carry tool calls are never skipped. * test(guardrails): expect one end-of-stream scan when the terminal chunk is sampled Update sampled cadence expectations and use tuple-backed scan state Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| chat | ||
| completion | ||
| embeddings/guardrail_translation | ||
| evals | ||
| image_generation | ||
| realtime | ||
| responses | ||
| speech | ||
| transcriptions | ||
| vector_store_files | ||
| vector_stores | ||
| videos | ||
| test_cost_calculation.py | ||
| test_data_residency.py | ||
| test_gpt5_transformation.py | ||
| test_is_model_gpt_5_model.py | ||
| test_o_series_transformation.py | ||
| test_openai_common_utils.py | ||
| test_openai_empty_response.py | ||
| test_openai_file_content_streaming.py | ||
| test_openai_image_edit_transformation.py | ||
| test_openai_workload_identity.py | ||
| test_use_chat_completions_api_no_leak.py | ||