mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-10 22:41:41 +00:00
4f7b20ec10
* fix(guardrails): skip streaming guardrail rounds that re-scan cleared output Streaming guardrails scanned the finished answer twice at end of stream whenever the chunk count landed on a multiple of the sampling rate, ran sampled rounds whose payload was identical to the previous one, and on /v1/messages could scan an empty text before the first content chunk. Every redundant round is a paid guardrail provider call. Each endpoint handler now exposes a scan key describing what a round would hand to apply_guardrail (the text so far, plus tool calls once the stream has ended), and the unified streaming hook skips a sampled or end-of-stream round whose key equals the last scanned one or carries nothing to scan yet. Rounds that carry tool calls are never skipped. * test(guardrails): expect one end-of-stream scan when the terminal chunk is sampled Update sampled cadence expectations and use tuple-backed scan state Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| batches | ||
| chat | ||
| experimental_pass_through | ||
| files | ||
| messages | ||
| __init__.py | ||
| test_anthropic_common_utils.py | ||
| test_anthropic_count_tokens_transformation.py | ||
| test_anthropic_files_and_batches.py | ||
| test_anthropic_output_format_filter.py | ||
| test_anthropic_reasoning_effort.py | ||
| test_anthropic_schema_filter.py | ||
| test_anthropic_structured_output.py | ||
| test_azure_ai_cache_pricing.py | ||
| test_cost_calculation_dict_safety.py | ||
| test_count_tokens_oauth.py | ||
| test_message_sanitization.py | ||