mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
A chat-completions call on a responses-mode deployment wraps the BaseResponsesAPIStreamingIterator in a CustomStreamWrapper and both layers share one Logging object, so a mid-stream failure has two dispatch sites: the inner iterator fires the failure handlers before re-raising and the outer wrapper fires them again for the same exception. The dedup that stops the second dispatch is has_logged_async_failure, which mark_logging_complete must keep setting for failure events on streams: the stream=True carve-out applies to success events only, because a stream is one request whose failure must be logged once, while success markers are per-chunk and cannot be one-shot. Before the logging split this marker was skipped for every event type on a stream, so CustomLogger.async_log_failure_event fired twice for the same exception object with identical start_time/end_time, and the fire-and-forget outer task often ran after the router had started a fallback hop, so the duplicate got attributed to whatever deployment the logging object described at that moment (reported in #46557 as a fallback deployment showing a ~50% phantom error rate). Pin both failure shapes with a regression test that rebuilds the exact bridge stack (SSE responses iterator, responses-to-chat translation iterator, CustomStreamWrapper, one Logging object) and asserts exactly one async failure event per failed stream, both after streamed output and before the first chunk. Verified both directions: on current main exactly one event is dispatched; reverting mark_logging_complete to the pre-split behavior (skip the marker for all event types on streams) makes both tests fail with two events carrying identical exception objects and identical timestamps, matching the issue report. Refs #46557 Signed-off-by: choksi2212 <choksi2212@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| litellm_responses_transformation | ||
| __init__.py | ||
| test_litellm_responses_transformation_transformation.py | ||
| test_responses_bridge_provider_propagation.py | ||