fix(responses): merge text-only items with output items in SSE recovery
Some checks are pending
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run

When recovering output from raw SSE, OUTPUT_ITEM_DONE and OUTPUT_TEXT_DONE
events were treated as mutually exclusive fallbacks. If a stream emitted
OUTPUT_ITEM_DONE for some output indices and only OUTPUT_TEXT_DONE for
others, the text-only items at the missing indices were silently dropped.

Merge both dicts before returning, with OUTPUT_ITEM_DONE entries taking
precedence at any shared index (preserving the existing behavior covered
by test_transform_response_preserves_output_item_when_text_done_arrives_later).

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
This commit is contained in:
Cursor Agent 2026-05-05 02:57:58 +00:00
parent 503314cd12
commit 52f42b219a
No known key found for this signature in database
2 changed files with 39 additions and 4 deletions

View file

@ -735,11 +735,16 @@ class LiteLLMResponsesTransformationHandler(CompletionTransformationBridge):
recovered_text_only_items=recovered_text_only_items,
)
if recovered_output_items:
return [item for _, item in sorted(recovered_output_items.items())]
# Merge text-only items into the recovered output items. Real
# OUTPUT_ITEM_DONE events take precedence at any given output_index,
# but text-only items at indices without a matching OUTPUT_ITEM_DONE
# must still be preserved (e.g. multi-output responses where some
# indices only emitted OUTPUT_TEXT_DONE).
merged_items: Dict[int, Dict[str, Any]] = {**recovered_text_only_items}
merged_items.update(recovered_output_items)
if recovered_text_only_items:
return [item for _, item in sorted(recovered_text_only_items.items())]
if merged_items:
return [item for _, item in sorted(merged_items.items())]
return []

View file

@ -743,6 +743,36 @@ def test_transform_response_preserves_output_item_when_text_done_arrives_later()
assert result.choices[0].message.content == "Complete output item text"
def test_recover_output_items_merges_text_only_items_at_distinct_indices():
"""When OUTPUT_ITEM_DONE covers some indices and OUTPUT_TEXT_DONE covers
others, both must be preserved instead of treating them as mutually
exclusive fallbacks."""
from litellm.completion_extras.litellm_responses_transformation.transformation import (
LiteLLMResponsesTransformationHandler,
)
raw_sse = "\n".join(
[
'data: {"type":"response.output_item.done","output_index":0,"item":{"type":"message","id":"msg_item_0","role":"assistant","status":"completed","content":[{"type":"output_text","text":"From OUTPUT_ITEM_DONE","annotations":[]}]}}',
'data: {"type":"response.output_text.done","output_index":1,"content_index":0,"item_id":"msg_text_1","text":"From OUTPUT_TEXT_DONE only"}',
"data: [DONE]",
"",
]
)
recovered = (
LiteLLMResponsesTransformationHandler._recover_output_items_from_raw_sse(
raw_sse
)
)
assert len(recovered) == 2
assert recovered[0]["id"] == "msg_item_0"
assert recovered[0]["content"][0]["text"] == "From OUTPUT_ITEM_DONE"
assert recovered[1]["id"] == "msg_text_1"
assert recovered[1]["content"][0]["text"] == "From OUTPUT_TEXT_DONE only"
def test_transform_response_prefers_completed_output_from_raw_sse():
from litellm.completion_extras.litellm_responses_transformation.transformation import (
LiteLLMResponsesTransformationHandler,