mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
fix(responses): merge text-only items with output items in SSE recovery
Some checks are pending
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Some checks are pending
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
When recovering output from raw SSE, OUTPUT_ITEM_DONE and OUTPUT_TEXT_DONE events were treated as mutually exclusive fallbacks. If a stream emitted OUTPUT_ITEM_DONE for some output indices and only OUTPUT_TEXT_DONE for others, the text-only items at the missing indices were silently dropped. Merge both dicts before returning, with OUTPUT_ITEM_DONE entries taking precedence at any shared index (preserving the existing behavior covered by test_transform_response_preserves_output_item_when_text_done_arrives_later). Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
This commit is contained in:
parent
503314cd12
commit
52f42b219a
2 changed files with 39 additions and 4 deletions
|
|
@ -735,11 +735,16 @@ class LiteLLMResponsesTransformationHandler(CompletionTransformationBridge):
|
|||
recovered_text_only_items=recovered_text_only_items,
|
||||
)
|
||||
|
||||
if recovered_output_items:
|
||||
return [item for _, item in sorted(recovered_output_items.items())]
|
||||
# Merge text-only items into the recovered output items. Real
|
||||
# OUTPUT_ITEM_DONE events take precedence at any given output_index,
|
||||
# but text-only items at indices without a matching OUTPUT_ITEM_DONE
|
||||
# must still be preserved (e.g. multi-output responses where some
|
||||
# indices only emitted OUTPUT_TEXT_DONE).
|
||||
merged_items: Dict[int, Dict[str, Any]] = {**recovered_text_only_items}
|
||||
merged_items.update(recovered_output_items)
|
||||
|
||||
if recovered_text_only_items:
|
||||
return [item for _, item in sorted(recovered_text_only_items.items())]
|
||||
if merged_items:
|
||||
return [item for _, item in sorted(merged_items.items())]
|
||||
|
||||
return []
|
||||
|
||||
|
|
|
|||
|
|
@ -743,6 +743,36 @@ def test_transform_response_preserves_output_item_when_text_done_arrives_later()
|
|||
assert result.choices[0].message.content == "Complete output item text"
|
||||
|
||||
|
||||
def test_recover_output_items_merges_text_only_items_at_distinct_indices():
|
||||
"""When OUTPUT_ITEM_DONE covers some indices and OUTPUT_TEXT_DONE covers
|
||||
others, both must be preserved instead of treating them as mutually
|
||||
exclusive fallbacks."""
|
||||
from litellm.completion_extras.litellm_responses_transformation.transformation import (
|
||||
LiteLLMResponsesTransformationHandler,
|
||||
)
|
||||
|
||||
raw_sse = "\n".join(
|
||||
[
|
||||
'data: {"type":"response.output_item.done","output_index":0,"item":{"type":"message","id":"msg_item_0","role":"assistant","status":"completed","content":[{"type":"output_text","text":"From OUTPUT_ITEM_DONE","annotations":[]}]}}',
|
||||
'data: {"type":"response.output_text.done","output_index":1,"content_index":0,"item_id":"msg_text_1","text":"From OUTPUT_TEXT_DONE only"}',
|
||||
"data: [DONE]",
|
||||
"",
|
||||
]
|
||||
)
|
||||
|
||||
recovered = (
|
||||
LiteLLMResponsesTransformationHandler._recover_output_items_from_raw_sse(
|
||||
raw_sse
|
||||
)
|
||||
)
|
||||
|
||||
assert len(recovered) == 2
|
||||
assert recovered[0]["id"] == "msg_item_0"
|
||||
assert recovered[0]["content"][0]["text"] == "From OUTPUT_ITEM_DONE"
|
||||
assert recovered[1]["id"] == "msg_text_1"
|
||||
assert recovered[1]["content"][0]["text"] == "From OUTPUT_TEXT_DONE only"
|
||||
|
||||
|
||||
def test_transform_response_prefers_completed_output_from_raw_sse():
|
||||
from litellm.completion_extras.litellm_responses_transformation.transformation import (
|
||||
LiteLLMResponsesTransformationHandler,
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue