scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-01 02:02:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 91

fix(anthropic): flush held content chunk in async streaming path

Browse source 
Mirror the sync __next__ behavior: always flush a buffered
holding_chunk after the stream ends, even when usage was already
merged + emitted. Previously the async __anext__ kept the flush
inside the 'if not self.queued_usage_chunk:' guard, silently
dropping the last content delta on the proxy's primary path.

Co-authored-by: Yassin Kortam <yassin@berri.ai>
This commit is contained in:
Cursor Agent 2026-05-27 14:29:29 +00:00
parent 88ec260771
commit 108798b78f
No known key found for this signature in database
1 changed files with 8 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py
View file

@ -614,9 +614,14 @@ class AnthropicStreamWrapper(AdapterCompletionStreamWrapper):
)
self.holding_stop_reason_chunk = None
if self.holding_chunk is not None:
self.chunk_queue.append(self.holding_chunk)
self.holding_chunk = None
# Always flush any buffered content delta, even when usage has
# already been merged + emitted: dropping it would silently lose
# provider-emitted content, which is worse than the SSE ordering
# nit of trailing a content chunk after the final message_delta
# (the prior async ``__anext__`` behavior).
if self.holding_chunk is not None:
self.chunk_queue.append(self.holding_chunk)
self.holding_chunk = None
if not self.sent_last_message:
self.sent_last_message = True