scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 51

fix(streaming): skip role-only first chunk when logprobs.content is empty

Browse source 
When logprobs are requested, OpenAI's first streaming chunk has
role='assistant' and logprobs=ChoiceLogprobs(content=[]). The Azure
streaming fix (PR #25638) made is_chunk_non_empty() treat any first
chunk with a non-None role as non-empty, which caused this role-only
chunk to be emitted. Callers checking 'logprobs in chunk.choices[0]'
then crashed on logprobs.content[0] (IndexError: list index out of range).

Guard the role condition: don't emit the first chunk as non-empty when
logprobs is present but has no content entries. Azure's use case (no
logprobs, empty choices[]) is unaffected.
This commit is contained in:
Ishaan Jaffer 2026-04-13 16:42:15 -07:00
parent 473f5dc4ba
commit bd5740a367
No known key found for this signature in database
1 changed files with 4 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
litellm/litellm_core_utils/streaming_handler.py
View file

@ -839,6 +839,10 @@ class CustomStreamWrapper:
not self.sent_first_chunk
and hasattr(model_response.choices[0].delta, "role")
and model_response.choices[0].delta.role is not None
and not (
model_response.choices[0].logprobs is not None
and not model_response.choices[0].logprobs.content
)
)
):
return True