scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 152

revert: remove empty-choices chunk filter — drops usage-bearing final SSE chunk

Browse source 
The _has_empty_choices filter stripped the OpenAI usage-bearing trailing chunk
{"choices":[]"usage":{...}} which caused stream_chunk_builder to fall back to
estimated token counts instead of real provider usage. The try/except around
stream_chunk_builder already handles any IndexError on empty choices safely.
This commit is contained in:
Sameer Kankute 2026-06-29 15:13:52 +05:30
parent 7704572065
commit 924a245edd
No known key found for this signature in database
1 changed files with 0 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
litellm/proxy/common_request_processing.py
View file

@ -2465,17 +2465,6 @@ class ProxyBaseLLMRequestProcessing:
return
elif not isinstance(first_chunk, str) and not hasattr(first_chunk, "choices"):
return
# Strip trailing usage-only / empty-choices chunks (e.g. OpenAI's final SSE
# `data: {"choices":[],"usage":{...}}`) so stream_chunk_builder can safely
# inspect the last chunk's finish_reason without an IndexError.
def _has_empty_choices(c: Any) -> bool:
if isinstance(c, dict):
return c.get("choices") == []
return bool(hasattr(c, "choices") and getattr(c, "choices") == [])
chunks = [c for c in chunks if not _has_empty_choices(c)]
if not chunks:
return
# Optimization, not a correctness guard: dispatch_success_handlers is the
# authoritative de-dup via has_dispatched_final_stream_success. This just
# skips the stream_chunk_builder assembly when completion already logged.