litellm/ui/litellm-dashboard/src
devin-ai-integration[bot] 02ef44c388
fix(streaming): convert upstream logprobs so streams with logprobs end with their finish chunk (#33549)
* fix(streaming): normalize provider logprobs to prevent MockValSer serialization crash

* fix(streaming): convert upstream chunk logprobs to litellm ChoiceLogprobs so the finish chunk serializes

A streaming chat completion with logprobs on the OpenAI SDK path attached the
SDK's own ChoiceLogprobs object to the outgoing chunk when the chunk had no
content to rebuild from, which is the provider's finish chunk (empty delta,
logprobs for the end token, finish_reason). The SDK parses stream chunks
without building that nested class, so its pydantic serializer is still a
MockValSer and model_dump_json fails. The proxy then sent an error frame in
place of the finish chunk, and when the same chunk also carried usage the SDK
wrapper itself raised MidStreamFallbackError.

The attach site now validates upstream logprobs through a TypeAdapter that
dumps any non-litellm pydantic model before validating into litellm's
ChoiceLogprobs, and passes litellm instances through unchanged.

* fix(streaming): keep upstream logprobs that don't fit ChoiceLogprobs as plain JSON so the stream still finishes

* fix(streaming): build ChoiceLogprobs' serializer up front and keep nonconforming logprobs as JSON on chunk rebuilds

ChoiceLogprobs inherits the openai SDK's deferred build, so a fresh worker
whose first logprobs arrive on the finish chunk still had no serializer.
Foreign logprobs models now dump in full, so a missing top_logprobs becomes
[] and SDK callers keep attribute access, and chunk rebuilds for content and
usage keep logprobs that don't fit as plain JSON instead of raising.

* fix(streaming): keep only the logprobs keys upstream sent and type refusal logprobs

* fix(types): pass nonconforming non-streaming logprobs through instead of failing the response

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-30 04:41:02 -07:00
..
app feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00
components feat(usage): show subscription-covered providers and account fees separately (#44772) 2026-10-30 03:58:36 -07:00
contexts feat(ui): add LITELLM_UI_USE_SI_TERMINOLOGY to show SI instead of AI in the Admin UI (#47440) 2026-10-27 22:54:39 +00:00
data feat(ui): add LITELLM_UI_USE_SI_TERMINOLOGY to show SI instead of AI in the Admin UI (#47440) 2026-10-27 22:54:39 +00:00
hooks feat(telemetry): add the Admin UI telemetry settings page and gated page navigation events (#45494) 2026-10-13 20:12:11 -07:00
i18n feat(ui): add i18next plumbing and move the Workflows page onto it (#49144) 2026-10-29 17:25:48 -07:00
lib fix(streaming): convert upstream logprobs so streams with logprobs end with their finish chunk (#33549) 2026-10-30 04:41:02 -07:00
utils refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
next-types.d.ts feat(traces): tracing development seed (#44363) 2026-10-03 09:59:51 +00:00
types.ts added and ran prettier autoformatter 2025-10-04 18:19:48 -07:00