Commit graph

4 commits

Author SHA1 Message Date
yujonglee
271b4f0306
refactor(rust): one stream codec shape per API format (#45996)
Chat Completions, Messages and Responses each get a streaming module with
the same three names, event_stream, decode and encode, built on a shared
SSE helper in base_llm/sse.rs that owns the [DONE] marker. Callers use the
module path instead of vendor-prefixed names. Encoding matches what the
Python proxy writes to clients: Messages frames carry an event line, the
OpenAI formats are bare data frames.

Responses gets one event type for SSE and WebSocket. ResponsesEventType
names every event on OpenAI's streaming-events reference, with unknown
names riding in Other, and is_terminal and is_failure follow Python's
iterator. The framer adds a bulk decode_all and an SseBlockCodec that
keeps each event's wire bytes. An event the stream ends inside is dropped,
as the SSE spec and the openai SDK's decoder do.

ChatCompletionChunk no longer requires id, created or object, since
compatible hosts omit them.
2026-10-13 15:30:14 -07:00
devin-ai-integration[bot]
74ad633e53
refactor(rust): rename litellm-framing crate to litellm-framer (#44683)
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 22:32:00 +00:00
devin-ai-integration[bot]
3c93ea1697
refactor(framer): replace Framer trait with tokio-util codecs (#43193)
* refactor(framer): replace Framer trait with tokio-util codecs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(framer): port SSE and AWS event stream framing to codecs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): drop clone on Copy capabilities in messages request test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): use field init shorthand in messages request test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-25 12:18:32 -07:00
Yujong Lee
1ef094bb41 feat(rust): add standalone framing crate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 00:30:43 +00:00