litellm/litellm-rust
Yujong Lee edebb26e57 refactor(rust): own the SSE parser and yield smithy messages from the framing crate
The AWS event stream framer now yields aws_smithy_types Message directly
instead of copying headers and payload into a second struct, which halves
its allocations (16 to 8 per frame) and cuts wall time by about 30%. It
also parses from a slice and advances the buffer rather than splitting and
freezing it.

SSE parsing no longer goes through sse-stream. The new parser follows the
WHATWG event stream grammar (unknown fields ignored, colonless lines are
empty fields, last event line wins, non-digit retry and NUL ids ignored,
events without data are not dispatched), caps an event at MAX_FRAME_BYTES
so an unterminated upstream cannot grow memory without bound, and reports
upstream body failures as Error::Body like the AWS path does. Allocations
stay at two per event and wall time is within a few percent.

The Framer trait had no generic consumer, so both parsers are plain
functions now. MissingStreamData in litellm-core is gone because a
dispatched SSE event always carries data.
2026-09-17 15:49:14 -07:00
..
crates refactor(rust): own the SSE parser and yield smithy messages from the framing crate 2026-09-17 15:49:14 -07:00
.gitignore feat: add LiteLLM Rust workspace with Mistral OCR bridge (#31033) 2026-06-23 13:16:47 -07:00
Cargo.lock refactor(rust): own the SSE parser and yield smithy messages from the framing crate 2026-09-17 15:49:14 -07:00
Cargo.toml merge: resolve conflicts with main for anthropic layout rename 2026-09-17 18:09:27 +00:00