litellm/litellm-rust/crates
Yujong Lee edebb26e57 refactor(rust): own the SSE parser and yield smithy messages from the framing crate
The AWS event stream framer now yields aws_smithy_types Message directly
instead of copying headers and payload into a second struct, which halves
its allocations (16 to 8 per frame) and cuts wall time by about 30%. It
also parses from a slice and advances the buffer rather than splitting and
freezing it.

SSE parsing no longer goes through sse-stream. The new parser follows the
WHATWG event stream grammar (unknown fields ignored, colonless lines are
empty fields, last event line wins, non-digit retry and NUL ids ignored,
events without data are not dispatched), caps an event at MAX_FRAME_BYTES
so an unterminated upstream cannot grow memory without bound, and reports
upstream body failures as Error::Body like the AWS path does. Allocations
stay at two per event and wall time is within a few percent.

The Framer trait had no generic consumer, so both parsers are plain
functions now. MissingStreamData in litellm-core is gone because a
dispatched SSE event always carries data.
2026-09-17 15:49:14 -07:00
..
auth refactor(rust): extract auth and cache crates 2026-09-16 18:12:11 +00:00
auth-aws refactor(rust): extract auth and cache crates 2026-09-16 18:12:11 +00:00
auth-azure refactor(rust): extract auth and cache crates 2026-09-16 18:12:11 +00:00
auth-gcp refactor(rust): extract auth and cache crates 2026-09-16 18:12:11 +00:00
cache refactor(rust): extract auth and cache crates 2026-09-16 18:12:11 +00:00
cache-memory fix(rust): evict in-memory cache entries at the expiry boundary 2026-09-16 18:30:45 +00:00
cache-redis fix(rust): address Redis cache review findings 2026-09-16 23:19:28 +00:00
core refactor(rust): own the SSE parser and yield smithy messages from the framing crate 2026-09-17 15:49:14 -07:00
framer refactor(rust): own the SSE parser and yield smithy messages from the framing crate 2026-09-17 15:49:14 -07:00
python-bridge merge: port OCR request validation and upstream error mapping onto main's dispatch layout 2026-09-17 18:33:20 +00:00
python-interop refactor(ocr): complete native lifecycle and preserve Azure auth (#40734) 2026-09-12 11:56:49 -07:00
token-counter test(rust): assert merge cost scales linearly instead of a wall-clock bound 2026-09-16 19:43:42 +00:00