mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
8d300d98d8
Classifying a turn is a read-modify-write over the session's stored state, and the read and the modify were happening on the request path while the write happened in the background flusher. Splitting one unit of work across two contexts is what produced all three findings from the last review round. The logging path now only stages the turn's facts; record_turn no longer takes a prisma client at all, so a database round trip in front of spend processing is not expressible. The flusher owns the whole read, fold and write per session, so a read that faults has no half-finished write to corrupt. It re-stages the turns through the same path a failed write already used, and the except branch that returned an empty state, then persisted it over real history, is gone rather than guarded. Folding at flush time also lets an interval's turns be sorted by start time before they are classified. Completion order is not start order, so two turns in flight together used to leave the earlier one read as a late arrival and dropped from every bucket; at arrival there is no later turn to compare against, so this was not fixable in the previous shape. StateUnavailable separates "the read failed" from "this session has no history", which were the same value before. Only the second one is writable. The staging ceiling counts turns rather than sessions, which is the quantity that actually bounds the memory a caller sending a fresh session id per request can make the proxy hold. Benchmark window dates are typed as dates on the route, so a malformed one is rejected by the framework instead of raising inside the aggregate, and an inverted range is a 400 instead of an empty dashboard that reads as no traffic |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||