mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
d7b36be54d
The streaming hot path in `streaming_chat_response_handler.stream_body_handler` was calling `serialize_output(full_output())` on every SSE event — rebuilding an HTML string of the entire accumulated output (text + reasoning + tool calls + images + citations) and emitting it via Socket.IO. For an N-token response this is O(N) work per token -> O(N^2) bytes across the WebSocket, then amplified again by Socket.IO's AsyncRedisManager pub/sub (x redis nodes x workers) and a third time by the frontend Markdown re-parser. A 2000-token reply at normal scale could push ~10,000x more bytes through Redis than the new tokens actually needed. Fix: when an SSE event only appended text characters to the currently active `message` block (the overwhelming common case), accumulate those chars into a `pending_text_delta` buffer and emit them as a lightweight `chat:message:delta` event. The frontend already handles that event via string append (`Chat.svelte:472-473`), so no frontend changes are needed. Any structural event - reasoning update, reasoning->message transition, tool call, image attachment, tag-block append, annotation source, etc. - still goes through the legacy full-serialize path, and when it does it clears the pending text delta buffer (because the full-content checkpoint already contains the accumulated text). Ordering is preserved by flushing pending deltas before each structural emit. The final `done: true` `chat:completion` emitted by the outer handler at line 4793 overwrites `message.content` with the canonical `serialize_output(output)`, reconciling the frontend to the exact backend state at the end of every response. Discovered and diagnosed by @Shirasawa; this is the Phase 1 minimum-viable landing. Phase 2 (typed block model, per-block rendering, Redis op log with resume-by-seq) is tracked as follow-up work. Impact summary: - Per-token WS payload: O(size_so_far) -> O(new_chars) - Redis pub/sub bytes: drops by ~10-100x on long responses - Frontend Markdown re-parse cost: now bounded by delta size per event, not full response size - Semantics: identical. Final message content reconciles via the existing `done: true` full-content checkpoint. Out of scope for this change: - Reasoning-block content still takes the full-serialize path per token (Phase 2 will incrementalize it via per-block deltas) - Tool-call argument streaming still takes the full-serialize path - DB persistence path (REALTIME_CHAT_SAVE) is untouched — its WS path already emitted raw SSE deltas and is unchanged |
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| test | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||