mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-17 23:52:29 +00:00
d7b36be54d
The streaming hot path in `streaming_chat_response_handler.stream_body_handler` was calling `serialize_output(full_output())` on every SSE event — rebuilding an HTML string of the entire accumulated output (text + reasoning + tool calls + images + citations) and emitting it via Socket.IO. For an N-token response this is O(N) work per token -> O(N^2) bytes across the WebSocket, then amplified again by Socket.IO's AsyncRedisManager pub/sub (x redis nodes x workers) and a third time by the frontend Markdown re-parser. A 2000-token reply at normal scale could push ~10,000x more bytes through Redis than the new tokens actually needed. Fix: when an SSE event only appended text characters to the currently active `message` block (the overwhelming common case), accumulate those chars into a `pending_text_delta` buffer and emit them as a lightweight `chat:message:delta` event. The frontend already handles that event via string append (`Chat.svelte:472-473`), so no frontend changes are needed. Any structural event - reasoning update, reasoning->message transition, tool call, image attachment, tag-block append, annotation source, etc. - still goes through the legacy full-serialize path, and when it does it clears the pending text delta buffer (because the full-content checkpoint already contains the accumulated text). Ordering is preserved by flushing pending deltas before each structural emit. The final `done: true` `chat:completion` emitted by the outer handler at line 4793 overwrites `message.content` with the canonical `serialize_output(output)`, reconciling the frontend to the exact backend state at the end of every response. Discovered and diagnosed by @Shirasawa; this is the Phase 1 minimum-viable landing. Phase 2 (typed block model, per-block rendering, Redis op log with resume-by-seq) is tracked as follow-up work. Impact summary: - Per-token WS payload: O(size_so_far) -> O(new_chars) - Redis pub/sub bytes: drops by ~10-100x on long responses - Frontend Markdown re-parse cost: now bounded by delta size per event, not full response size - Semantics: identical. Final message content reconciles via the existing `done: true` full-content checkpoint. Out of scope for this change: - Reasoning-block content still takes the full-serialize path per token (Phase 2 will incrementalize it via per-block deltas) - Tool-call argument streaming still takes the full-serialize path - DB persistence path (REALTIME_CHAT_SAVE) is untouched — its WS path already emitted raw SSE deltas and is unchanged |
||
|---|---|---|
| .. | ||
| access_control | ||
| images | ||
| mcp | ||
| telemetry | ||
| actions.py | ||
| anthropic.py | ||
| asgi_middleware.py | ||
| audit.py | ||
| auth.py | ||
| automations.py | ||
| channels.py | ||
| chat.py | ||
| code_interpreter.py | ||
| embeddings.py | ||
| files.py | ||
| filter.py | ||
| groups.py | ||
| headers.py | ||
| logger.py | ||
| middleware.py | ||
| misc.py | ||
| models.py | ||
| oauth.py | ||
| payload.py | ||
| pdf_generator.py | ||
| plugin.py | ||
| rate_limit.py | ||
| redis.py | ||
| response.py | ||
| sanitize.py | ||
| security_headers.py | ||
| session_pool.py | ||
| task.py | ||
| tools.py | ||
| validate.py | ||
| webhook.py | ||