mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
8289ac7de3
Adds a bounded Redis stream log for every in-flight assistant message so
clients can reconnect mid-stream (page refresh, network drop, device
switch) and catch up on frames they missed without re-fetching the full
chat from the database.
Problem this solves
-------------------
With ENABLE_REALTIME_CHAT_SAVE=False (default), the backend does not
write the assistant message to the DB until the stream finishes. If the
client refreshes the page mid-stream, chat load from the DB returns
nothing for the in-progress message and the response appears to vanish
until the stream eventually completes. Users staring at an empty chat
while the backend quietly keeps emitting tokens into the void.
Design
------
* Every outbound WS envelope gets stamped with a monotonic per-message
`seq` inside `get_event_emitter` and appended to a bounded Redis
stream keyed `{REDIS_KEY_PREFIX}:stream:{message_id}`.
MAXLEN ~ 2000 entries, TTL 1h as a safety net.
* Clients track `message.lastSeq` in Chat.svelte as events arrive. On
chat load (mid-stream refresh) and on socket reconnect they emit
`resume-stream {chat_id, message_id, last_seq}`.
* The server authenticates the session, verifies the user owns the
chat, XRANGEs the log, filters by `seq > last_seq`, and emits the
missed envelopes to THAT session only (via `to=sid`) so live listeners
in the user room keep receiving their normal live stream unchanged.
* The existing chat event handler drops any envelope with
`seq <= message.lastSeq`, making replay idempotent against live
frames that race the replay after a reconnect.
* When an event with `done: True` fires, a background task truncates
the log after a 30s grace window so late reconnects still catch the
finalization; anything beyond that resumes from the now-up-to-date DB.
Orthogonality
-------------
Zero touches to middleware.py or the streaming hot path. The log stores
whatever gets emitted; any future change to emit shape
(chat:message:delta, per-block ops, JSON Patch, ...) is logged and
replayed verbatim with no coupling.
Graceful degradation
--------------------
No-op when Redis is not configured (WEBSOCKET_MANAGER != 'redis'). In
that deployment mode, refresh during streaming retains the current
behavior of waiting for the stream to complete.
Auth model
----------
The log is keyed by message_id only. The resume handler must do a chat
ownership check (Chats.get_chat_by_id_and_user_id) before replaying, so
a malicious client cannot read another user's stream by guessing a
message_id.
|
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| test | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||