mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-20 00:11:27 +00:00
ce22e0bb15
Every streamed event that persists to a chat (status updates, citations, file attachments, message content) serialized the entire conversation JSON three times: a null-byte check of the stored row, a second sanitize of the whole blob after merging in the event payload and the flush of the UPDATE itself. The middle pass rescans megabytes of already-clean history for null bytes that can only come from the small incoming payload, so long chats pay for their full history on every single event. The write paths now sanitize just the incoming message, message id and status dict and keep the row-level sanitize, so legacy rows with null bytes still self-heal as before. Median per-event write time (sqlite, orjson): 1 MB chat 15.0 ms to 11.1 ms, 4 MB 65.2 ms to 52.4 ms, 10 MB 159.9 ms to 124.5 ms, roughly 20 percent less per event. As a side effect the chat_message dual write now receives the sanitized message; previously null bytes in non-content fields were cleaned in the blob but written raw to chat_message, which failed that insert on PostgreSQL. Verified byte-identical rows against the previous implementation across nine scenarios covering null bytes in every input, legacy dirty rows, a missing title and a NULL chat column. |
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| events.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||