mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-05 02:41:34 +00:00
perf(ui): avoid structuredClone per token during streaming
During LLM streaming, the reactive sync in ResponseMessage.svelte called structuredClone(source) on every token because content always differs. This deep-cloned the entire message object including statusHistory, sources, code_executions, and annotations — all data that never changes during streaming. Replace the fast-path clone with Object.assign to update only the fields that change during streaming (content, done, sources, code_executions, statusHistory). Object.assign is used instead of direct property assignments because Svelte 4 compiles each message.prop = value into a separate $$invalidate() call — five assignments = five invalidations per token. Object.assign is a function call Svelte does not instrument, so only the final message = message triggers a single $$invalidate, matching the old approach. Real-world A/B test (1278 tokens, Chrome): structuredClone 62ms vs Object.assign 18ms — 3.4x faster, 44ms saved per stream.
This commit is contained in:
parent
b1d40f3409
commit
0dd5fa8575
1 changed files with 13 additions and 4 deletions
|
|
@ -125,12 +125,21 @@
|
|||
$: if (history.messages) {
|
||||
const source = history.messages[messageId];
|
||||
if (source) {
|
||||
// Fast path: O(1) check on the fields that change most often (content during streaming, done at end)
|
||||
// Avoids 2x O(n) JSON.stringify calls that are always true during streaming anyway
|
||||
if (message.content !== source.content || message.done !== source.done) {
|
||||
message = structuredClone(source);
|
||||
// Streaming fast path: use Object.assign to sync fields that change
|
||||
// during streaming instead of deep-cloning the entire message object
|
||||
// on every token. Object.assign avoids per-property $$invalidate
|
||||
// calls that Svelte 4 compiles for direct property assignments.
|
||||
Object.assign(message, {
|
||||
content: source.content,
|
||||
done: source.done,
|
||||
sources: source.sources,
|
||||
code_executions: source.code_executions,
|
||||
statusHistory: source.statusHistory
|
||||
});
|
||||
message = message;
|
||||
} else if (!equal(message, source)) {
|
||||
// Slow path: full comparison for infrequent changes (sources, annotations, status, etc.)
|
||||
// Structural change (annotations, files, embeds, etc.): full clone
|
||||
message = structuredClone(source);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue