perf(ui): avoid structuredClone per token during streaming

During LLM streaming, the reactive sync in ResponseMessage.svelte
called structuredClone(source) on every token because content always
differs. This deep-cloned the entire message object including
statusHistory, sources, code_executions, and annotations — all data
that never changes during streaming.

Replace the fast-path clone with Object.assign to update only the
fields that change during streaming (content, done, sources,
code_executions, statusHistory). Object.assign is used instead of
direct property assignments because Svelte 4 compiles each
message.prop = value into a separate $$invalidate() call — five
assignments = five invalidations per token. Object.assign is a
function call Svelte does not instrument, so only the final
message = message triggers a single $$invalidate, matching the old
approach.

Real-world A/B test (1278 tokens, Chrome): structuredClone 62ms vs
Object.assign 18ms — 3.4x faster, 44ms saved per stream.
This commit is contained in:
silentoplayz 2026-06-08 13:40:23 -04:00
parent b1d40f3409
commit 0dd5fa8575

View file

@ -125,12 +125,21 @@
$: if (history.messages) {
const source = history.messages[messageId];
if (source) {
// Fast path: O(1) check on the fields that change most often (content during streaming, done at end)
// Avoids 2x O(n) JSON.stringify calls that are always true during streaming anyway
if (message.content !== source.content || message.done !== source.done) {
message = structuredClone(source);
// Streaming fast path: use Object.assign to sync fields that change
// during streaming instead of deep-cloning the entire message object
// on every token. Object.assign avoids per-property $$invalidate
// calls that Svelte 4 compiles for direct property assignments.
Object.assign(message, {
content: source.content,
done: source.done,
sources: source.sources,
code_executions: source.code_executions,
statusHistory: source.statusHistory
});
message = message;
} else if (!equal(message, source)) {
// Slow path: full comparison for infrequent changes (sources, annotations, status, etc.)
// Structural change (annotations, files, embeds, etc.): full clone
message = structuredClone(source);
}
}