scriptoriumadmin/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-11-14 09:21:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

perf: stop resending the whole reply on every reasoning chunk with OpenRouter-style providers (#33698)

Browse source 
Providers such as OpenRouter attach reasoning details to every reasoning chunk, and each of those chunks made Open WebUI send the entire reply so far to the browser and, with Redis, rewrite the copy kept for page reloads. The browser never shows these details, so a long reasoning reply moved tens of megabytes for a few kilobytes of text. Those chunks now stream like any other reasoning text, and the details are still saved with the reply and still sent back to the provider on the next request.

Measured against a simulated OpenRouter-style provider streaming 1,500 reasoning chunks:

| | before | after |
|---|---|---|
| Data sent to the browser | 73.6 MB | 0.73 MB |
| Redis writes for the reload copy | 1,504 | 19 |
This commit is contained in:
Classic298 2026-11-05 14:30:39 +01:00 • committed by GitHub
parent b6d861645b
commit f59552efff
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 0 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
backend/open_webui/utils/middleware.py
View file

@ -5299,11 +5299,6 @@ async def streaming_chat_response_handler(response, ctx):
reasoning_item.setdefault('reasoning_details', []),
reasoning_detail_items,
)
await flush_pending_delta_data()
await event_emitter(
{'type': 'chat:completion', 'data': {'output': full_output()}}
)
stream_size = 0
data = None
if value: