mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
b2a9be27c5
Every streamed token appended to the response text through a dict slot, which allocates a new string and copies everything received so far. A long answer pays that copy thousands of times on the event loop, so every other request on the worker waits behind it. The appends now clear the slot before growing the string, which leaves a single reference and lets CPython resize it in place. This covers the six accumulation points in the Chat Completions streaming path. The Responses API transport copies its items before appending, so it holds two references and is unchanged. Measured on CPython 3.12 with 10-character deltas: | accumulated text | before | after | |---|---|---| | 100 KB | 5.97 ms | 0.98 ms | | 250 KB | 29.53 ms | 2.83 ms | | 1 MB | 659 ms | 15.5 ms | Results match the replaced expressions for every input where the key exists and holds a string, key order and exception type included. Buffering into a list and joining at the end was the alternative, but each delta is read back immediately by the stream save, and joining per read measured 666 ms at 250 KB. |
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| events.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||