mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
e30ed01b05
stream_wrapper without a content handler iterates aiohttp's response.content, which reads line by line: every line costs a buffer scan, a slice, a bytes concat, a generator resume and its own ASGI response message. A typical SSE event is two lines (the data line and the blank separator), so every upstream token event became two yields and two transport writes even on routes where the body is never inspected. stream_wrapper now takes passthrough=True, which iterates response.content.iter_any(): the exact same bytes, one yield per network read, no line scanning. It is applied only to routes no internal consumer parses line-by-line: the ollama pull/push/create/generate proxies and its v1 completions, chat completions, messages and responses endpoints, plus the openai embeddings, responses and catch-all proxies. The two internally consumed chat routes keep line iteration, which the streaming middleware and the Ollama-to-OpenAI converter require; the ollama send_request signature documents that constraint. Benchmark (local aiohttp SSE server, 500 events, consumed through stream_wrapper): | metric | before (readline) | after (iter_any) | | --- | --- | --- | | stream consumption time | 1.46 ms | 0.62 ms | | generator yields + response writes per stream | 1000 | 1 | The single yield is a loopback artifact (the whole body arrives in one buffered read); over a real network it becomes one yield per TCP read instead of two per SSE event. Functionally verified: line mode and passthrough mode produce byte-identical output for the same stream, and passthrough always yields fewer, larger chunks. |
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| events.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||