mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-15 23:32:40 +00:00
1d6735ff0b
orjson emits U+2028, U+2029 and U+0085 raw, where stdlib `json.dumps` escapes them under its default `ensure_ascii=True`. Python treats all three as line boundaries, so with `ENABLE_ORJSON` set, one of them inside model output splits a `data: {...}` SSE frame in half. Both halves then fail to parse and the delta is dropped with no error.
`utils/middleware.py` reassembles frames with `splitlines()`, so an affected response silently loses content on the direct API path. External clients are exposed as well: httpx's `LineDecoder` reimplements the same line-boundary semantics, so any SDK reading the OpenAI-compatible stream through `aiter_lines` breaks on a raw separator.
The three characters are escaped on the way out of `ORJSONCodec.dumps`. That restores parity with stdlib and fixes every reader at once, rather than patching one consumer and leaving external clients broken. They are the complete set: of the ten code points `splitlines()` treats as boundaries, the other seven are below U+0020, where JSON already forces an escape.
The membership guard is load bearing. Calling `translate` unconditionally costs roughly 1.5 us on a typical SSE chunk against 0.115 us for the serialization it wraps, so it would spend more than orjson saves. The three scans cost about 0.04 us.
Payloads containing none of the three are returned unchanged, byte for byte. With `ENABLE_ORJSON` unset, which is the default, none of this code runs.
U+2028 and U+2029 are common in text extracted from PDFs and word processor documents, so the realistic trigger is a model quoting an uploaded file back to the user.
|
||
|---|---|---|
| .. | ||
| access_control | ||
| images | ||
| mcp | ||
| telemetry | ||
| actions.py | ||
| anthropic.py | ||
| asgi_middleware.py | ||
| audit.py | ||
| auth.py | ||
| automations.py | ||
| calendar.py | ||
| channels.py | ||
| chat.py | ||
| chat_fork.py | ||
| chat_id.py | ||
| chat_variables.py | ||
| code_interpreter.py | ||
| context_compaction.py | ||
| embeddings.py | ||
| files.py | ||
| filter.py | ||
| groups.py | ||
| headers.py | ||
| json_codec.py | ||
| json_response.py | ||
| logger.py | ||
| memory.py | ||
| middleware.py | ||
| misc.py | ||
| model_ids.py | ||
| models.py | ||
| notifications.py | ||
| oauth.py | ||
| payload.py | ||
| pdf_generator.py | ||
| plugin.py | ||
| rate_limit.py | ||
| redis.py | ||
| response.py | ||
| sanitize.py | ||
| security_headers.py | ||
| session_pool.py | ||
| subagents.py | ||
| task.py | ||
| terminals.py | ||
| timers.py | ||
| tools.py | ||
| validate.py | ||
| valves.py | ||
| webhook.py | ||