scriptoriumadmin/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-11-14 09:21:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

perf: strip whitespace in the tool-argument completion guard

Browse source 
The character guard only looked at the first and last byte of the buffer, so a
chunk that ended in a space still triggered a full parse and a tool argument
with leading whitespace fell back to parsing on every chunk. Stripping first
collapses both cases to a single parse at the end of the stream.

Measured on a streamed tool call, parses and wall time for the whole stream,
orjson on the left of the slash and stdlib json on the right:

| argument shape | before | after |
|---|---|---|
| 20 KB string, char-by-char deltas | 3678 parses, 56 / 28 ms | 1 parse, 3.1 / 3.1 ms |
| 200 KB, 20-char deltas | 1473 parses, 176 / 63 ms | 1 parse, 3.1 / 2.8 ms |
| 8 KB prose, leading whitespace | 715 parses, 5.5 / 2.5 ms | 1 parse, 0.18 ms |
| 8 KB of spaces inside a value | 713 parses, 6.0 / 2.9 ms | 1 parse, 0.83 / 0.72 ms |

The strip costs about 20 ns per delta on arguments that have no whitespace at
either end, which is where the old form was already optimal: a 20 KB compact
argument goes from 191 to 216 us over 1786 deltas. Soundness is unchanged, the
guard can still only skip a parse that would have failed: 2660892 buffers
(exhaustive to length 6 over a JSON-lexical alphabet, every prefix of 26 named
cases with a trailing byte appended, and every codepoint below U+3000 after a
complete document) with zero cases where a parse would have succeeded.
This commit is contained in:
Classic298 2026-08-31 03:37:41 +02:00
parent 653e2ce3b4
commit 38193ebd00
1 changed files with 1 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
backend/open_webui/utils/anthropic.py
View file

@ -878,11 +878,7 @@ async def openai_stream_to_anthropic_stream(openai_stream_generator, model: str
if (
tool['started']
and not tool['stopped']
and (
tool['arguments'][-1] == '}'
or tool['arguments'][-1].isspace()
or tool['arguments'][0] != '{'
)
and (tool['arguments'].rstrip()[-1:] == '}' or tool['arguments'].lstrip()[:1] != '{')
):
try:
JSONCodec.loads(tool['arguments'])