scriptoriumadmin/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-08-28 05:27:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

perf: use the orjson codec for outbound Ollama request bodies (#27811)

Browse source 
`routers/ollama.py` serializes the outbound body with stdlib `json` on six inference paths: `/api/chat`, the OpenAI-compatible completions and chat completions proxies, embeddings, the Anthropic messages proxy, and responses. All six carry a full conversation or an embedding batch.

They now go through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. Every one is passed to `send_request`, which hands it to aiohttp as `data=`; aiohttp encodes `str` as UTF-8 and derives `Content-Length` from the encoded bytes. None is hashed, cached, length-measured or persisted.

Admin model management keeps stdlib: `/api/unload`, `/api/pull`, `/api/delete` and `/api/show` serialize fixed one- or two-key dicts, as do the blob download and upload progress events and the error frames. Codec dispatch on those costs about what it saves.

With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
This commit is contained in:
Classic298 2026-07-31 23:26:38 +02:00 • committed by GitHub
parent 006a63e641
commit ace84b4ae9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 6 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
backend/open_webui/routers/ollama.py
View file

@ -1149,7 +1149,7 @@ async def generate_chat_completion(
return await send_request(
f'{url}/api/chat',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, api_configs),
user=user,
stream=form_data.stream,
@ -1244,7 +1244,7 @@ async def generate_openai_completion(
return await send_request(
f'{url}/v1/completions',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, api_configs),
user=user,
stream=payload.get('stream', False),
@ -1295,7 +1295,7 @@ async def generate_openai_embeddings(
return await send_request(
f'{url}/v1/embeddings',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, (await Config.get('ollama.api_configs', {}))),
user=user,
metadata=metadata,
@ -1352,7 +1352,7 @@ async def generate_openai_chat_completion(
return await send_request(
f'{url}/v1/chat/completions',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, api_configs),
user=user,
stream=payload.get('stream', False),
@ -1404,7 +1404,7 @@ async def generate_anthropic_messages(
return await send_request(
f'{url}/v1/messages',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, api_configs),
user=user,
stream=payload.get('stream', False),
@ -1462,7 +1462,7 @@ async def generate_responses(
return await send_request(
f'{url}/v1/responses',
payload=json.dumps(payload),
payload=JSONCodec.dumps(payload),
key=get_api_key(url_idx, url, api_configs),
user=user,
stream=payload.get('stream', False),