Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args: free_disk:false name:main suffix:]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true
USE_CUDA_VER=cu126
free_disk:true name:cuda126 suffix:-cuda126]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args: free_disk:false name:main suffix:]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true
USE_CUDA_VER=cu126
free_disk:true name:cuda126 suffix:-cuda126]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Has been cancelled
Raising GLOBAL_LOG_LEVEL to WARNING buys quieter output but not less work: 241 INFO call sites interpolate their payload into an f-string before the logging call gets to drop it. The heaviest is get_doc, which logs every chunk id and metadata dict in a collection, so on the full-context retrieval path that is the entire knowledge base, once per chat request.
That one line at WARNING, CPython 3.12:
| knowledge base | payload | before | after |
| -------------- | ------- | -------- | ------- |
| top-k of 3 | 1.2 kB | 3.8 us | 0.07 us |
| 500 chunks | 201 kB | 583.6 us | 0.08 us |
| 5000 chunks | 2.0 MB | 5.8 ms | 0.15 us |
The lazy form log.info('query_doc:result %s %s', result.ids, result.metadatas) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. Output at INFO is byte-identical. Two sites that already built their message eagerly, one str concat and one % operator, move to the same lazy form.
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
`routers/ollama.py` serializes the outbound body with stdlib `json` on six inference paths: `/api/chat`, the OpenAI-compatible completions and chat completions proxies, embeddings, the Anthropic messages proxy, and responses. All six carry a full conversation or an embedding batch.
They now go through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. Every one is passed to `send_request`, which hands it to aiohttp as `data=`; aiohttp encodes `str` as UTF-8 and derives `Content-Length` from the encoded bytes. None is hashed, cached, length-measured or persisted.
Admin model management keeps stdlib: `/api/unload`, `/api/pull`, `/api/delete` and `/api/show` serialize fixed one- or two-key dicts, as do the blob download and upload progress events and the error frames. Codec dispatch on those costs about what it saves.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
Swap the JSON encoder/decoder used across the backend from stdlib json to
orjson when ENABLE_ORJSON is set — HTTP request bodies, JSONResponse
bodies, upstream provider responses, SSE chunks, and socket.io/Redis
payloads.
The flag defaults to off, in which case the app uses stdlib json and
engineio's codec verbatim, so default behaviour is unchanged.
- json_codec exports JSONCodec (stdlib json or the orjson codec) and
SOCKETIO_JSON (engineio's codec or the orjson codec); call sites import
JSONCodec and stay implementation-agnostic
- apply_orjson_http_json() is a no-op when the flag is off, leaving
starlette's Request.json / JSONResponse.render untouched
- the orjson codec falls back to the stdlib for inputs orjson rejects
(non-str dict keys, ints beyond 64 bits, NaN literals)
- orjson is imported only when the flag is on
- FastAPI(default_response_class=...) is deliberately not used: an
explicit default disables the Pydantic direct-to-bytes fast path for
response_model routes
stream_wrapper without a content handler iterates aiohttp's response.content, which reads line by line: every line costs a buffer scan, a slice, a bytes concat, a generator resume and its own ASGI response message. A typical SSE event is two lines (the data line and the blank separator), so every upstream token event became two yields and two transport writes even on routes where the body is never inspected.
stream_wrapper now takes passthrough=True, which iterates response.content.iter_any(): the exact same bytes, one yield per network read, no line scanning. It is applied only to routes no internal consumer parses line-by-line: the ollama pull/push/create/generate proxies and its v1 completions, chat completions, messages and responses endpoints, plus the openai embeddings, responses and catch-all proxies. The two internally consumed chat routes keep line iteration, which the streaming middleware and the Ollama-to-OpenAI converter require; the ollama send_request signature documents that constraint.
Benchmark (local aiohttp SSE server, 500 events, consumed through stream_wrapper):
| metric | before (readline) | after (iter_any) |
| --- | --- | --- |
| stream consumption time | 1.46 ms | 0.62 ms |
| generator yields + response writes per stream | 1000 | 1 |
The single yield is a loopback artifact (the whole body arrives in one buffered read); over a real network it becomes one yield per TCP read instead of two per SSE event.
Functionally verified: line mode and passthrough mode produce byte-identical output for the same stream, and passthrough always yields fewer, larger chunks.
get_ollama_versions was the only Ollama route besides the static health check without an authentication dependency, so an anonymous caller could read the configured backend's version string and, by walking url_idx until the lookup raised, count the configured backends.
Nothing depends on the route being public. The frontend wrapper takes a token and sends it on every call, and its three call sites (admin model management, the model selector and the About panel) all pass an authenticated token, so the client already treats this as an authenticated route. Add the same get_verified_user dependency the sibling routes carry.
Co-authored-by: Grg0rry <Grg0rry@users.noreply.github.com>
Custom per-connection headers can now forward the user's groups to
upstream backends via two new template placeholders:
- {{USER_GROUPS}}: comma-separated group names
- {{USER_GROUP_IDS}}: comma-separated group ids
The group lookup is async, so get_custom_headers becomes an async
wrapper around the sync template substitution (parse_custom_headers)
and fetches groups lazily — only when a header value actually
references a groups placeholder. The external document loader path
runs in a worker thread without an event loop, so Loader.aload
prefetches the groups before offloading and passes them through to
ExternalDocumentLoader.
Claude-Session: https://claude.ai/code/session_01EbBEfTyu8fFJmC13rnQthT
Co-authored-by: Claude <noreply@anthropic.com>
The per-request Ollama handlers (chat, generate, embed, embeddings,
and the OpenAI-compat completions/chat-completions/messages/responses
endpoints) fetched 'ollama.api_configs' up to three times and
'ollama.base_urls' separately within a single request — the .get()
default-argument pattern made the second api_configs fetch
unconditional, and get_api_key() triggered a third. Up to four
sequential SELECTs per request collapse to one.
A new get_ollama_connection_config() helper fetches base_urls and
api_configs together in one batched Config.get_many where both are
needed; handlers that only need api_configs fetch it once into a
local. Admin operations (pull/push/copy/delete) and the TTL-cached
model-list path are deliberately left untouched.
Resolution semantics (str(idx) key first, url-key legacy fallback,
same defaults) are unchanged.
Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD
Co-authored-by: Claude <noreply@anthropic.com>
- Move setModels() and setToolServers() out of the loaded gate in
+layout.svelte so the page renders immediately instead of blocking
behind slow/unreachable connection timeouts
- Track failed Ollama backend URLs from /api/tags and skip them in the
subsequent /api/ps fan-out, eliminating a redundant second timeout
- Silence the autocomplete toast that fires during the transient empty
model state
Adds a {{USER_AGENT}} custom-header placeholder that relays the inbound
client's User-Agent to upstream model backends, so providers see the real
client instead of Open WebUI's internal aiohttp UA. This makes upstream
usage/cost attribution and backend telemetry possible, and is opt-in
per-connection (no global flag): admins add {{USER_AGENT}} to a connection's
custom headers in Admin > Settings > Connections.
The placeholder is sourced from the live inbound request (with a metadata
fallback for detached RAG/tool calls), so it resolves on every prompt-sending
path, not just chat completions:
- OpenAI completions, Responses API, and proxy — all route through
get_headers_and_cookies, which now passes the request into get_custom_headers.
- Anthropic Messages API (/api/v1/messages) — already covered, it delegates
to the chat completion handler.
- Ollama (/api/chat, /v1/completions, /v1/chat/completions, /v1/messages,
/v1/responses) — previously had no custom-header support at all; send_request
now applies per-connection custom headers (with templating) for every
Ollama prompt endpoint.
Custom headers are applied after the built-in user-info headers so explicit
admin-configured headers take precedence. The other existing placeholders
({{CHAT_ID}}, {{USER_ID}}, ...) now also work on the newly covered paths.
Frontend: the connection editor's Headers field is now shown for Ollama
connections too (previously gated to non-Ollama), so the placeholder can be
configured there.
Ref: open-webui/open-webui#26159
routers/openai.py and routers/ollama.py decorated get_all_models with
`@cached(key=lambda _, user: ...)`. In aiocache 0.12, `key=` is a STATIC
cache key: get_cache_key returns `self.key` verbatim (the lambda object)
without calling it, so every caller collides to ONE shared entry within
the TTL. The intended per-user namespacing never happened — one user's
permission-filtered model list could be served to another user (or an
anonymous caller) during the cache window.
The per-call hook is `key_builder=` (called as key_builder(func, *args,
**kwargs)). Switch both sites to key_builder with a (func, request,
user=None) signature so the key is built per call. user=None mirrors
ollama's optional-user signature and stays correct whether user is passed
positionally, as a kwarg, or omitted.
Verified offline: old form returns the same key object for distinct users
(collision); new form yields distinct openai_all_models_<id> /
ollama_all_models_<id> keys, and the unauthenticated base key when no user.
These two were the only @cached(key=lambda ...) sites in the backend.
The Ollama model upload and download handlers perform three stages
of sync file I/O on multi-GB model files inside async handlers:
1. Persist upload — file.file.read() + write() in a loop
2. SHA-256 hash — calculate_sha256() reads the file sequentially
3. Read for blob push — open().read() loads entire model into memory
All three stages block the event loop for the duration of the I/O.
For a 4GB model, each stage freezes the event loop for 10+ seconds
while every other user's request stalls.
Wrap each blocking stage in asyncio.to_thread() in both handlers:
- upload_model(): persist upload, calculate_sha256, blob read
- download_file_stream(): calculate_sha256, blob read
Benchmark (full pipeline: write + SHA-256 + read back, 3 trials):
- 200MB: max jitter 145ms → 1ms (145x improvement)
- 500MB: max jitter 1,026ms → 1ms (1,026x improvement)
File I/O is pure I/O — no GIL contention. asyncio.to_thread()
completely eliminates event loop blocking.
bypass_system_prompt is an internal flag used by utils/middleware.py and utils/chat.py to skip applying the model system prompt on recursive base-model calls, but it was still declared as a positional argument on the openai/ollama chat-completion route handlers, so FastAPI bound it from the query string. Move it to request.state so external clients cannot set it, matching how bypass_filter is handled.
Drop the argument from both route signatures and read getattr(request.state, 'bypass_system_prompt', False); utils/chat.py sets request.state.bypass_system_prompt alongside bypass_filter and drops the kwarg from the two route-handler calls (the recursive self-calls keep it). Mirrors c0385f60b.
Co-authored-by: anishgirianish <161533316+anishgirianish@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args: free_disk:false name:main suffix:]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true
USE_CUDA_VER=cu126
free_disk:true name:cuda126 suffix:-cuda126]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args: free_disk:false name:main suffix:]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true
USE_CUDA_VER=cu126
free_disk:true name:cuda126 suffix:-cuda126]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Has been cancelled
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Has been cancelled
These four endpoints checked model existence but never verified the user has read access via AccessGrants, allowing any authenticated user to use restricted models.
Uses the canonical check_model_access helper from utils.access_control.
The /responses proxy endpoint only required authentication via
get_verified_user but did not check per-model access grants. This
allowed any authenticated user to access any model through this
endpoint, bypassing the access control system.
Extract a shared check_model_access helper into utils/access_control
and replace all inline access control blocks across openai.py and
ollama.py (7 locations) with calls to this helper. This eliminates
code duplication and prevents future policy drift between endpoints.
CWE-862: Missing Authorization
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:H (6.5 Medium)
Ollama recently added Responses API support via its OpenAI-compatible
endpoint (/v1/responses). This adds a proxy endpoint to the Ollama
router that forwards requests to Ollama's /v1/responses, applying
the same model resolution, access control, and prefix_id handling
used by the existing /v1/chat/completions and /v1/messages proxies.
Uses a typed ResponsesForm Pydantic model with required model field
and extra='allow' for forward compatibility, consistent with other
endpoint schemas in the file.
This allows API consumers (Codex, Claude Code, etc.) to use the
Responses API directly with Ollama-hosted models without requiring
a separate OpenAI-compatible connection.
