Commit graph

37 commits

Author SHA1 Message Date
Classic298
72fdf238a8
perf: optional orjson JSON codec behind ENABLE_ORJSON (#27583)
Swap the JSON encoder/decoder used across the backend from stdlib json to
orjson when ENABLE_ORJSON is set — HTTP request bodies, JSONResponse
bodies, upstream provider responses, SSE chunks, and socket.io/Redis
payloads.

The flag defaults to off, in which case the app uses stdlib json and
engineio's codec verbatim, so default behaviour is unchanged.

- json_codec exports JSONCodec (stdlib json or the orjson codec) and
  SOCKETIO_JSON (engineio's codec or the orjson codec); call sites import
  JSONCodec and stay implementation-agnostic
- apply_orjson_http_json() is a no-op when the flag is off, leaving
  starlette's Request.json / JSONResponse.render untouched
- the orjson codec falls back to the stdlib for inputs orjson rejects
  (non-str dict keys, ints beyond 64 bits, NaN literals)
- orjson is imported only when the flag is on
- FastAPI(default_response_class=...) is deliberately not used: an
  explicit default disables the Pydantic direct-to-bytes fast path for
  response_model routes
2026-07-27 03:45:37 -04:00
Timothy Jaeryang Baek
95d590b360 refac 2026-07-26 23:03:32 -04:00
Timothy Jaeryang Baek
8b206de48e refac 2026-07-26 22:59:23 -04:00
Classic298
18d004cabe
chore: drop python-jose, nothing imports it (#27444)
The migration to joserfc completed the job but left the old dependency pinned. `python-jose` now has zero imports anywhere in the backend: the only `jose` references left are `joserfc` in `utils/oauth.py`, and a repo-wide search for `from jose`, `import jose` or `python_jose` returns nothing outside the three pin files.

Removing it also removes `ecdsa` and `rsa` from the image, which were pulled in only by python-jose. `uv lock` confirms that: it drops exactly those three packages and nothing else, because google-auth 2.55 depends on cryptography and pyasn1-modules rather than rsa. That is worth having beyond the size saving, since `ecdsa` ships a documented Minerva-style timing side-channel in its P-256 signing path that upstream has declined to fix, so keeping it in the image means shipping a flagged crypto library that nothing calls.

Verified by blocking the `jose` module at import time and importing the backend anyway:

```
PASS  import open_webui.utils.auth
PASS  import open_webui.utils.oauth
PASS  import open_webui.main
jose in sys.modules: False
PASS  create_token/decode_token round trip
```

One user-visible consequence worth stating: Tools and Functions run in the same interpreter, so a third-party plugin that imports `jose` directly stops working after this. Nothing in Open WebUI itself does, and PyJWT remains a dependency, but a plugin relying on a library the application never declared for that purpose is the only thing this can break.

`uv.lock` was edited surgically rather than regenerated, to avoid the unrelated whole-file churn a newer uv version introduces. The result was diffed against real `uv lock` output and matches it exactly apart from that version's cosmetic fields.
2026-07-26 18:18:17 -04:00
Classic298
3ce734c6c6
fix: bump uvicorn to 0.51.0 to move off the legacy websocket implementation (#27553)
Uvicorn's `--ws auto` selected its `websockets_impl` protocol on 0.41.0, which is built on `websockets.legacy`. That module raises `AssertionError` in `_drain_helper` during keepalive pings and kills the websocket connection. Each crash runs the Socket.IO `disconnect` handler and drops the session from `SESSION_POOL`, so every subsequent server-to-browser call fails. The most visible symptom is the Pyodide code execution tool, which reaches the browser through `sio.call('events', ...)` and returns `{"stderr": "Client session disconnected."}` on every run.

Uvicorn 0.50.0 changed `--ws auto` to select the sans-io implementation whenever websockets is installed, and deprecated the legacy one. Bumping the pin therefore fixes this on every launch path at once, without adding a `--ws` flag to the startup scripts. Doing nothing is not stable either: websockets is unpinned apart from uvicorn's own `>=13.0` floor, and `websockets.legacy` is removed outright in websockets 17, which turns the current AssertionError into an ImportError on a fresh install.

Bumping to 0.51.0 rather than the minimum 0.50.0 also picks up the sans-io keepalive pings added in 0.44.0, so raw websocket endpoints keep the idle-timeout behaviour they have today behind a reverse proxy. Uvicorn 0.51.0 drops colorama from its `standard` extra and raises the httptools floor to 0.8.0, which the lockfile already satisfies.

Verified on the bumped pin: the backend boots, `/health` returns 200, `--ws auto` resolves to `WebSocketsSansIOProtocol`, a Socket.IO client completes a websocket handshake against the running app, and a bidirectional `sio.call` round trip succeeds. The unit test suite reports an identical 2273 passed / 7 failed on 0.41.0 and 0.51.0, with the 7 failures unrelated to uvicorn.

Fixes #27550
2026-07-26 17:55:44 -04:00
Timothy Jaeryang Baek
ff11ff5a3e refac 2026-07-24 01:11:23 -04:00
G30
4f823774ab
fix: migrate deprecated authlib.jose import to joserfc in oauth utils (#27310) 2026-07-23 21:54:59 -04:00
Classic298
32242a6788
perf: 40% LESS CPU usage: cut per-instance CPU cost of shared socket.io Redis pub/sub channel (#27282)
* perf: cut per-instance CPU cost of shared socket.io Redis pub/sub channel

Profiling a multi-instance deployment (py-spy --gil) showed ~44% of worker
CPU in the socket.io pub/sub listener. Two causes, two fixes:

- Add hiredis so redis-py parses the RESP protocol in C instead of pure
  Python (redis/_parsers/resp3.py alone accounted for ~28% of GIL samples;
  redis-py auto-selects the hiredis parser when importable).

- Subclass AsyncRedisManager to drop emits whose target room has no local
  participants before upstream _handle_emit re-encodes the full packet.
  Every instance receives every emit published on the shared channel, so
  with N instances all but the hosting one were paying full packet
  re-serialization per message just to deliver it to nobody. Broadcasts
  (room=None) are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q9CQ9qnp3sZGYQQwztsCJT

* fix: restrict pub/sub emit early-out to string rooms

Adversarial review against python-socketio 5.16.2 found one divergence
from upstream: for a degenerate empty-sequence room (emit to room=[]) on
an instance whose namespace has no local clients, the filter's
get_participants probe raises IndexError from room[0] where upstream
returns silently at the namespace guard and still publishes to Redis.
Open WebUI only ever emits to scalar string rooms or room=None, so the
case is unreachable today; guard on isinstance(room, str) anyway so any
non-string room shape passes through to upstream behavior unchanged.

Every open-webui emit uses a string room, so the fast path still covers
all real traffic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q9CQ9qnp3sZGYQQwztsCJT

* Update requirements.txt

* Update pyproject.toml

* Update requirements-min.txt

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-23 12:11:38 -04:00
Classic298
b295a20b9d
chore: bump Python backend dependencies, drop unused peewee (#25786)
* chore: bump Python backend dependencies, drop unused peewee

Minor/patch + reviewed major bumps across requirements.txt,
requirements-min.txt, pyproject.toml and uv.lock; playwright image bumped in
docker-compose.playwright.yaml. peewee/peewee-migrate removed (zero imports).

Security-relevant: cryptography 46->48, authlib 1.6.10->1.7.2, PyJWT 2.11->2.13,
requests 2.33.1->2.34.2, RestrictedPython 8.1->8.2, pillow 12.1.1->12.2.0.
Reviewed majors: redis 7->8, pymilvus ->2.6.14, azure-search-documents 11->12,
chardet 5->7, unstructured 0.18->0.22, pycrdt 0.12->0.13.

Testing:
- Resolution: `uv lock` resolves the full bumped set with no conflicts; uv.lock
  regenerated to match (peewee dropped, every pin including
  azure-search-documents==12.0.0 resolves).
- Per-dependency contract tests (external tests repo, unit/deps/): 105 files,
  2205 passed / 6 skipped, ruff-clean. One file per dependency pins the symbols,
  signatures and behaviour the backend actually uses, so an API removal/rename in
  a bumped version fails loudly instead of at runtime. Offline/deterministic.
- End-to-end embed->retrieve test driving transformers + sentence-transformers +
  chromadb together through Open WebUI's real RAG path (cached model, in-memory
  chroma, semantic retrieval asserted).
- Install/startup/health resolution gate added to the dep-bump workflow and the
  integration suite (uv/pip resolve + uvicorn /health + Playwright dev visibility).
- Bugs surfaced while testing each got an isolated fix branch + regression test:
  Mistral OCR aiohttp FilePayload (#25779), chroma has_collection (#25780),
  aiocache per-user model-cache key (security), otel semconv deprecation,
  pydub/audioop <3.13 note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Bump python-multipart 0.0.22 -> 0.0.27 (CVE-2026-42561, CVE-2026-40347)

0.0.22 is affected by two DoS CVEs in the multipart parser that
Starlette/FastAPI run for every multipart/form-data request, so any
authenticated user hitting an upload endpoint can trigger them:
- CVE-2026-42561: unbounded part-header count/size -> CPU exhaustion (fixed 0.0.27)
- CVE-2026-40347: large multipart preamble/epilogue DoS (fixed 0.0.26)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 02:02:18 -05:00
Timothy Jaeryang Baek
3d1e355df7 refac 2026-04-24 18:20:10 +09:00
Timothy Jaeryang Baek
9771898c58 refac 2026-04-24 17:04:47 +09:00
Timothy Jaeryang Baek
f685edd161 refac 2026-04-13 23:40:09 -05:00
Timothy Jaeryang Baek
9a8c4da67d chore: deps bump 2026-04-13 21:46:32 -05:00
Timothy Jaeryang Baek
39ea7bf63d chore: dep bump 2026-04-13 21:45:17 -05:00
Timothy Jaeryang Baek
27169124f2 refac: async db 2026-04-12 14:22:11 -05:00
Classic298
2108f420ea
chore: dep bump (#22305)
* chore: dep bump

* revert: Brotli dependency bump (1.2.0 -> 1.1.0)
2026-03-07 17:12:22 -06:00
Shirasawa
a36692b4a2
Merge pull request #22231 from ShirasawaSama/patch-10
fix: add missing beautifulsoup4 to backend requirements
2026-03-04 13:53:50 -06:00
Timothy Jaeryang Baek
850a864b02 refac 2026-02-12 16:54:32 -06:00
Classic298
55169e69c0
chore: Dep bump (#21261)
* Update pyproject.toml

* Update requirements-min.txt

* Update package versions in requirements.txt

Updated various package versions in requirements.txt to latest releases.
2026-02-09 16:15:39 -06:00
Classic298
643c661a6f
chore: Updates minor/patch versions for 21 Python backend dependencies (#21059) 2026-01-31 16:51:08 -06:00
Classic298
716f2986b9
dep bump (#20735) 2026-01-17 21:44:32 +04:00
Timothy Jaeryang Baek
3a57233dd4 chore: aiohttp 2026-01-09 18:10:27 +04:00
Classic298
f5455d48c4
Chore: dup bump for NPM and PIP (#20386)
* Update pyproject.toml

* Update aiohttp version to 3.13.3

* Update aiohttp version to 3.13.3

* Update pyproject.toml

* Update requirements.txt

* Update package.json
2026-01-05 22:34:25 +04:00
Classic298
a2f8e41fbc
chore: dep bump (#20315)
* dep bump

* update
2026-01-03 18:14:25 +04:00
Classic298
b3371033be
chore(deps): update and synchronize backend dependencies (#20225)
* chore(deps): update and synchronize backend dependencies

- Updated dependencies in requirements files and pyproject.toml to latest versions.
- Preserved pinned versions for av, pinecone, and pyarrow.
- Added missing dependencies to pyproject.toml to match requirements.txt.
- Ensured all dependency versions are synchronized across files.

* Update pyproject.toml
2025-12-30 14:05:56 +04:00
Timothy Jaeryang Baek
9405628e46 chore: dep-min 2025-12-29 00:24:49 +04:00
Classic298
60c93b4ccc
chore: dep bump (#20077)
* Update pyproject.toml

* Update requirements-min.txt

* Update requirements.txt
2025-12-21 10:52:06 -05:00
Classic298
cd170735c5
chore: dep bump (#20012)
* Update requirements-min.txt

* Update pyproject.toml

* Update requirements.txt

* Update pyproject.toml

* Update requirements.txt

* Update requirements-min.txt
2025-12-20 07:30:30 -05:00
Classic298
1c62be4406
chore: dep bump (#19937)
* Update requirements.txt

* Update requirements-min.txt

* Update pyproject.toml
2025-12-13 14:42:08 -05:00
Classic298
44e41806f2
chore: dep bump across many dependencies (#19850)
* Update pyproject.toml (#101)

* Update pyproject.toml

* Update requirements.txt

* Update requirements-min.txt

* Upgrade Playwright version to 1.57.0

* Update langchain-community version to 0.3.29

* Update requirements.txt

* Update requirements-min.txt
2025-12-09 15:28:21 -05:00
Classic298
cdd75ade50
BREAKING/CAUTION: chore: chromadb dep bump - needs testing (#19780)
* Update requirements.txt

* Update requirements-min.txt

* Update pyproject.toml
2025-12-06 08:03:04 -05:00
Classic298
4f50571b53
Chore: dep bump (#19667)
* Update pyproject.toml

* Update requirements-min.txt

* Update requirements.txt

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
2025-12-02 02:34:57 -05:00
Classic298
4df5b7eb2e
fix: update dependency to prevent rediss:// failure (#19488)
* Update pyproject.toml

* Update requirements.txt

* Update requirements-min.txt
2025-11-25 16:28:58 -05:00
Timothy Jaeryang Baek
363ef194d8 chore: bump python-socketio==5.14.0 2025-11-25 05:49:30 -05:00
Classic298
6a095099d5
chore: add chardet (#19458)
* Update pyproject.toml

* Update requirements-min.txt

* Update requirements.txt

* Update requirements-min.txt

* Update requirements.txt

* Update pyproject.toml
2025-11-25 04:52:25 -05:00
Timothy Jaeryang Baek
76acdabdc3 chore: mcp bump 2025-11-19 02:18:59 -05:00
Timothy Jaeryang Baek
a4b2dc22c4 wip: requirements-min 2025-11-13 19:24:32 -05:00