mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
68a74da70b
* fix(retrieval): serialize local embedding and reranking on MPS On Apple Silicon the server process is killed outright (SIGSEGV or SIGTRAP, no traceback) partway through answering any question that retrieves from a knowledge base with hybrid search and a local reranking model. The client sees a dropped connection and the answer is lost. Hybrid search fans its queries out concurrently and every task calls the same shared local model on a worker thread. Torch's Metal shader cache is a process-wide singleton whose lookup tables have no lock, so two of those threads racing inside it corrupt the cache and take the process down with it. Guard the local SentenceTransformer and CrossEncoder calls with a shared lock that is only a real lock when the selected device is MPS. CPU and CUDA installs keep the concurrency they have today, and external reranking endpoints are untouched. Reranking several queries on a Mac now runs one at a time, which is the cost of the process staying alive. Verified by driving the real hybrid-search fan-out with 16 concurrent queries: peak simultaneous entries into the local model drops from 16 to 1 on MPS, stays at 16 on CPU, and the returned documents, scores and ordering are byte-identical in every case. Fixes #29722 * fix(evaluations): serialize the leaderboard embedder against retrieval on MPS The leaderboard's tag-similarity search builds its own SentenceTransformer, and on Apple Silicon sentence-transformers places it on the MPS device. It runs on a worker thread, so an admin running a leaderboard search while anyone queries a knowledge base puts two threads into torch's Metal backend at the same time, which kills the server process outright with no traceback. Move the lock added for the retrieval path into env.py, beside the device selection that decides whether MPS is used at all, and take it around the leaderboard's embedding calls as well. Sharing one lock between the two modules is the whole point, since two separate locks would still let a leaderboard search collide with a retrieval query. Only inference is guarded, matching the retrieval path. Model construction stays as it is here and in the retrieval routers. Verified by driving the leaderboard similarity path and retrieval reranking from six threads against one instrumented model: peak simultaneous entries drops from six to one on MPS, and the similarity scores are unchanged. Related to #29722. |
||
|---|---|---|
| .. | ||
| data | ||
| open_webui | ||
| .dockerignore | ||
| .gitignore | ||
| dev.sh | ||
| requirements-min.txt | ||
| requirements.txt | ||
| start.sh | ||
| start_windows.bat | ||