scriptoriumadmin/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-10-09 03:18:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

fix: bundle NLTK punkt_tab in Docker image for airgapped environments (#21165)

Browse source 
Pre-download NLTK punkt_tab during Docker build instead of at runtime.
This fixes document extraction failures in offline/airgapped environments
where the container cannot download the tokenizer data after restarts.
Fixes #21150
This commit is contained in:
Classic298 2026-02-05 21:11:00 +01:00 • committed by GitHub
parent d3f2cf7474
commit 9ed8f50d40
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Dockerfile
View file

@ -143,6 +143,7 @@ RUN pip3 install --no-cache-dir uv && \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')" && \
python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \
python -c "import nltk; nltk.download('punkt_tab')"; \
else \
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --no-cache-dir && \
uv pip install --system -r requirements.txt --no-cache-dir && \
@ -151,6 +152,7 @@ RUN pip3 install --no-cache-dir uv && \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')" && \
python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \
python -c "import nltk; nltk.download('punkt_tab')"; \
fi; \
fi; \
mkdir -p /app/backend/data && chown -R $UID:$GID /app/backend/data/ && \