diff --git a/.dockerignore b/.dockerignore index 2b4f7b5fcf..55ab35b985 100644 --- a/.dockerignore +++ b/.dockerignore @@ -18,3 +18,6 @@ uploads **/*.db _test backend/data/* + +.venv +.git diff --git a/.env.example b/.env.example index 22b128ce8f..313024511a 100644 --- a/.env.example +++ b/.env.example @@ -22,6 +22,9 @@ ENABLE_RAG_CSV_SUMMARY=false # Set to true to preserve backing file records, storage blobs, and per-file vectors when files are removed from knowledge bases. ENABLE_KNOWLEDGE_FILE_RETENTION=false +# Comma-separated chunk metadata keys to expose to the model alongside retrieved content. +RAG_SOURCE_METADATA_KEYS='' + # Set to false to disable workspace Tools and Functions. ENABLE_PLUGINS=true diff --git a/.github/ISSUE_TEMPLATE/bug_report.yaml b/.github/ISSUE_TEMPLATE/bug_report.yaml index 34663b5eda..510bdce1ba 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yaml +++ b/.github/ISSUE_TEMPLATE/bug_report.yaml @@ -12,6 +12,8 @@ body: Before submitting, search open and closed [Issues](https://github.com/open-webui/open-webui/issues) and [Discussions](https://github.com/open-webui/open-webui/discussions). The issue may already be reported or fixed on `dev`. + **Test on the latest release AND on `dev`, right before you submit this report, not last week.** A huge share of reports are for bugs already fixed on `dev`, sometimes weeks earlier, because the reporter only tested an old version and never rechecked. Reports that don't reproduce on latest and on current `dev` at submission time will be closed without further discussion, no exceptions. + Please do not open a code pull request for this report unless a maintainer asks for one, or the change is only i18n/localization. If you want to share code as reference, include it here as a local diff or patch. Actionable reproduction details are the most useful next step. Security vulnerabilities must not be reported publicly. Use the [GitHub security page](https://github.com/open-webui/open-webui/security) instead. @@ -23,12 +25,10 @@ body: options: - label: I searched open and closed issues and discussions for an existing report. required: true - - label: I checked whether this is already fixed on the `dev` branch or latest source. + - label: I reproduced this bug on the latest release AND on the current `dev` branch, right before submitting this report. I did not just check an old version or rely on a check from days ago. required: true - label: I understand that maintainers want a well-written issue before any code pull request. required: true - - label: I am using the latest available version of Open WebUI for my install method. - required: true - label: This is not a security vulnerability. required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index a568080607..6eaf02f215 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,31 +1,30 @@ # Pull Request -Thanks for wanting to improve Open WebUI. The most useful contribution is usually a clear, well-written Issue, not an unsolicited code pull request. +**Do not open a code pull request unless a maintainer has explicitly requested it or the change is limited to i18n/localization.** -Open a code pull request only when a maintainer asks for one, or when the change is only i18n/localization. For real, reproducible bugs, start with a well-described [Issue](https://github.com/open-webui/open-webui/issues). For feature requests, UI/UX changes, behavior changes, architecture changes, suspected fixes, or unconfirmed approaches, start with an active [Discussion](https://github.com/open-webui/open-webui/discussions). +The most useful way to help is to give us a clear understanding of the problem: report reproducible bugs in [Issues](https://github.com/open-webui/open-webui/issues) and share proposals in [Discussions](https://github.com/open-webui/open-webui/discussions). We use that context to evaluate solutions and refine the implementation internally, accounting for the broader codebase and ongoing work. External implementations usually require substantial reworking to fit the project's standards, and coordinating those revisions usually takes more effort than developing the solution internally. Please follow this process before investing time in a pull request. PRs opened outside these guidelines are generally closed without review. -Before continuing, make sure the linked Issue or Discussion explains the user-facing problem, the expected outcome, the affected workflow, and any examples, logs, screenshots, constraints, or reproduction details needed for maintainers to evaluate it. +## Maintainer Request -If you have implementation notes, include them as reference in the Issue or Discussion. If you want to share code as reference, include it there as a local diff, patch, or branch note. Do not open a pull request for reference code. - -Unsolicited PRs may be closed without review, especially when they introduce product, architecture, compatibility, dependency, or maintenance decisions that have not been discussed. +Link the maintainer's request for this PR, or state that the change is limited to i18n/localization. ## Checklist +- [ ] I have read and I understand the [contribution policy](https://docs.openwebui.com/contributing/#submit-code). - [ ] This PR targets the `dev` branch. - [ ] This PR links to a well-described, confirmed Issue or active Discussion: `Closes #___` / `Relates to #___`. - [ ] A maintainer explicitly asked me to open this PR, or this PR only updates i18n/localization. - [ ] The change is one logical unit with no unrelated commits. - [ ] I matched nearby code patterns and avoided unnecessary new settings, abstractions, or dependencies. - [ ] I manually tested the changed workflow and any nearby behavior that could be affected. +- [ ] I have not added or rewritten automated tests, fixtures, snapshots, or testing infrastructure unless a maintainer explicitly requested them. - [ ] I updated relevant docs, including the [Open WebUI Docs Repository](https://github.com/open-webui/docs), if needed. - [ ] I added screenshots for UI changes, and a recording when motion or interaction matters. - [ ] I reviewed any AI-generated code before submitting it. @@ -50,9 +49,11 @@ Use one of the following prefixes: Describe the change, the problem it solves, and the impact on users. -## Testing +## Verification -List the exact manual checks you ran. Include commands, setup details, screenshots, or recordings where helpful. +Describe how you reproduced the problem and manually checked the behavior before and after the change. Include exact steps, setup details, and relevant logs, screenshots, or recordings. Report results from relevant existing checks and anything you could not verify. + +Do not add or rewrite automated tests unless a maintainer explicitly requests them. Tests that repeat an implementation's assumptions can pass while preserving the same mistake; maintainers determine the regression coverage needed. Do not remove, disable, or weaken existing tests to make the change pass. ## Changelog Entry diff --git a/.github/workflows/regression.yaml b/.github/workflows/regression.yaml new file mode 100644 index 0000000000..986278ffd1 --- /dev/null +++ b/.github/workflows/regression.yaml @@ -0,0 +1,45 @@ +# ───────────────────────────────────────────────────────────────────────────── +# Tests — run the open-webui/tests unit suite against a release candidate +# Release pull requests go from dev into main and are titled with the version +# ───────────────────────────────────────────────────────────────────────────── +name: Tests + +on: + pull_request: + branches: [main, dev] + types: [opened, synchronize, reopened, edited] + +# An edit must not cancel a running suite: the replacement run would skip it and still report green. +concurrency: + group: regression-${{ github.ref }} + cancel-in-progress: ${{ github.event.action != 'edited' }} + +jobs: + # Into main only the release branch counts; into dev any version title does, so the + # suite can be exercised outside a release. Expressions have no regex, hence the literal prefixes. + regression: + name: Suite + if: >- + (github.event.pull_request.base.ref == 'dev' || + github.event.pull_request.head.ref == 'dev') && + (github.event.action != 'edited' || github.event.changes.title != null) && + (startsWith(github.event.pull_request.title, '0.') || + startsWith(github.event.pull_request.title, '1.')) + permissions: + contents: read + uses: open-webui/tests/.github/workflows/regression.yml@main + with: + open-webui-ref: ${{ github.event.pull_request.head.sha }} + + # Single check to require in branch protection. + result: + name: Result + needs: [regression] + if: always() + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: {} + steps: + - name: Fail unless the suite passed or was not required + if: needs.regression.result != 'success' && needs.regression.result != 'skipped' + run: exit 1 diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 66ac54f76c..6625e2d9e2 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -34,10 +34,21 @@ Examples of unacceptable behavior include: - **Spamming and promotional exploitation.** Sharing irrelevant product promotions or self-promotion in the community is not allowed unless it directly contributes value to the discussion. - Posting low-effort, hard to read, essay-length AI generated comments or other forms of low-quality, hard to parse content that puts the burden of understanding on the reader. +### How We Develop the Project + +Development is led by the maintainers, and code pull requests are reserved for work we explicitly request or exceptional contributions we choose to consider at our discretion. We use actionable reports and concrete use cases to understand problems, then evaluate, revise, and implement the appropriate approach internally. We assess each change against the project's architecture, existing behavior, quality standards, and future direction before settling on an implementation. Resolving a reported problem requires that broader context, and a working external patch usually requires substantial rewriting to meet the project's standards. Reviewing the patch, explaining the required changes, and coordinating successive revisions usually takes more effort than developing the solution internally. Fragmented commit histories, branches that have not been rebased, unresolved conflicts, and lengthy or unverified AI-generated comments add cleanup and discussion that delay the underlying work. Maintainers remain responsible for testing, documenting, supporting, and maintaining every accepted change, so we choose the approach based on the whole product and its ongoing maintenance. Clear reports, reproduction details, and relevant context give us what we need to make those decisions and develop the solution. A polished implementation, clean commit history, or completed checklist does not establish an exception to this process, and opening an issue or discussion is not an invitation to submit a PR. Wait for an explicit maintainer request before investing in a PR; unsolicited submissions are generally closed without review, and requested PRs remain subject to maintainer judgment. + ### Feedback and Community Engagement -- **Constructive feedback is encouraged, but hostile or entitled behavior will result in immediate action.** If you disagree with elements of the project, we encourage you to offer meaningful improvements or fork the project if necessary. Healthy discussions and technical disagreements are welcome only when handled with professionalism. -- **Respect contributors' time and efforts.** No one is entitled to personalized or on-demand assistance. This is a community built on collaboration and shared effort; demanding or demeaning behavior undermines that trust and will not be allowed. +Participation should help maintainers understand a concrete problem while respecting the project's priorities and available capacity. Please follow the [issue templates](.github/ISSUE_TEMPLATE) and [pull request policy](.github/pull_request_template.md) before submitting anything. + +- **Make reports actionable.** Search existing issues and discussions, check the latest version and whether the problem is already addressed on `dev`, and use the appropriate template. Bug reports should describe a reproducible problem, the affected workflow, expected and actual behavior, and relevant evidence. Feature requests should explain the user-facing need; broader product, UX, architecture, or maintenance questions belong in Discussions. Report security concerns privately through the [security reporting process](https://github.com/open-webui/open-webui/security). +- **Share the problem before investing in code.** Start with an actionable issue or discussion and leave implementation planning to the maintainers. An issue or discussion alone is not an invitation to submit a PR. Please wait for an explicit request before opening one; any exception is at the maintainers' discretion. Implementation notes, local diffs, or patches may be shared as reference in the relevant issue or discussion. +- **Respect maintainers' discretion.** Submitting an issue, proposal, or pull request does not create an obligation to respond, review, implement, or merge it. Maintainers set the project's direction and defer or close submissions based on scope, quality, maintenance cost, or available capacity. Unsolicited pull requests are generally closed without review. +- **Keep discussion focused and concise.** Provide new information when it helps evaluate the problem. Repeated bumps, duplicate submissions, unsolicited direct messages seeking attention, or pressure for timelines place an unnecessary burden on contributors. +- **Respect decisions and boundaries.** Technical disagreement is welcome when expressed professionally. Reopening a declined request or continuing to press for a different outcome without new, relevant information is not constructive. You are free to explore a different direction in your own fork. + +Participants are expected to respect maintainers' decisions and the contribution process. Harassment, hostility, or repeated disregard for these boundaries will result in enforcement under this Code of Conduct. ### Zero Tolerance: No Warnings, Immediate Action diff --git a/Dockerfile b/Dockerfile index d70244583b..08d7a71eec 100644 --- a/Dockerfile +++ b/Dockerfile @@ -26,6 +26,9 @@ ARG GID=0 ######## WebUI frontend ######## FROM --platform=$BUILDPLATFORM node:22-alpine3.20 AS build ARG BUILD_HASH +ARG USE_SLIM +ARG UID +ARG GID # Set Node.js options (heap limit Allocation failed - JavaScript heap out of memory) # ENV NODE_OPTIONS="--max-old-space-size=4096" @@ -40,7 +43,14 @@ RUN npm ci --force COPY . . ENV APP_BUILD_HASH=${BUILD_HASH} -RUN npm run build +RUN npm run build && \ + if [ "$USE_SLIM" = "true" ]; then find build -type f -name '*.map' -delete; fi + +# Prepare backend ownership before the final copy so static assets occupy one layer. +# Group 0 write access lets arbitrary OpenShift UIDs update these assets at startup. +RUN chown -R $UID:$GID /app/backend && \ + chgrp -R 0 /app/backend/open_webui/static && \ + chmod -R g=u /app/backend/open_webui/static ######## WebUI backend ######## FROM python:3.11-slim-bookworm AS base @@ -123,24 +133,33 @@ RUN echo -n 00000000-0000-0000-0000-000000000000 > $HOME/.cache/chroma/telemetry # Make sure the user has access to the app and root directory RUN chown -R $UID:$GID /app $HOME -# Install common system dependencies +# Slim cannot bundle a local model server or GPU runtime. +RUN if [ "$USE_SLIM" = "true" ] && { [ "$USE_CUDA" = "true" ] || [ "$USE_OLLAMA" = "true" ]; }; then \ + echo "USE_SLIM cannot be combined with USE_CUDA or USE_OLLAMA" >&2; exit 1; fi + +# Keep the slim runtime free of local document/audio processing tools. +# Git-based tool requirements require the standard image. RUN apt-get update && \ apt-get install -y --no-install-recommends \ - git build-essential pandoc gcc curl jq ca-certificates \ - libmariadb-dev \ - python3-dev \ - ffmpeg libsm6 libxext6 zstd \ - && rm -rf /var/lib/apt/lists/* + curl jq ca-certificates \ + && if [ "$USE_SLIM" != "true" ]; then \ + apt-get install -y --no-install-recommends \ + git build-essential pandoc gcc libmariadb-dev ffmpeg libsm6 libxext6; \ + fi && if [ "$USE_OLLAMA" = "true" ]; then \ + apt-get install -y --no-install-recommends zstd; \ + fi && rm -rf /var/lib/apt/lists/* # install python dependencies -COPY --chown=$UID:$GID ./backend/requirements.txt ./requirements.txt +COPY --chown=$UID:$GID ./backend/requirements*.txt ./ # Set UV_LINK_MODE to copy to prevent 0-byte file corruption in QEMU arm64 cross-builds ENV UV_LINK_MODE=copy -RUN set -e; \ - pip3 install --no-cache-dir uv; \ - if [ "$USE_CUDA" = "true" ]; then \ +RUN --mount=from=ghcr.io/astral-sh/uv:0.12.10,source=/uv,target=/bin/uv \ + set -e; \ + if [ "$USE_SLIM" = "true" ]; then \ + uv pip install --system -r requirements-slim.txt --no-cache-dir; \ + elif [ "$USE_CUDA" = "true" ]; then \ # If you use CUDA the whisper and embedding model will be downloaded on first use # fix: pin torch<=2.9.1 - torch 2.10.0 aarch64 wheels cause SIGILL on ARM devices (RPi 4 Cortex-A72) #21349 pip3 install 'torch<=2.9.1' torchvision torchaudio --index-url https://download.pytorch.org/whl/$USE_CUDA_DOCKER_VER --no-cache-dir; \ @@ -149,7 +168,6 @@ RUN set -e; \ python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')"; \ python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \ python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \ - python -c "import nltk; nltk.download('punkt_tab', download_dir='/usr/local/share/nltk_data')"; \ else \ pip3 install 'torch<=2.9.1' torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --no-cache-dir; \ uv pip install --system -r requirements.txt --no-cache-dir; \ @@ -158,7 +176,6 @@ RUN set -e; \ python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')"; \ python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \ python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \ - python -c "import nltk; nltk.download('punkt_tab', download_dir='/usr/local/share/nltk_data')"; \ fi; \ fi; \ mkdir -p /app/backend/data; chown -R $UID:$GID /app/backend/data/; \ @@ -187,19 +204,8 @@ COPY --chown=$UID:$GID --from=build /app/build /app/build COPY --chown=$UID:$GID --from=build /app/CHANGELOG.md /app/CHANGELOG.md COPY --chown=$UID:$GID --from=build /app/package.json /app/package.json -# copy backend files -COPY --chown=$UID:$GID ./backend . - -# The backend rewrites its bundled static assets (favicons, splash, manifest, -# loader.js, ...) under open_webui/static at startup. Make that directory -# writable by an arbitrary UID -- which under OpenShift's restricted SCC is -# always a member of GID 0 -- so those writes don't fail with EACCES and crash -# the boot log with "[Errno 13] Permission denied". `chmod -R g=u` mirrors the -# owner bits onto the group (the Red Hat arbitrary-UID idiom). This is applied -# unconditionally because it targets a directory the app writes on every start; -# the broader, opt-in USE_PERMISSION_HARDENING below covers the rest of /app. -RUN chgrp -R 0 /app/backend/open_webui/static && \ - chmod -R g=u /app/backend/open_webui/static +# copy backend files with the ownership and static permissions prepared above +COPY --from=build /app/backend . EXPOSE 8080 diff --git a/README.md b/README.md index f3499e38f5..ab0fdd1345 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,8 @@ For more information, be sure to check out our [Open WebUI Documentation](https: - 🤖 **Models & Agents**: Wrap any base model with custom instructions, tools, and knowledge to build specialized agents. Supports dynamic variables, per-user/group access control, and community preset imports via [Open WebUI Community](https://openwebui.com/). +- ⚡ **Agentic Execution with [Open Terminal](https://github.com/open-webui/open-terminal)**: Give your agents a terminal and filesystem to carry out multi-step tasks. Let them analyze data, run scripts, fix errors, and produce files directly in chat. Scale to teams with **[Terminals (Enterprise)](https://github.com/open-webui/terminals)** for per-user isolated environments, resource limits, and automatic lifecycle management. + - 📝 **Notes**: A dedicated workspace for content outside conversations. Draft with a rich editor, use AI to rewrite selected text, and attach notes to any chat for full-context injection. - 📢 **Channels**: Real-time shared spaces where your team and AI models collaborate in one timeline. Tag models to draft or critique, with threads, reactions, pins, and access control. diff --git a/backend/open_webui/config.py b/backend/open_webui/config.py index a0855f4689..5765a26ab6 100644 --- a/backend/open_webui/config.py +++ b/backend/open_webui/config.py @@ -17,6 +17,7 @@ from authlib.integrations.starlette_client import OAuth from pydantic import BaseModel from open_webui.env import ( + USE_SLIM, DATA_DIR, DATABASE_URL, ENABLE_ADMIN_CHAT_ACCESS, @@ -227,6 +228,7 @@ if CUSTOM_NAME: #################################### ENABLE_DIRECT_CONNECTIONS = os.getenv('ENABLE_DIRECT_CONNECTIONS', 'False').lower() == 'true' +ENABLE_DIRECT_INTEGRATIONS = os.getenv('ENABLE_DIRECT_INTEGRATIONS', 'False').lower() == 'true' #################################### # OLLAMA_BASE_URL @@ -498,12 +500,12 @@ CODE_INTERPRETER_PYODIDE_PROMPT = """ # Vector Database #################################### -VECTOR_DB = os.getenv('VECTOR_DB', 'chroma') +VECTOR_DB = os.getenv('VECTOR_DB', 'pgvector' if USE_SLIM else 'chroma') # Chroma CHROMA_DATA_PATH = f'{DATA_DIR}/vector_db' -if VECTOR_DB == 'chroma': +if VECTOR_DB == 'chroma' and not USE_SLIM: import chromadb CHROMA_TENANT = os.getenv('CHROMA_TENANT', chromadb.DEFAULT_TENANT) @@ -644,7 +646,7 @@ SSL_ASSERT_FINGERPRINT = os.getenv('SSL_ASSERT_FINGERPRINT', None) ELASTICSEARCH_INDEX_PREFIX = os.getenv('ELASTICSEARCH_INDEX_PREFIX', 'open_webui_collections') # Pgvector PGVECTOR_DB_URL = os.getenv('PGVECTOR_DB_URL', DATABASE_URL) -if VECTOR_DB == 'pgvector' and not PGVECTOR_DB_URL.startswith('postgres'): +if not USE_SLIM and VECTOR_DB == 'pgvector' and not PGVECTOR_DB_URL.startswith('postgres'): raise ValueError( 'Pgvector requires setting PGVECTOR_DB_URL or using Postgres with vector extension as the primary database.' ) @@ -739,6 +741,10 @@ else: except Exception: PGVECTOR_IVFFLAT_LISTS = 100 +PGVECTOR_ITERATIVE_SCAN = os.getenv('PGVECTOR_ITERATIVE_SCAN', 'relaxed_order').strip().lower() +if PGVECTOR_ITERATIVE_SCAN not in ('off', 'relaxed_order', 'strict_order'): + PGVECTOR_ITERATIVE_SCAN = 'relaxed_order' + # openGauss OPENGAUSS_DB_URL = os.getenv('OPENGAUSS_DB_URL', DATABASE_URL) @@ -805,7 +811,7 @@ ORACLE_DB_POOL_MAX = int(os.getenv('ORACLE_DB_POOL_MAX', 10)) ORACLE_DB_POOL_INCREMENT = int(os.getenv('ORACLE_DB_POOL_INCREMENT', 1)) -if VECTOR_DB == 'oracle23ai': +if not USE_SLIM and VECTOR_DB == 'oracle23ai': if not ORACLE_DB_USER or not ORACLE_DB_PASSWORD or not ORACLE_DB_DSN: raise ValueError('Oracle23ai requires setting ORACLE_DB_USER, ORACLE_DB_PASSWORD, and ORACLE_DB_DSN.') if ORACLE_DB_USE_WALLET and (not ORACLE_WALLET_DIR or not ORACLE_WALLET_PASSWORD): @@ -1270,6 +1276,9 @@ AZURE_AI_SEARCH_ENDPOINT = os.getenv('AZURE_AI_SEARCH_ENDPOINT', '') AZURE_AI_SEARCH_INDEX_NAME = os.getenv('AZURE_AI_SEARCH_INDEX_NAME', '') EXA_API_KEY = os.getenv('EXA_API_KEY', '') +EXA_MAX_CONTENT_LENGTH = int(os.environ['EXA_MAX_CONTENT_LENGTH']) if os.getenv('EXA_MAX_CONTENT_LENGTH') else None +if EXA_MAX_CONTENT_LENGTH is not None and EXA_MAX_CONTENT_LENGTH <= 0: + raise ValueError('EXA_MAX_CONTENT_LENGTH must be a positive integer or unset') PERPLEXITY_API_KEY = os.getenv('PERPLEXITY_API_KEY', '') @@ -1293,6 +1302,12 @@ TAVILY_API_KEY = os.getenv('TAVILY_API_KEY', '') TAVILY_EXTRACT_DEPTH = os.getenv('TAVILY_EXTRACT_DEPTH', 'basic') +STAAN_API_KEY = os.getenv('STAAN_API_KEY', '') + +STAAN_MARKET = os.getenv('STAAN_MARKET', 'en-us') + +STAAN_MAX_SNIPPETS = int(os.getenv('STAAN_MAX_SNIPPETS', '0')) + PLAYWRIGHT_WS_URL = os.getenv('PLAYWRIGHT_WS_URL', '') PLAYWRIGHT_TIMEOUT = int(os.getenv('PLAYWRIGHT_TIMEOUT', '10000')) @@ -1697,6 +1712,7 @@ if default_prompt_suggestions == []: ] DEFAULT_PROMPT_SUGGESTIONS = default_prompt_suggestions +DEFAULT_PROMPT_SUGGESTIONS_I18N = {} try: model_order_list = JSONCodec.loads(os.getenv('MODEL_ORDER_LIST', '[]')) @@ -2162,6 +2178,7 @@ else: class BannerModel(BaseModel): + i18n: dict[str, dict[str, str]] | None = None id: str type: str title: str | None = None @@ -2834,6 +2851,7 @@ LDAP_ATTRIBUTE_FOR_GROUPS = os.getenv('LDAP_ATTRIBUTE_FOR_GROUPS', 'memberOf') DEFAULT_CONFIG = { 'direct.enable': ENABLE_DIRECT_CONNECTIONS, + 'direct.integrations.enable': ENABLE_DIRECT_INTEGRATIONS, 'ollama.enable': ENABLE_OLLAMA_API, 'ollama.base_urls': OLLAMA_BASE_URLS, 'ollama.api_configs': OLLAMA_API_CONFIGS, @@ -2998,6 +3016,7 @@ DEFAULT_CONFIG = { 'web.search.azure_ai_search_endpoint': AZURE_AI_SEARCH_ENDPOINT, 'web.search.azure_ai_search_index_name': AZURE_AI_SEARCH_INDEX_NAME, 'web.search.exa_api_key': EXA_API_KEY, + 'web.search.exa_max_content_length': EXA_MAX_CONTENT_LENGTH, 'web.search.perplexity_api_key': PERPLEXITY_API_KEY, 'web.search.perplexity_model': PERPLEXITY_MODEL, 'web.search.perplexity_search_context_usage': PERPLEXITY_SEARCH_CONTEXT_USAGE, @@ -3009,6 +3028,9 @@ DEFAULT_CONFIG = { 'web.search.sougou_api_sk': SOUGOU_API_SK, 'web.search.tavily_api_key': TAVILY_API_KEY, 'web.search.tavily_extract_depth': TAVILY_EXTRACT_DEPTH, + 'web.search.staan_api_key': STAAN_API_KEY, + 'web.search.staan_market': STAAN_MARKET, + 'web.search.staan_max_snippets': STAAN_MAX_SNIPPETS, 'web.loader.playwright_ws_url': PLAYWRIGHT_WS_URL, 'web.loader.playwright_timeout': PLAYWRIGHT_TIMEOUT, 'web.loader.firecrawl_api_key': FIRECRAWL_API_KEY, @@ -3095,7 +3117,9 @@ DEFAULT_CONFIG = { 'ui.default_models': DEFAULT_MODELS, 'ui.default_pinned_models': DEFAULT_PINNED_MODELS, 'ui.default_interface_settings': DEFAULT_INTERFACE_SETTINGS, + 'ui.i18n': {}, 'ui.prompt_suggestions': DEFAULT_PROMPT_SUGGESTIONS, + 'ui.prompt_suggestions_i18n': DEFAULT_PROMPT_SUGGESTIONS_I18N, 'ui.model_order_list': MODEL_ORDER_LIST, 'models.default_metadata': DEFAULT_MODEL_METADATA, 'models.default_params': DEFAULT_MODEL_PARAMS, diff --git a/backend/open_webui/env.py b/backend/open_webui/env.py index b17df7340a..5936ad9939 100644 --- a/backend/open_webui/env.py +++ b/backend/open_webui/env.py @@ -7,7 +7,9 @@ import pkgutil import re import shutil import sys +import threading import traceback +from contextlib import nullcontext from pathlib import Path from typing import Any, Optional from uuid import uuid4 @@ -40,12 +42,13 @@ except ImportError: print('dotenv not installed, skipping...') DOCKER = os.getenv('DOCKER', 'False').lower() == 'true' +USE_SLIM = os.getenv('USE_SLIM_DOCKER', 'False').lower() == 'true' USE_CUDA = os.getenv('USE_CUDA_DOCKER', 'false') DEVICE_TYPE = 'cpu' _cuda_error: Optional[str] = None -if USE_CUDA.lower() == 'true': +if not USE_SLIM and USE_CUDA.lower() == 'true': try: import torch # noqa: E402 @@ -57,7 +60,7 @@ if USE_CUDA.lower() == 'true': os.environ['USE_CUDA_DOCKER'] = 'false' USE_CUDA = 'false' -if sys.platform == 'darwin' and DEVICE_TYPE == 'cpu': +if not USE_SLIM and sys.platform == 'darwin' and DEVICE_TYPE == 'cpu': try: import torch # noqa: E402 @@ -66,6 +69,9 @@ if sys.platform == 'darwin' and DEVICE_TYPE == 'cpu': except Exception: pass +# Torch MPS inference is not thread-safe and a concurrent call kills the whole process. +MPS_INFERENCE_LOCK = threading.Lock() if DEVICE_TYPE == 'mps' else nullcontext() + #################################### # LOGGING #################################### @@ -245,8 +251,6 @@ if FROM_INIT_PY: STATIC_DIR = Path(os.getenv('STATIC_DIR', OPEN_WEBUI_DIR / 'static')) -FONTS_DIR = Path(os.getenv('FONTS_DIR', OPEN_WEBUI_DIR / 'static' / 'fonts')) - FRONTEND_BUILD_DIR = Path(os.getenv('FRONTEND_BUILD_DIR', BASE_DIR / 'build')).resolve() if FROM_INIT_PY: @@ -367,6 +371,9 @@ ENABLE_QUERIES_CACHE = os.getenv('ENABLE_QUERIES_CACHE', 'False').lower() == 'tr ENABLE_ADMIN_CHAT_ACCESS = os.getenv('ENABLE_ADMIN_CHAT_ACCESS', 'True').lower() == 'true' RAG_SYSTEM_CONTEXT = os.getenv('RAG_SYSTEM_CONTEXT', 'False').lower() == 'true' +# Empty by default: chunk metadata also holds internal bookkeeping (file hashes, collection names, scores). +RAG_SOURCE_METADATA_KEYS = [key.strip() for key in os.getenv('RAG_SOURCE_METADATA_KEYS', '').split(',') if key.strip()] + #################################### # REDIS #################################### @@ -381,6 +388,14 @@ try: except ValueError: REDIS_RESPONSE_STREAM_TTL = 3600 +# Seconds a task survives without a heartbeat. 0 disables expiry. +try: + REDIS_TASK_TTL = int(os.getenv('REDIS_TASK_TTL', '300')) + if REDIS_TASK_TTL != 0 and REDIS_TASK_TTL < 60: + REDIS_TASK_TTL = 300 +except ValueError: + REDIS_TASK_TTL = 300 + REDIS_SENTINEL_HOSTS = os.getenv('REDIS_SENTINEL_HOSTS', '') REDIS_SENTINEL_PORT = os.getenv('REDIS_SENTINEL_PORT', '26379') @@ -839,7 +854,7 @@ MINERU_MAX_MARKDOWN_BYTES = ( # When enabled, skips pydub-based preprocessing (format conversion, compression, # and chunked splitting) before sending files to processing engines. Useful when # the upstream provider handles these steps or when ffmpeg is unavailable. -BYPASS_PYDUB_PREPROCESSING = os.getenv('BYPASS_PYDUB_PREPROCESSING', 'False').lower() == 'true' +BYPASS_PYDUB_PREPROCESSING = USE_SLIM or os.getenv('BYPASS_PYDUB_PREPROCESSING', 'False').lower() == 'true' # When disabled (default), the OpenAI catch-all proxy endpoint (/{path:path}) # is blocked. Enable only if you need direct passthrough to upstream OpenAI- @@ -979,6 +994,7 @@ FORWARD_USER_INFO_HEADER_USER_NAME = os.getenv('FORWARD_USER_INFO_HEADER_USER_NA FORWARD_USER_INFO_HEADER_USER_ID = os.getenv('FORWARD_USER_INFO_HEADER_USER_ID', 'X-OpenWebUI-User-Id') FORWARD_USER_INFO_HEADER_USER_EMAIL = os.getenv('FORWARD_USER_INFO_HEADER_USER_EMAIL', 'X-OpenWebUI-User-Email') FORWARD_USER_INFO_HEADER_USER_ROLE = os.getenv('FORWARD_USER_INFO_HEADER_USER_ROLE', 'X-OpenWebUI-User-Role') +FORWARD_USER_INFO_HEADER_AUTH_TYPE = os.getenv('FORWARD_USER_INFO_HEADER_AUTH_TYPE', 'X-OpenWebUI-Auth-Type') FORWARD_SESSION_INFO_HEADER_MESSAGE_ID = os.getenv('FORWARD_SESSION_INFO_HEADER_MESSAGE_ID', 'X-OpenWebUI-Message-Id') FORWARD_SESSION_INFO_HEADER_CHAT_ID = os.getenv('FORWARD_SESSION_INFO_HEADER_CHAT_ID', 'X-OpenWebUI-Chat-Id') @@ -1037,6 +1053,11 @@ ENABLE_CHAT_RESPONSE_BASE64_IMAGE_URL_CONVERSION = ( ) ENABLE_API_OUTLET_FILTERS = os.getenv('ENABLE_API_OUTLET_FILTERS', 'True').lower() == 'true' +# Opt in to CPython's in-place string append optimization for streamed responses. +# Off by default for a staged rollout. Only a host already out of memory can lose +# text here; the default path (a full copy per chunk) raises there too. +ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND = os.getenv('ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND', 'False').lower() == 'true' + # When enabled, uses a hardcoded extension-to-MIME dictionary as a last-resort # fallback when both mimetypes.guess_type() and file.meta.content_type fail to # determine the content type. This can help on minimal container images (e.g. diff --git a/backend/open_webui/internal/db.py b/backend/open_webui/internal/db.py index 3ad1ca553a..7fb0d17b19 100644 --- a/backend/open_webui/internal/db.py +++ b/backend/open_webui/internal/db.py @@ -27,6 +27,7 @@ from open_webui.env import ( DATABASE_URL, ENABLE_DB_MIGRATIONS, OPEN_WEBUI_DIR, + USE_SLIM, ) from open_webui.utils.json_codec import JSONCodec from sqlalchemy import Dialect, MetaData, create_engine, event, types @@ -142,6 +143,17 @@ class JSONField(types.TypeDecorator): # TEXT-backed JSON storage return JSONField(length=self.impl.length) +if USE_SLIM: + if make_url(DATABASE_URL).get_backend_name() not in ('sqlite', 'postgresql', 'postgres'): + raise ValueError( + 'Slim requires SQLite or PostgreSQL for DATABASE_URL. Use the standard image for other databases.' + ) + if DATABASE_ENABLE_IAM_TOKEN_AUTH: + raise ValueError( + 'AWS RDS IAM authentication requires the standard image. Slim supports PostgreSQL database credentials.' + ) + + # Normalize SSL params from the URL once; the sync engine needs them # reattached in canonical libpq form for psycopg2. _url_without_ssl, _ssl_dict = extract_ssl_params_from_url(DATABASE_URL) diff --git a/backend/open_webui/main.py b/backend/open_webui/main.py index 67a14f16b2..e7c2915130 100644 --- a/backend/open_webui/main.py +++ b/backend/open_webui/main.py @@ -74,7 +74,9 @@ from open_webui.config import ( seed_registered_defaults, ) from open_webui.constants import ERROR_MESSAGES, TASKS +from open_webui.utils.recurrence import RecurrenceEvaluationTimeout from open_webui.env import ( + USE_SLIM, AIOHTTP_CLIENT_SESSION_SSL, AUDIT_EXCLUDED_PATHS, AUDIT_INCLUDED_PATHS, @@ -106,12 +108,14 @@ from open_webui.env import ( MAX_BODY_LOG_SIZE, # Redis REDIS_KEY_PREFIX, + REDIS_TASK_TTL, REDIS_URL, RESET_CONFIG_ON_START, SAFE_MODE, SCIM_TOKEN, VERSION, WEBSOCKET_HEARTBEAT_INTERVAL, + WEBSOCKET_MANAGER, # Admin Account Runtime Creation WEBUI_ADMIN_EMAIL, WEBUI_ADMIN_NAME, @@ -188,6 +192,7 @@ from open_webui.socket.main import ( get_user_id_from_session_pool, periodic_session_pool_cleanup, periodic_usage_pool_cleanup, + redis_event_listener, ) from open_webui.socket.main import ( app as socket_app, @@ -199,6 +204,7 @@ from open_webui.tasks import ( list_task_ids_by_item_id, list_tasks, redis_task_command_listener, + redis_task_heartbeat, stop_item_tasks, stop_task, ) # Import from tasks.py @@ -385,6 +391,11 @@ async def lifespan(app: FastAPI): if app.state.redis is not None: app.state.redis_task_command_listener = asyncio.create_task(redis_task_command_listener(app)) + if REDIS_TASK_TTL > 0: + app.state.redis_task_heartbeat = asyncio.create_task(redis_task_heartbeat(app)) + + if WEBSOCKET_MANAGER == 'redis': + app.state.redis_event_listener = asyncio.create_task(redis_event_listener()) app.state.periodic_usage_pool_cleanup = asyncio.create_task(periodic_usage_pool_cleanup()) app.state.periodic_session_pool_cleanup = asyncio.create_task(periodic_session_pool_cleanup()) @@ -472,6 +483,12 @@ async def lifespan(app: FastAPI): if hasattr(app.state, 'redis_task_command_listener'): app.state.redis_task_command_listener.cancel() + if hasattr(app.state, 'redis_task_heartbeat'): + app.state.redis_task_heartbeat.cancel() + + if hasattr(app.state, 'redis_event_listener'): + app.state.redis_event_listener.cancel() + app.state.periodic_usage_pool_cleanup.cancel() app.state.periodic_session_pool_cleanup.cancel() app.state.scheduler_worker_loop.cancel() @@ -494,6 +511,12 @@ app = FastAPI( lifespan=lifespan, ) + +@app.exception_handler(RecurrenceEvaluationTimeout) +async def recurrence_timeout_handler(request: Request, exc: RecurrenceEvaluationTimeout): + return JSONResponse(status_code=400, content={'detail': str(exc)}) + + # Used by readiness checks to gate traffic until startup work is done. app.state.startup_complete = False @@ -1463,8 +1486,8 @@ async def chat_completion( async def run_initial_title_generation(): try: await background_tasks_handler(title_ctx) - except Exception as e: - log.debug('Error generating initial chat title: %s', e) + except Exception: + log.exception('Error generating initial chat title') asyncio.create_task(run_initial_title_generation()) else: @@ -1623,6 +1646,9 @@ async def chat_completion( async def process_chat(request, form_data, user, metadata, model, tasks=None): try: + ctx = None + if metadata.get('assistant_message_id'): + ctx = await build_chat_response_context(request, form_data, user, model, metadata, tasks, []) form_data, metadata, events = await process_chat_payload(request, form_data, user, metadata, model) if await drain_approved_tool_calls(request, form_data, user, model, metadata): @@ -1633,12 +1659,15 @@ async def chat_completion( # When the upstream provider returns an error (e.g. HTTP 400 # content-filter, quota exceeded), generate_chat_completion # returns a JSONResponse instead of raising. Detect this and - # raise so the except-block below emits chat:message:error + - # chat:tasks:cancel, unblocking the frontend. + # raise so the except-block below emits a terminal + # chat:message:error, unblocking the frontend. if isinstance(response, JSONResponse) and response.status_code >= 400: raise Exception(get_response_error_detail(response)) - ctx = await build_chat_response_context(request, form_data, user, model, metadata, tasks, events) + if ctx is None: + ctx = await build_chat_response_context(request, form_data, user, model, metadata, tasks, events) + else: + ctx.update(form_data=form_data, metadata=metadata, events=events) return await process_chat_response(response, ctx) except asyncio.CancelledError: @@ -1667,6 +1696,7 @@ async def chat_completion( { 'parentId': metadata.get('user_message_id', None), 'error': {'content': error_detail}, + 'done': True, }, ) @@ -1675,12 +1705,9 @@ async def chat_completion( await event_emitter( { 'type': 'chat:message:error', - 'data': {'error': {'content': error_detail}}, + 'data': {'error': {'content': error_detail}, 'done': True}, } ) - await event_emitter( - {'type': 'chat:tasks:cancel'}, - ) except Exception: pass @@ -2235,6 +2262,7 @@ async def get_app_config(request: Request): 'auth.enable_api_keys', 'ui.enable_password_change_form', 'direct.enable', + 'direct.integrations.enable', 'folders.enable', 'folders.max_file_count', 'channels.enable', @@ -2260,7 +2288,9 @@ async def get_app_config(request: Request): 'ui.default_models', 'ui.default_pinned_models', 'ui.default_interface_settings', + 'ui.i18n', 'ui.prompt_suggestions', + 'ui.prompt_suggestions_i18n', 'code_execution.engine', 'code_interpreter.engine', 'audio.tts.engine', @@ -2283,6 +2313,7 @@ async def get_app_config(request: Request): 'name': app.state.WEBUI_NAME, 'version': VERSION, 'default_locale': str(DEFAULT_LOCALE), + 'i18n': config.get('ui.i18n') or {}, 'oauth': { # Hide providers (and thus the login buttons / auto-redirect) when OAuth # is disabled, without clearing the admin's provider configuration. @@ -2294,6 +2325,7 @@ async def get_app_config(request: Request): 'auto_redirect': config.get('oauth.auto_redirect'), }, 'features': { + 'slim': USE_SLIM, # --- Public: required by login/signup page pre-auth --- 'auth': WEBUI_AUTH, 'auth_trusted_header': bool(WEBUI_AUTH_TRUSTED_EMAIL_HEADER), @@ -2317,6 +2349,7 @@ async def get_app_config(request: Request): 'enable_public_active_users_count': ENABLE_PUBLIC_ACTIVE_USERS_COUNT, 'enable_easter_eggs': ENABLE_EASTER_EGGS, 'enable_direct_connections': config.get('direct.enable'), + 'enable_direct_integrations': config.get('direct.integrations.enable', False), 'enable_plugins': ENABLE_PLUGINS, 'enable_folders': config.get('folders.enable'), 'folder_max_file_count': config.get('folders.max_file_count'), @@ -2361,6 +2394,7 @@ async def get_app_config(request: Request): 'default_models': config.get('ui.default_models'), 'default_pinned_models': config.get('ui.default_pinned_models'), 'default_prompt_suggestions': config.get('ui.prompt_suggestions'), + 'default_prompt_suggestions_i18n': config.get('ui.prompt_suggestions_i18n'), **({'user_count': user_count} if user_count is not None else {}), 'code': { 'engine': config.get('code_execution.engine'), @@ -2575,8 +2609,8 @@ async def get_app_latest_release_version(user=Depends(get_verified_user)): return {'current': VERSION, 'latest': latest_version[1:]} except Exception as e: - log.debug(e) - return {'current': VERSION, 'latest': VERSION} + log.warning(f'Version update check failed: {e}') + return {'current': VERSION, 'latest': None} @app.get('/api/changelog') diff --git a/backend/open_webui/models/auths.py b/backend/open_webui/models/auths.py index 629c5eb6c5..b9f8b4b2a0 100644 --- a/backend/open_webui/models/auths.py +++ b/backend/open_webui/models/auths.py @@ -9,7 +9,7 @@ from typing import Optional import bcrypt from open_webui.internal.db import Base, JSONField, get_async_db_context from open_webui.models.users import User, UserModel, UserProfileImageResponse, Users -from open_webui.utils.validate import validate_profile_image_url +from open_webui.utils.validate import validate_image_url from pydantic import BaseModel, field_validator from sqlalchemy import Boolean, Column, String, Text, delete, select, update from sqlalchemy.exc import IntegrityError @@ -87,7 +87,7 @@ class SignupForm(BaseModel): @classmethod def check_profile_image_url(cls, v: str | None) -> str | None: if v is not None: - return validate_profile_image_url(v) + return validate_image_url(v) return v diff --git a/backend/open_webui/models/automations.py b/backend/open_webui/models/automations.py index 11a906670b..8c7f6089e3 100644 --- a/backend/open_webui/models/automations.py +++ b/backend/open_webui/models/automations.py @@ -312,6 +312,7 @@ class AutomationTable: rows = result.scalars().all() from open_webui.utils.automations import next_run_ns + from open_webui.utils.recurrence import RecurrenceEvaluationTimeout # Batch-fetch user timezones so rescheduling respects each # user's local timezone instead of falling back to server time. @@ -323,13 +324,20 @@ class AutomationTable: tz_result = await db.execute(select(User.id, User.timezone).where(User.id.in_(user_ids))) timezone_by_user_id = {uid: tz for uid, tz in tz_result.all()} + claimed = [] for row in rows: + try: + next_run_at = await next_run_ns(row.data.get('rrule', ''), tz=timezone_by_user_id.get(row.user_id)) + except RecurrenceEvaluationTimeout: + log.warning('Skipping automation %s: recurrence evaluation timed out', row.id) + continue row.last_run_at = now_ns - row.next_run_at = next_run_ns(row.data.get('rrule', ''), tz=timezone_by_user_id.get(row.user_id)) + row.next_run_at = next_run_at + claimed.append(row) await db.commit() - return [AutomationModel.model_validate(r) for r in rows] + return [AutomationModel.model_validate(r) for r in claimed] #################### diff --git a/backend/open_webui/models/calendar.py b/backend/open_webui/models/calendar.py index 3a3147a7ad..fd7390fcea 100644 --- a/backend/open_webui/models/calendar.py +++ b/backend/open_webui/models/calendar.py @@ -8,7 +8,7 @@ from open_webui.constants import ERROR_MESSAGES from open_webui.models.access_grants import AccessGrantModel, AccessGrants from open_webui.models.groups import Groups from open_webui.models.users import User, UserModel, UserResponse -from pydantic import BaseModel, ConfigDict, Field, field_validator +from pydantic import BaseModel, ConfigDict, Field from sqlalchemy import ( JSON, BigInteger, @@ -179,6 +179,20 @@ class CalendarUpdateForm(BaseModel): access_grants: Optional[list[dict]] = None +async def validate_calendar_rrule(value: Optional[str]) -> None: + if value: + from open_webui.utils.recurrence import rrule_interval_seconds + + try: + interval = await rrule_interval_seconds(value) + except ValueError: + raise + except Exception as e: + raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) from e + if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: + raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) + + class CalendarEventForm(BaseModel): calendar_id: str title: str @@ -193,22 +207,6 @@ class CalendarEventForm(BaseModel): meta: Optional[dict] = None attendees: Optional[list[dict]] = None - @field_validator('rrule') - @classmethod - def reject_sub_daily_rrule(cls, value: Optional[str]) -> Optional[str]: - if value: - from open_webui.utils.automations import rrule_interval_seconds - - try: - interval = rrule_interval_seconds(value) - except ValueError: - raise - except Exception as e: - raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) - if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: - raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) - return value - class CalendarEventUpdateForm(BaseModel): calendar_id: Optional[str] = None @@ -225,22 +223,6 @@ class CalendarEventUpdateForm(BaseModel): is_cancelled: Optional[bool] = None attendees: Optional[list[dict]] = None - @field_validator('rrule') - @classmethod - def reject_sub_daily_rrule(cls, value: Optional[str]) -> Optional[str]: - if value: - from open_webui.utils.automations import rrule_interval_seconds - - try: - interval = rrule_interval_seconds(value) - except ValueError: - raise - except Exception as e: - raise ValueError(ERROR_MESSAGES.AUTOMATION_INVALID_RRULE(e)) - if interval is not None and interval < MIN_CALENDAR_RRULE_INTERVAL_SECONDS: - raise ValueError(ERROR_MESSAGES.CALENDAR_RRULE_TOO_FREQUENT) - return value - class RSVPForm(BaseModel): status: str # 'accepted' | 'declined' | 'tentative' | 'pending' @@ -465,6 +447,7 @@ class CalendarEventTable: async def insert_new_event( self, user_id: str, form_data: CalendarEventForm, db: Optional[AsyncSession] = None ) -> Optional[CalendarEventModel]: + await validate_calendar_rrule(form_data.rrule) async with get_async_db_context(db) as db: now = int(time.time_ns()) event = CalendarEvent( @@ -695,6 +678,7 @@ class CalendarEventTable: async def update_event_by_id( self, id: str, form_data: CalendarEventUpdateForm, db: Optional[AsyncSession] = None ) -> Optional[CalendarEventModel]: + await validate_calendar_rrule(form_data.rrule) async with get_async_db_context(db) as db: result = await db.execute(select(CalendarEvent).filter(CalendarEvent.id == id)) event = result.scalars().first() diff --git a/backend/open_webui/models/channels.py b/backend/open_webui/models/channels.py index c30e2347b9..6742815116 100644 --- a/backend/open_webui/models/channels.py +++ b/backend/open_webui/models/channels.py @@ -10,7 +10,7 @@ from open_webui.models.access_grants import ( ) from open_webui.models.groups import Groups from open_webui.models.users import User -from open_webui.utils.validate import validate_profile_image_url +from open_webui.utils.validate import validate_image_url from pydantic import BaseModel, ConfigDict, Field, field_validator from sqlalchemy import ( JSON, @@ -253,7 +253,7 @@ class ChannelWebhookForm(BaseModel): def check_profile_image_url(cls, v: Optional[str]) -> Optional[str]: if v is None: return v - return validate_profile_image_url(v) + return validate_image_url(v) class ChannelTable: diff --git a/backend/open_webui/models/chats.py b/backend/open_webui/models/chats.py index fc972cb4bc..2af71fd944 100644 --- a/backend/open_webui/models/chats.py +++ b/backend/open_webui/models/chats.py @@ -258,6 +258,7 @@ class ChatForm(BaseModel): class ChatImportForm(ChatForm): meta: dict | None = {} pinned: bool | None = False + archived: bool | None = False current_message_id: str | None = None created_at: int | None = None updated_at: int | None = None @@ -267,11 +268,6 @@ class ChatsImportForm(BaseModel): chats: list[ChatImportForm] -class ChatTitleMessagesForm(BaseModel): - title: str - messages: list[dict] - - class ChatTitleForm(BaseModel): title: str @@ -309,6 +305,7 @@ class ChatTitleIdResponse(BaseModel): last_read_at: int | None = None snippet: str | None = None active: bool = False + archived: bool = False class SharedChatResponse(BaseModel): @@ -649,6 +646,7 @@ class ChatTable: 'meta': form_data.meta, 'variables': form_data.variables or {}, 'pinned': form_data.pinned, + 'archived': form_data.archived, 'folder_id': form_data.folder_id, 'current_message_id': form_data.current_message_id or self.get_current_message_id(form_data.chat), 'created_at': (form_data.created_at if form_data.created_at else int(time.time())), @@ -1990,10 +1988,8 @@ class ChatTable: ] # Extract folder names - folders = await Folders.search_folders_by_names( - user_id, - [word.replace('folder:', '') for word in search_text_words if word.startswith('folder:')], - ) + folder_names = [word.replace('folder:', '') for word in search_text_words if word.startswith('folder:')] + folders = await Folders.search_folders_by_names(user_id, folder_names) folder_ids = [folder.id for folder in folders] is_pinned = None @@ -2037,7 +2033,7 @@ class ChatTable: else: stmt = stmt.filter(Chat.share_id.is_(None)) - if folder_ids: + if folder_names: stmt = stmt.filter(Chat.folder_id.in_(folder_ids)) # Check if the database dialect is either 'sqlite' or 'postgresql' @@ -2540,18 +2536,15 @@ class ChatTable: except Exception: return False - async def move_chats_by_user_id_and_folder_id( + async def move_chats_by_folder_id( self, - user_id: str, folder_id: str, new_folder_id: str | None, db: AsyncSession | None = None, ) -> bool: try: async with get_async_db_context(db) as session: - await session.execute( - update(Chat).filter_by(user_id=user_id, folder_id=folder_id).values(folder_id=new_folder_id) - ) + await session.execute(update(Chat).filter_by(folder_id=folder_id).values(folder_id=new_folder_id)) await session.commit() return True diff --git a/backend/open_webui/models/knowledge.py b/backend/open_webui/models/knowledge.py index 848232d6f2..9361da9ad9 100644 --- a/backend/open_webui/models/knowledge.py +++ b/backend/open_webui/models/knowledge.py @@ -1069,6 +1069,26 @@ class KnowledgeTable: for child_id in child_ids: await self._delete_files_in_subtree(child_id, db=db) + async def get_files_by_id_and_directory_id( + self, + knowledge_id: str, + directory_id: str, + db: Optional[AsyncSession] = None, + ) -> list[FileModel]: + """Get all files in a directory and its subdirectories.""" + async with get_async_db_context(db) as db: + directory_ids = [directory_id] + for parent_id in directory_ids: + result = await db.execute(select(KnowledgeDirectory.id).filter_by(parent_id=parent_id)) + directory_ids.extend(result.scalars().all()) + result = await db.execute( + select(File) + .join(KnowledgeFile, File.id == KnowledgeFile.file_id) + .filter(KnowledgeFile.knowledge_id == knowledge_id) + .filter(KnowledgeFile.directory_id.in_(directory_ids)) + ) + return [FileModel.model_validate(file) for file in result.scalars().all()] + async def move_file_to_directory( self, knowledge_id: str, diff --git a/backend/open_webui/models/models.py b/backend/open_webui/models/models.py index 17083e4b42..00a7a2650e 100755 --- a/backend/open_webui/models/models.py +++ b/backend/open_webui/models/models.py @@ -10,17 +10,13 @@ from open_webui.models.access_grants import AccessGrantModel, AccessGrants from open_webui.models.groups import Groups from open_webui.models.users import User, UserModel, UserResponse, Users from open_webui.utils.misc import json_text_variants -from open_webui.utils.validate import validate_profile_image_url -from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator +from open_webui.utils.validate import validate_image_url +from pydantic import BaseModel, ConfigDict, Field, ValidationInfo, field_validator, model_validator from sqlalchemy import BigInteger, Boolean, Column, String, Text, cast, delete, func, or_, select, update from sqlalchemy.ext.asyncio import AsyncSession log = logging.getLogger(__name__) -# Track invalid profile_image_url values we've already warned about so we -# don't flood the logs on every DB read (the validator fires per-row). -_warned_profile_urls: set[str] = set() - def normalize_model_tags(tags: Any) -> list[dict[str, str]]: if not isinstance(tags, list): @@ -79,26 +75,24 @@ class ModelMeta(BaseModel): """Metadata for a workspace model entry (profile, description, tags, capabilities).""" profile_image_url: str | None = None + background_image_url: str | None = None description: str | None = Field(default=None, description='User-facing description of the model.') + i18n: dict[str, Any] | None = None capabilities: dict | None = None knowledge: list[Any] | None = None model_config = ConfigDict(extra='allow') - @field_validator('profile_image_url', mode='before') + @field_validator('profile_image_url', 'background_image_url', mode='before') @classmethod - def check_profile_image_url(cls, v: str | None) -> str | None: + def check_image_url(cls, v: str | None, info: ValidationInfo) -> str | None: if v is None: return v try: - return validate_profile_image_url(v) + return validate_image_url(v, file_only=info.field_name == 'background_image_url') except ValueError: - if v not in _warned_profile_urls: - _warned_profile_urls.add(v) - log.warning( - 'Clearing invalid profile_image_url stored in DB (likely a legacy SVG data-URI): %.80s…', - v, - ) + if info.field_name == 'background_image_url': + raise return None @field_validator('knowledge', mode='before') @@ -175,7 +169,7 @@ class ModelAccessListResponse(BaseModel): class ModelForm(BaseModel): model_config = ConfigDict(extra='ignore') - id: str + id: str = Field(pattern=r'^\S+$') base_model_id: str | None = None name: str meta: ModelMeta @@ -248,11 +242,14 @@ class ModelsTable: return models async def get_models( - self, writable_by_user_id: str | None = None, db: AsyncSession | None = None + self, writable_by_user_id: str | None = None, db: AsyncSession | None = None, ids: list[str] | None = None ) -> list[ModelUserResponse]: async with get_async_db_context(db) as db: stmt = select(Model).filter(Model.base_model_id != None) + if ids is not None: + stmt = stmt.filter(Model.id.in_(ids)) + if writable_by_user_id: user_group_ids = { group.id for group in await Groups.get_groups_by_member_id(writable_by_user_id, db=db) @@ -290,13 +287,15 @@ class ModelsTable: ) return models - async def get_model_owners_attaching_file(self, file_id: str, db: AsyncSession | None = None) -> dict[str, str]: - """Map of model id to owner id for workspace models whose knowledge attaches this file.""" + async def get_model_owner_ids_by_file_id( + self, file_id: str, db: AsyncSession | None = None, include_background: bool = False + ) -> dict[str, str]: + """Return model IDs mapped to owner IDs for models referencing the file.""" async with get_async_db_context(db) as db: # File ids are server-generated uuids, so the text match can only over-match. result = await db.execute( select(Model.id, Model.user_id, Model.meta).filter( - Model.base_model_id.is_not(None), cast(Model.meta, String).like(f'%"{file_id}"%') + Model.base_model_id.is_not(None), cast(Model.meta, String).like(f'%{file_id}%') ) ) return { @@ -306,6 +305,7 @@ class ModelsTable: isinstance(item, dict) and item.get('type') == 'file' and item.get('id') == file_id for item in meta.get('knowledge') or [] ) + or (include_background and meta.get('background_image_url') == f'/api/v1/files/{file_id}/content') } @staticmethod @@ -448,11 +448,13 @@ class ModelsTable: return ModelListResponse(items=models, total=total) - async def get_model_meta_by_id(self, id: str, db: AsyncSession | None = None) -> tuple[dict, int | None]: - """Return (meta, updated_at) for a model, skipping access grant resolution.""" + async def get_model_meta_by_id( + self, id: str, db: AsyncSession | None = None + ) -> tuple[dict, str, int | None] | None: + """Return (meta, user_id, updated_at) for a model, skipping access grant resolution.""" try: async with get_async_db_context(db) as db: - result = await db.execute(select(Model.meta, Model.updated_at).filter_by(id=id)) + result = await db.execute(select(Model.meta, Model.user_id, Model.updated_at).filter_by(id=id)) return result.first() except Exception: return None diff --git a/backend/open_webui/models/prompts.py b/backend/open_webui/models/prompts.py index e6e07eacc7..f0e4a3e9b8 100644 --- a/backend/open_webui/models/prompts.py +++ b/backend/open_webui/models/prompts.py @@ -506,14 +506,16 @@ class PromptsTable: ) # Update prompt fields - prompt.name = form_data.name prompt.command = form_data.command - prompt.content = form_data.content - prompt.data = form_data.data or prompt.data - prompt.meta = form_data.meta or prompt.meta - if form_data.tags is not None: - prompt.tags = form_data.tags + if form_data.is_production: + prompt.name = form_data.name + prompt.content = form_data.content + prompt.data = form_data.data or prompt.data + prompt.meta = form_data.meta or prompt.meta + + if form_data.tags is not None: + prompt.tags = form_data.tags if form_data.access_grants is not None: await AccessGrants.set_access_grants('prompt', prompt.id, form_data.access_grants, db=session) @@ -531,7 +533,7 @@ class PromptsTable: 'command': prompt.command, 'data': form_data.data or {}, 'meta': form_data.meta or {}, - 'tags': prompt.tags or [], + 'tags': form_data.tags if form_data.tags is not None else (prompt.tags or []), 'access_grants': [grant.model_dump() for grant in current_access_grants], } diff --git a/backend/open_webui/models/skills.py b/backend/open_webui/models/skills.py index cff4778394..9aabdb5bf1 100644 --- a/backend/open_webui/models/skills.py +++ b/backend/open_webui/models/skills.py @@ -33,6 +33,7 @@ class Skill(Base): class SkillMeta(BaseModel): + i18n: dict[str, dict[str, str]] | None = None tags: Optional[list[str]] = [] diff --git a/backend/open_webui/models/tools.py b/backend/open_webui/models/tools.py index ddb4d78581..60c4ab9f6d 100644 --- a/backend/open_webui/models/tools.py +++ b/backend/open_webui/models/tools.py @@ -34,6 +34,7 @@ class Tool(Base): # database table definition class ToolMeta(BaseModel): + i18n: dict[str, dict[str, str]] | None = None description: str | None = None manifest: dict | None = {} has_user_valves: bool = False diff --git a/backend/open_webui/models/users.py b/backend/open_webui/models/users.py index b7633759f5..0235b7d787 100644 --- a/backend/open_webui/models/users.py +++ b/backend/open_webui/models/users.py @@ -4,12 +4,18 @@ from __future__ import annotations import datetime import time -from typing import Optional +from typing import Literal, Optional from open_webui.env import DATABASE_USER_ACTIVE_STATUS_UPDATE_INTERVAL from open_webui.internal.db import Base, JSONField, get_async_db_context from open_webui.utils.misc import throttle -from open_webui.utils.validate import validate_profile_image_url -from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator +from open_webui.utils.validate import validate_image_url +from pydantic import ( + BaseModel, + ConfigDict, + Field, + field_validator, + model_validator, +) from sqlalchemy import ( JSON, BigInteger, @@ -36,6 +42,97 @@ from sqlalchemy.ext.asyncio import AsyncSession #################### +class InterfaceTitleSettings(BaseModel): + model_config = ConfigDict(extra='forbid') + + auto: bool | None = None + + +class InterfaceImageCompressionSize(BaseModel): + model_config = ConfigDict(extra='forbid') + + width: int | float | Literal[''] | None = None + height: int | float | Literal[''] | None = None + + +class InterfaceFloatingActionButton(BaseModel): + model_config = ConfigDict(extra='forbid') + + id: str + label: str + input: bool + prompt: str + + +class InterfaceSettings(BaseModel): + """Fields owned by the Interface settings panel; not the entire user UI dict.""" + + model_config = ConfigDict(extra='forbid') + + autoTags: bool | None = None + autoFollowUps: bool | None = None + highContrastMode: bool | None = None + detectArtifacts: bool | None = None + responseAutoCopy: bool | None = None + showUsername: bool | None = None + showUpdateToast: bool | None = None + showChangelog: bool | None = None + showEmojiInCall: bool | None = None + voiceInterruption: bool | None = None + displayMultiModelResponsesInTabs: bool | None = None + chatFadeStreamingText: bool | None = None + richTextInput: bool | None = None + showFormattingToolbar: bool | None = None + insertPromptAsRichText: bool | None = None + promptAutocomplete: bool | None = None + insertSuggestionPrompt: bool | None = None + keepFollowUpPrompts: bool | None = None + insertFollowUpPrompt: bool | None = None + regenerateMenu: bool | None = None + enableMessageQueue: bool | None = None + largeTextAsFile: bool | None = None + copyFormatted: bool | None = None + collapseCodeBlocks: bool | None = None + renderMarkdownInUserMessages: bool | None = None + renderMarkdownInAssistantMessages: bool | None = None + expandDetails: bool | None = None + chatHoverPreview: bool | None = None + renderMarkdownInPreviews: bool | None = None + chatBubble: bool | None = None + widescreenMode: bool | None = None + splitLargeChunks: bool | None = None + scrollOnBranchChange: bool | None = None + scrollOnResponseGeneration: bool | None = None + showFilesOnTerminalSelect: bool | None = None + temporaryChatByDefault: bool | None = None + userLocation: bool | None = None + showChatTitleInTab: bool | None = None + iframeSandboxAllowScripts: bool | None = None + iframeSandboxAllowSameOrigin: bool | None = None + iframeSandboxAllowForms: bool | None = None + iframeSandboxAllowDownloads: bool | None = None + terminalPreviewAllowSameOrigin: bool | None = None + stylizedPdfExport: bool | None = None + hapticFeedback: bool | None = None + ctrlEnterToSend: bool | None = None + showFloatingActionButtons: bool | None = None + imageCompression: bool | None = None + imageCompressionInChannels: bool | None = None + + landingPageMode: Literal['', 'chat'] | None = None + chatDirection: Literal['LTR', 'RTL', 'auto'] | None = None + terminalFileDisplay: Literal['sidebar', 'inline'] | None = None + defaultUploadContext: Literal['full', 'focused'] | None = None + webSearch: Literal['always'] | None = None + models: list[str] | None = None + backgroundImageUrl: str | None = None + fontFamily: str | None = None + textScale: float | None = None + title: InterfaceTitleSettings | None = None + imageCompressionSize: InterfaceImageCompressionSize | None = None + floatingActionButtons: list[InterfaceFloatingActionButton] | None = None + + class UserSettings(BaseModel): ui: dict | None = {} model_config = ConfigDict(extra='allow') @@ -180,7 +277,7 @@ class UpdateProfileForm(BaseModel): @field_validator('profile_image_url') @classmethod def check_profile_image_url(cls, v: str) -> str: - return validate_profile_image_url(v) + return validate_image_url(v) class UserGroupIdsModel(UserModel): @@ -270,7 +367,7 @@ class UserUpdateForm(BaseModel): def check_profile_image_url(cls, v: str | None) -> str | None: if v is None: return v - return validate_profile_image_url(v) + return validate_image_url(v) class UsersTable: @@ -286,7 +383,7 @@ class UsersTable: db: AsyncSession | None = None, ) -> UserModel | None: try: - profile_image_url = validate_profile_image_url(profile_image_url) + profile_image_url = validate_image_url(profile_image_url) except ValueError: profile_image_url = '/user.png' @@ -657,7 +754,7 @@ class UsersTable: db: AsyncSession | None = None, ) -> UserModel | None: try: - profile_image_url = validate_profile_image_url(profile_image_url) + profile_image_url = validate_image_url(profile_image_url) except ValueError: profile_image_url = '/user.png' @@ -706,7 +803,9 @@ class UsersTable: return None scim = dict(user.scim or {}) scim[provider] = {'external_id': external_id} - user.scim = scim + if scim != user.scim: + user.scim = scim + user.updated_at = int(time.time()) await session.commit() return UserModel.model_validate(user) @@ -729,7 +828,18 @@ class UsersTable: if not user: return None user_settings = dict(user.settings or {}) + updated = dict(updated) + ui_settings = updated.pop('ui', None) user_settings.update(updated) + if ui_settings is not None: + # UI updates are field-level patches: omission keeps a value; null resets it. + current_ui_settings = dict(user_settings.get('ui') or {}) + for key, value in ui_settings.items(): + if value is None: + current_ui_settings.pop(key, None) + else: + current_ui_settings[key] = value + user_settings['ui'] = current_ui_settings user.settings = user_settings await session.commit() return UserModel.model_validate(user) diff --git a/backend/open_webui/retrieval/loaders/local.py b/backend/open_webui/retrieval/loaders/local.py new file mode 100644 index 0000000000..fad7a930be --- /dev/null +++ b/backend/open_webui/retrieval/loaders/local.py @@ -0,0 +1,121 @@ +from importlib import import_module +from pathlib import Path + +from bs4 import BeautifulSoup +from langchain_core.documents import Document + + +class TextLoader: + def __init__(self, file_path, encoding=None): + self.file_path = str(file_path) + self.encoding = encoding + + def load(self) -> list[Document]: + try: + text = Path(self.file_path).read_text(encoding=self.encoding) + except Exception as e: + raise RuntimeError(f'Error loading {self.file_path}') from e + return [ + Document( + page_content=text, + metadata={'source': self.file_path}, + ) + ] + + +class HTMLLoader(TextLoader): + def load(self) -> list[Document]: + with open(self.file_path, encoding=self.encoding) as file: + soup = BeautifulSoup(file, 'lxml') + return [ + Document( + page_content=soup.get_text(), + metadata={'source': self.file_path, 'title': str(soup.title.string) if soup.title else ''}, + ) + ] + + +class DocxLoader(TextLoader): + def load(self) -> list[Document]: + import docx2txt + + return [ + Document( + page_content=docx2txt.process(Path(self.file_path).expanduser()), + metadata={'source': self.file_path}, + ) + ] + + +class UnstructuredLoader: + def __init__(self, file_path, file_format, mode='single', **kwargs): + # Match the optional-package check; format dependencies are loaded when parsing. + import_module('unstructured') + self.file_path = file_path + self.file_format = file_format + self.mode = mode + self.kwargs = kwargs + + def load(self) -> list[Document]: + file_format = self.file_format + if file_format in ('doc', 'ppt', 'pptx'): + from unstructured.file_utils.filetype import detect_filetype + + legacy_format = 'doc' if file_format == 'doc' else 'ppt' + try: + import_module('magic') + except ImportError: + is_legacy = Path(self.file_path).suffix == f'.{legacy_format}' + else: + is_legacy = detect_filetype(self.file_path).name.lower() == legacy_format + file_format = legacy_format if is_legacy else legacy_format + 'x' + elif file_format == 'msg': + from unstructured.file_utils.filetype import detect_filetype + + detected = detect_filetype(self.file_path).name + if detected not in ('EML', 'MSG'): + raise ValueError(f'Unsupported email file type: {detected}') + file_format = 'email' if detected == 'EML' else 'msg' + + module = import_module(f'unstructured.partition.{file_format}') + elements = getattr(module, f'partition_{file_format}')(filename=self.file_path, **self.kwargs) + metadata = {'source': str(self.file_path)} + if self.mode == 'elements': + return [ + Document( + page_content=str(element), + metadata={ + **metadata, + **element.metadata.to_dict(), + 'category': element.category, + 'element_id': element.id, + }, + ) + for element in elements + ] + return [Document(page_content='\n\n'.join(map(str, elements)), metadata=metadata)] + + +class DocumentIntelligenceLoader: + def __init__(self, file_path, api_endpoint, api_key=None, azure_credential=None, api_model='prebuilt-layout'): + if (api_key is None) == (azure_credential is None): + raise ValueError('Provide exactly one of api_key or azure_credential.') + self.file_path = file_path + self.api_endpoint = api_endpoint + self.api_key = api_key + self.azure_credential = azure_credential + self.api_model = api_model + + def load(self) -> list[Document]: + from azure.ai.documentintelligence import DocumentIntelligenceClient + from azure.core.credentials import AzureKeyCredential + + credential = self.azure_credential if self.azure_credential is not None else AzureKeyCredential(self.api_key) + with DocumentIntelligenceClient(self.api_endpoint, credential) as client, open(self.file_path, 'rb') as file: + result = client.begin_analyze_document( + self.api_model, + body=file, + content_type='application/octet-stream', + output_content_format='markdown', + ).result() + return [Document(page_content=result.content, metadata=result.as_dict())] diff --git a/backend/open_webui/retrieval/loaders/main.py b/backend/open_webui/retrieval/loaders/main.py index f84b2a2b1b..74221f1aea 100644 --- a/backend/open_webui/retrieval/loaders/main.py +++ b/backend/open_webui/retrieval/loaders/main.py @@ -7,27 +7,29 @@ import zipfile import ftfy import requests +from fastapi import HTTPException from azure.identity import DefaultAzureCredential -from langchain_community.document_loaders import ( - AzureAIDocumentIntelligenceLoader, - BSHTMLLoader, - CSVLoader, - Docx2txtLoader, - PyPDFLoader, - TextLoader, -) from langchain_core.documents import Document from open_webui.env import ( AIOHTTP_CLIENT_SESSION_SSL, GLOBAL_LOG_LEVEL, + USE_SLIM, MINERU_MAX_MARKDOWN_BYTES, REQUESTS_VERIFY, ) from open_webui.retrieval.loaders.datalab_marker import DatalabMarkerLoader from open_webui.retrieval.loaders.external_document import ExternalDocumentLoader +from open_webui.retrieval.loaders.local import ( + DocumentIntelligenceLoader, + DocxLoader, + HTMLLoader, + TextLoader, + UnstructuredLoader, +) from open_webui.retrieval.loaders.mineru import MinerULoader from open_webui.retrieval.loaders.mistral import MistralLoader from open_webui.retrieval.loaders.paddleocr_vl import PADDLEOCR_VL_SUPPORTED_EXTENSIONS, PaddleOCRVLLoader +from open_webui.retrieval.loaders.pdf import PDFLoader from open_webui.utils.headers import get_user_groups_for_custom_headers from open_webui.utils.json_codec import JSONCodec @@ -50,6 +52,7 @@ known_source_ext = [ 'h', 'c', 'cs', + 'ino', 'sql', 'log', 'ini', @@ -88,6 +91,7 @@ known_source_ext = [ 'yaml', 'yml', 'toml', + 'svg', ] known_archive_ext = {'docx', 'epub', 'odt', 'pptx', 'xlsx'} @@ -160,7 +164,21 @@ class CSVLoaderWithSummary: self.encoding = encoding def load(self) -> list[Document]: - docs = CSVLoader(self.file_path, encoding=self.encoding).load() + docs = [] + try: + with open(self.file_path, newline='', encoding=self.encoding) as file: + for index, row in enumerate(csv.DictReader(file)): + fields = [] + for key, value in row.items(): + if isinstance(value, str): + value = value.strip() + elif isinstance(value, list): + value = ','.join(v.strip() for v in value) + fields.append(f'{key.strip() if key is not None else key}: {value}') + content = '\n'.join(fields) + docs.append(Document(page_content=content, metadata={'source': self.file_path, 'row': index})) + except Exception as e: + raise RuntimeError(f'Error loading {self.file_path}') from e if os.getenv('ENABLE_RAG_CSV_SUMMARY', 'False').lower() == 'true': summary = get_csv_summary(self.filename, self.file_path, self.encoding) if summary: @@ -215,7 +233,7 @@ class TikaLoader: if self.extract_images == True: headers['X-Tika-PDFextractInlineImages'] = 'true' - endpoint_path = 'tika/json/text' if self.server_version == '4' else 'tika/text' + endpoint_path = 'tika/json/md' if self.server_version == '4' else 'tika/text' content_key = 'tk:content' if self.server_version == '4' else 'X-TIKA:content' endpoint = f'{self.url.rstrip("/")}/{endpoint_path}' @@ -273,8 +291,17 @@ class DoclingLoader: ) if r.ok: result = r.json() + # Docling reports failed and skipped conversions inside HTTP 200 responses. + conversion_status = result.get('status') + if conversion_status in ['failure', 'skipped']: + error_details = ( + '; '.join(filter(None, (error.get('error_message') for error in result.get('errors', [])))) + or 'no error message provided' + ) + raise Exception(f'Error calling Docling: conversion status {conversion_status} - {error_details}') + document_data = result.get('document', {}) - md_content = document_data.get('md_content', '') + md_content = document_data.get('md_content') or '' text = md_content or '' metadata = {'Content-Type': self.mime_type} if self.mime_type else {} @@ -313,7 +340,11 @@ class Loader: def load(self, filename: str, file_content_type: str, file_path: str) -> list[Document]: loader = self._get_loader(filename, file_content_type, file_path) docs = loader.load() - return [Document(page_content=ftfy.fix_text(doc.page_content), metadata=doc.metadata) for doc in docs] + # ftfy's auto mode unescapes entities on every line before the first literal '<', rewriting the document. + return [ + Document(page_content=ftfy.fix_text(doc.page_content, unescape_html=False), metadata=doc.metadata) + for doc in docs + ] async def aload(self, filename: str, file_content_type: str, file_path: str) -> list[Document]: """ @@ -613,14 +644,14 @@ class Loader: ) ): if self.kwargs.get('DOCUMENT_INTELLIGENCE_KEY') != '': - loader = AzureAIDocumentIntelligenceLoader( + loader = DocumentIntelligenceLoader( file_path=file_path, api_endpoint=self.kwargs.get('DOCUMENT_INTELLIGENCE_ENDPOINT'), api_key=self.kwargs.get('DOCUMENT_INTELLIGENCE_KEY'), api_model=self.kwargs.get('DOCUMENT_INTELLIGENCE_MODEL'), ) else: - loader = AzureAIDocumentIntelligenceLoader( + loader = DocumentIntelligenceLoader( file_path=file_path, api_endpoint=self.kwargs.get('DOCUMENT_INTELLIGENCE_ENDPOINT'), azure_credential=DefaultAzureCredential(), @@ -666,8 +697,21 @@ class Loader: file_path=file_path, ) else: + if USE_SLIM: + if file_ext == 'csv': + return CSVLoaderWithSummary(file_path, filename, self._detect_text_encoding(file_path)) + if file_ext in ['htm', 'html']: + return HTMLLoader(file_path, encoding=self._detect_text_encoding(file_path)) + if file_ext in ['txt', 'md', 'markdown', 'rst', 'xml'] or self._is_text_file( + file_ext, file_content_type + ): + return TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) + raise HTTPException( + 503, + 'This file type requires an external document extractor in slim. Configure one that supports it.', + ) if file_ext == 'pdf': - loader = PyPDFLoader( + loader = PDFLoader( file_path, extract_images=self.kwargs.get('PDF_EXTRACT_IMAGES'), mode=self.kwargs.get('PDF_LOADER_MODE', 'page'), @@ -680,9 +724,7 @@ class Loader: ) elif file_ext == 'rst': try: - from langchain_community.document_loaders import UnstructuredRSTLoader - - loader = UnstructuredRSTLoader(file_path, mode='elements') + loader = UnstructuredLoader(file_path, 'rst', mode='elements') except ImportError: log.warning( "The 'unstructured' package is not installed. " @@ -692,9 +734,7 @@ class Loader: loader = TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_ext == 'xml': try: - from langchain_community.document_loaders import UnstructuredXMLLoader - - loader = UnstructuredXMLLoader(file_path) + loader = UnstructuredLoader(file_path, 'xml') except ImportError: log.warning( "The 'unstructured' package is not installed. " @@ -703,14 +743,12 @@ class Loader: ) loader = TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_ext in ['htm', 'html']: - loader = BSHTMLLoader(file_path, open_encoding='unicode_escape') + loader = HTMLLoader(file_path, encoding='unicode_escape') elif file_ext == 'md': loader = TextLoader(file_path, encoding=self._detect_text_encoding(file_path)) elif file_content_type == 'application/epub+zip': try: - from langchain_community.document_loaders import UnstructuredEPubLoader - - loader = UnstructuredEPubLoader(file_path) + loader = UnstructuredLoader(file_path, 'epub') except ImportError: raise ValueError( "Processing .epub files requires the 'unstructured' package. " @@ -720,12 +758,10 @@ class Loader: file_content_type == 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' or file_ext == 'docx' ): - loader = Docx2txtLoader(file_path) + loader = DocxLoader(file_path) elif file_ext == 'doc' or file_content_type == 'application/msword': try: - from langchain_community.document_loaders import UnstructuredWordDocumentLoader - - loader = UnstructuredWordDocumentLoader(file_path) + loader = UnstructuredLoader(file_path, 'doc') except ImportError: raise ValueError( "Processing .doc files requires the 'unstructured' package. " @@ -736,9 +772,7 @@ class Loader: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', ] or file_ext in ['xls', 'xlsx']: try: - from langchain_community.document_loaders import UnstructuredExcelLoader - - loader = UnstructuredExcelLoader(file_path) + loader = UnstructuredLoader(file_path, 'xlsx') except ImportError: log.warning( "The 'unstructured' package is not installed. " @@ -751,9 +785,7 @@ class Loader: 'application/vnd.openxmlformats-officedocument.presentationml.presentation', ] or file_ext in ['ppt', 'pptx']: try: - from langchain_community.document_loaders import UnstructuredPowerPointLoader - - loader = UnstructuredPowerPointLoader(file_path) + loader = UnstructuredLoader(file_path, 'ppt' if file_ext == 'ppt' else 'pptx') except ImportError: log.warning( "The 'unstructured' package is not installed. " @@ -763,12 +795,8 @@ class Loader: loader = PptxLoader(file_path) elif file_ext == 'msg': try: - from langchain_community.document_loaders import ( - UnstructuredEmailLoader, - ) - # unstructured parses .msg via python-oxmsg; avoids extract_msg's beautifulsoup4<4.14 conflict - loader = UnstructuredEmailLoader(file_path, process_attachments=False) + loader = UnstructuredLoader(file_path, 'msg', process_attachments=False) except ImportError: raise ValueError( "Processing .msg files requires the 'unstructured' package. " @@ -776,9 +804,7 @@ class Loader: ) elif file_ext == 'odt': try: - from langchain_community.document_loaders import UnstructuredODTLoader - - loader = UnstructuredODTLoader(file_path) + loader = UnstructuredLoader(file_path, 'odt') except ImportError: raise ValueError( "Processing .odt files requires the 'unstructured' package. " diff --git a/backend/open_webui/retrieval/loaders/pdf.py b/backend/open_webui/retrieval/loaders/pdf.py new file mode 100644 index 0000000000..38ecd3b1a9 --- /dev/null +++ b/backend/open_webui/retrieval/loaders/pdf.py @@ -0,0 +1,101 @@ +import datetime as dt +import io +import logging +from pathlib import Path + +from langchain_core.document_loaders import BaseLoader +from langchain_core.documents import Document + +log = logging.getLogger(__name__) + + +class PDFLoader(BaseLoader): + def __init__(self, file_path, *, extract_images=False, mode='page'): + if mode not in ('single', 'page'): + raise ValueError("PDF mode must be 'single' or 'page'") + self.file_path = str(Path(file_path).expanduser()) + self.extract_images = extract_images + self.mode = mode + self.ocr = None + + def lazy_load(self): + from pypdf import PdfReader + + with open(self.file_path, 'rb') as file: + reader = PdfReader(file) + metadata = {'producer': 'PyPDF', 'creator': 'PyPDF', 'creationdate': ''} + for key, value in (reader.metadata or {}).items(): + key = key.removeprefix('/').lower() + value = value if type(value) in (str, int) else str(value) + if key in ('creationdate', 'moddate') and isinstance(value, str): + try: + value = dt.datetime.strptime(value.replace("'", ''), 'D:%Y%m%d%H%M%S%z').isoformat() + except ValueError: + pass + metadata[key] = ( + value.strip() + if isinstance(value, str) and key not in ('creationdate', 'moddate', 'page_count', 'file_path') + else value + ) + metadata.update(source=self.file_path, total_pages=len(reader.pages)) + labels = reader.page_labels if self.mode == 'page' else None + texts = [] + for index, page in enumerate(reader.pages): + text = page.extract_text() + if self.extract_images: + image_text = self._extract_images(page) + if image_text: + text = self._merge_image_text(text, image_text) + text = text.strip() + if self.mode == 'page': + yield Document(page_content=text, metadata={**metadata, 'page': index, 'page_label': labels[index]}) + else: + texts.append(text) + if self.mode == 'single': + yield Document(page_content='\n\f'.join(texts), metadata=metadata) + + @staticmethod + def _merge_image_text(text, image_text): + # Insert before the final paragraphs/footer where possible, matching existing chunks. + position, separator = len(text), '\n\n' + for _ in range(2): + for delimiter in ('\n\n\n', '\n\n'): + found = text.rfind(delimiter, 0, position) + if found >= 0: + position, separator = found, delimiter + break + else: + break + return text[:position] + separator + image_text + text[position:] + + def _extract_images(self, page): + import numpy as np + from PIL import Image, UnidentifiedImageError + + if '/Resources' not in page or '/XObject' not in page['/Resources']: + return '' + texts = [] + xobjects = page['/Resources']['/XObject'] + for name in xobjects: + try: + stream = xobjects[name] + if stream.get('/Subtype') != '/Image': + continue + try: + # Encoded images, including CMYK JPEGs, can go straight to Pillow. + image = Image.open(io.BytesIO(stream.get_data())) + except UnidentifiedImageError: + image = stream.decode_as_image() + pixels = np.array(image.convert('RGB')) + except Exception as e: + log.warning('Skipping unreadable PDF image %s: %s', name, e) + continue + + if self.ocr is None: + from rapidocr import RapidOCR + + self.ocr = RapidOCR() + result = self.ocr(pixels) + if result and result.txts: + texts.append('\n'.join(result.txts).strip()) + return '\n\n' + '\n'.join(filter(None, texts)) + '\n\n' if any(texts) else '' diff --git a/backend/open_webui/retrieval/utils.py b/backend/open_webui/retrieval/utils.py index 294309984e..055f750122 100644 --- a/backend/open_webui/retrieval/utils.py +++ b/backend/open_webui/retrieval/utils.py @@ -10,13 +10,13 @@ from typing import Awaitable, Optional, Union from urllib.parse import quote import aiohttp +import numpy as np import requests -from huggingface_hub import snapshot_download +from fastapi import HTTPException from langchain_classic.retrievers import ( ContextualCompressionRetriever, EnsembleRetriever, ) -from langchain_community.retrievers import BM25Retriever from langchain_core.documents import Document from open_webui.config import ( RAG_EMBEDDING_CONTENT_PREFIX, @@ -32,7 +32,10 @@ from open_webui.env import ( BYPASS_RETRIEVAL_ACCESS_CONTROL, ENABLE_FORWARD_USER_INFO_HEADERS, ENABLE_RETRIEVAL_UNSCOPED_COLLECTIONS, + MPS_INFERENCE_LOCK, OFFLINE_MODE, + RAG_SOURCE_METADATA_KEYS, + USE_SLIM, ) from open_webui.models.access_grants import AccessGrants from open_webui.models.chats import Chats @@ -45,7 +48,7 @@ from open_webui.models.users import UserModel from open_webui.retrieval.loaders.youtube import YoutubeLoader from open_webui.retrieval.vector.async_client import ASYNC_VECTOR_DB_CLIENT from open_webui.retrieval.external import retrieve_external_knowledge -from open_webui.retrieval.vector.factory import VECTOR_DB_CLIENT +from open_webui.retrieval.vector.factory import get_vector_db_client from open_webui.retrieval.vector.main import GetResult, SearchResult from open_webui.retrieval.web.utils import get_web_loader from open_webui.utils.access_control.files import get_owner_accessible_folder_files, has_access_to_file @@ -62,6 +65,15 @@ from langchain_core.callbacks import CallbackManagerForRetrieverRun from langchain_core.retrievers import BaseRetriever +class BM25Retriever(BaseRetriever): + docs: list[Document] + vectorizer: Any + k: int + + def _get_relevant_documents(self, query: str, *, run_manager: CallbackManagerForRetrieverRun) -> list[Document]: + return self.vectorizer.get_top_n(query.split(), self.docs, n=self.k) + + def is_youtube_url(url: str) -> bool: youtube_regex = r'^(https?://)?(www\.)?(youtube\.com|youtu\.be)/.+$' return re.match(youtube_regex, url) is not None @@ -330,27 +342,23 @@ class VectorSearchRetriever(BaseRetriever): def query_doc(collection_name: str, query_embedding: list[float], k: int, user: UserModel = None): - try: - log.debug('query_doc:doc %s', collection_name) - result = VECTOR_DB_CLIENT.search( - collection_name=collection_name, - vectors=[query_embedding], - limit=k, - ) + log.debug('query_doc:doc %s', collection_name) + result = get_vector_db_client().search( + collection_name=collection_name, + vectors=[query_embedding], + limit=k, + ) - if result: - log.info('query_doc:result %s %s', result.ids, result.metadatas) + if result: + log.info('query_doc:result %s %s', result.ids, result.metadatas) - return result - except Exception as e: - log.exception(f'Error querying doc {collection_name} with limit {k}: {e}') - raise e + return result def get_doc(collection_name: str, user: UserModel = None): try: log.debug('get_doc:doc %s', collection_name) - result = VECTOR_DB_CLIENT.get(collection_name=collection_name) + result = get_vector_db_client().get(collection_name=collection_name) if result: log.info('query_doc:result %s %s', result.ids, result.metadatas) @@ -495,122 +503,116 @@ async def query_doc_with_hybrid_search( enable_enriched_texts: bool = False, native_hybrid_search: bool = True, ) -> dict: - try: - if native_hybrid_search and not enable_enriched_texts: - native_result = await query_doc_with_native_hybrid_search( - collection_name=collection_name, - query=query, - embedding_function=embedding_function, - k=k, - reranking_function=reranking_function, - k_reranker=k_reranker, - r=r, - hybrid_bm25_weight=hybrid_bm25_weight, - ) - if native_result is not None: - return native_result - - if collection_result is None: - collection_result = await ASYNC_VECTOR_DB_CLIENT.get(collection_name=collection_name) - - # First check if collection_result has the required attributes - if ( - not collection_result - or not hasattr(collection_result, 'documents') - or not hasattr(collection_result, 'metadatas') - ): - log.warning(f'query_doc_with_hybrid_search:no_docs {collection_name}') - return {'documents': [], 'metadatas': [], 'distances': []} - - # Now safely check the documents content after confirming attributes exist - if ( - not collection_result.documents - or len(collection_result.documents) == 0 - or not collection_result.documents[0] - ): - log.warning(f'query_doc_with_hybrid_search:no_docs {collection_name}') - return {'documents': [], 'metadatas': [], 'distances': []} - - log.debug('query_doc_with_hybrid_search:doc %s', collection_name) - - original_texts = collection_result.documents[0] - bm25_metadatas = [ - {**meta, CHUNK_HASH_KEY: _content_hash(original_texts[idx])} - for idx, meta in enumerate(collection_result.metadatas[0]) - ] - - bm25_texts = get_enriched_texts(collection_result) if enable_enriched_texts else original_texts - - bm25_retriever = BM25Retriever.from_texts( - texts=bm25_texts, - metadatas=bm25_metadatas, - ) - bm25_retriever.k = k - - vector_search_retriever = VectorSearchRetriever( + if native_hybrid_search and not enable_enriched_texts: + native_result = await query_doc_with_native_hybrid_search( collection_name=collection_name, + query=query, embedding_function=embedding_function, - top_k=k, - ) - - # Use CHUNK_HASH_KEY for dedup so enriched BM25 texts don't defeat RRF - if hybrid_bm25_weight <= 0: - ensemble_retriever = EnsembleRetriever( - retrievers=[vector_search_retriever], - weights=[1.0], - id_key=CHUNK_HASH_KEY, - ) - elif hybrid_bm25_weight >= 1: - ensemble_retriever = EnsembleRetriever( - retrievers=[bm25_retriever], - weights=[1.0], - id_key=CHUNK_HASH_KEY, - ) - else: - ensemble_retriever = EnsembleRetriever( - retrievers=[bm25_retriever, vector_search_retriever], - weights=[hybrid_bm25_weight, 1.0 - hybrid_bm25_weight], - id_key=CHUNK_HASH_KEY, - ) - - compressor = RerankCompressor( - embedding_function=embedding_function, - top_n=k_reranker, + k=k, reranking_function=reranking_function, - r_score=r, + k_reranker=k_reranker, + r=r, + hybrid_bm25_weight=hybrid_bm25_weight, + ) + if native_result is not None: + return native_result + + if collection_result is None: + collection_result = await ASYNC_VECTOR_DB_CLIENT.get(collection_name=collection_name) + + # First check if collection_result has the required attributes + if ( + not collection_result + or not hasattr(collection_result, 'documents') + or not hasattr(collection_result, 'metadatas') + ): + log.warning(f'query_doc_with_hybrid_search:no_docs {collection_name}') + return {'documents': [], 'metadatas': [], 'distances': []} + + # Now safely check the documents content after confirming attributes exist + if not collection_result.documents or len(collection_result.documents) == 0 or not collection_result.documents[0]: + log.warning(f'query_doc_with_hybrid_search:no_docs {collection_name}') + return {'documents': [], 'metadatas': [], 'distances': []} + + log.debug('query_doc_with_hybrid_search:doc %s', collection_name) + + original_texts = collection_result.documents[0] + bm25_metadatas = [ + {**meta, CHUNK_HASH_KEY: _content_hash(original_texts[idx])} + for idx, meta in enumerate(collection_result.metadatas[0]) + ] + + bm25_texts = get_enriched_texts(collection_result) if enable_enriched_texts else original_texts + + from rank_bm25 import BM25Okapi + + bm25_retriever = BM25Retriever( + docs=[Document(page_content=text, metadata=meta) for text, meta in zip(bm25_texts, bm25_metadatas)], + vectorizer=BM25Okapi([text.split() for text in bm25_texts]), + k=k, + ) + + vector_search_retriever = VectorSearchRetriever( + collection_name=collection_name, + embedding_function=embedding_function, + top_k=k, + ) + + # Use CHUNK_HASH_KEY for dedup so enriched BM25 texts don't defeat RRF + if hybrid_bm25_weight <= 0: + ensemble_retriever = EnsembleRetriever( + retrievers=[vector_search_retriever], + weights=[1.0], + id_key=CHUNK_HASH_KEY, + ) + elif hybrid_bm25_weight >= 1: + ensemble_retriever = EnsembleRetriever( + retrievers=[bm25_retriever], + weights=[1.0], + id_key=CHUNK_HASH_KEY, + ) + else: + ensemble_retriever = EnsembleRetriever( + retrievers=[bm25_retriever, vector_search_retriever], + weights=[hybrid_bm25_weight, 1.0 - hybrid_bm25_weight], + id_key=CHUNK_HASH_KEY, ) - compression_retriever = ContextualCompressionRetriever( - base_compressor=compressor, base_retriever=ensemble_retriever - ) + compressor = RerankCompressor( + embedding_function=embedding_function, + top_n=k_reranker, + reranking_function=reranking_function, + r_score=r, + ) - result = await compression_retriever.ainvoke(query) + compression_retriever = ContextualCompressionRetriever( + base_compressor=compressor, base_retriever=ensemble_retriever + ) - distances = [d.metadata.get('score') for d in result] - documents = [d.page_content for d in result] - metadatas = [d.metadata for d in result] + result = await compression_retriever.ainvoke(query) - # retrieve only min(k, k_reranker) items, sort and cut by distance if k < k_reranker - if k < k_reranker: - sorted_items = sorted(zip(distances, documents, metadatas), key=lambda x: x[0], reverse=True) - sorted_items = sorted_items[:k] + distances = [d.metadata.get('score') for d in result] + documents = [d.page_content for d in result] + metadatas = [d.metadata for d in result] - if sorted_items: - distances, documents, metadatas = map(list, zip(*sorted_items)) - else: - distances, documents, metadatas = [], [], [] + # retrieve only min(k, k_reranker) items, sort and cut by distance if k < k_reranker + if k < k_reranker: + sorted_items = sorted(zip(distances, documents, metadatas), key=lambda x: x[0], reverse=True) + sorted_items = sorted_items[:k] - result = { - 'distances': [distances], - 'documents': [documents], - 'metadatas': [metadatas], - } + if sorted_items: + distances, documents, metadatas = map(list, zip(*sorted_items)) + else: + distances, documents, metadatas = [], [], [] - log.info('query_doc_with_hybrid_search:result %s %s', result['metadatas'], result['distances']) - return result - except Exception as e: - log.exception(f'Error querying doc {collection_name} with hybrid search: {e}') - raise e + result = { + 'distances': [distances], + 'documents': [documents], + 'metadatas': [metadatas], + } + + log.info('query_doc_with_hybrid_search:result %s %s', result['metadatas'], result['distances']) + return result def merge_get_results(get_results: list[dict]) -> dict: @@ -731,7 +733,8 @@ async def query_collection( log.debug('Hybrid search failed, falling back to vector search: %s', e) results = [] - error = False + last_error = None + failed_collection_names = set() def process_query_collection(collection_name, query_embedding): try: @@ -742,11 +745,10 @@ async def query_collection( query_embedding=query_embedding, ) if result is not None: - return result.model_dump(), None - return None, None + return result.model_dump(), None, collection_name + return None, None, collection_name except Exception as e: - log.exception(f'Error when querying the collection: {e}') - return None, e + return None, e, collection_name # Sanitize: filter out None/empty queries to prevent embedding crashes # (e.g. when get_last_user_message returns None) @@ -767,14 +769,20 @@ async def query_collection( ] ) - for result, err in task_results: + for result, err, collection_name in task_results: if err is not None: - error = True + last_error = err + failed_collection_names.add(collection_name) elif result is not None: results.append(result) - if error and not results: - log.warning('All collection queries failed. No results returned.') + if failed_collection_names: + log.error( + 'query_collection: %s collection(s) had failing queries: %s', + len(failed_collection_names), + ', '.join(sorted(failed_collection_names)), + exc_info=last_error, + ) return merge_and_sort_query_results(results, k=k) @@ -791,7 +799,8 @@ async def query_collection_with_hybrid_search( enable_enriched_texts: bool = False, ) -> dict: results = [] - error = False + last_error = None + failed_collection_names = set() if not enable_enriched_texts: @@ -850,10 +859,9 @@ async def query_collection_with_hybrid_search( enable_enriched_texts=enable_enriched_texts, native_hybrid_search=False, ) - return result, None + return result, None, collection_name except Exception as e: - log.exception(f'Error when querying the collection with hybrid_search: {e}') - return None, e + return None, e, collection_name # Prepare tasks for all collections and queries # Avoid running any tasks for collections that failed to fetch data (have assigned None) @@ -867,13 +875,22 @@ async def query_collection_with_hybrid_search( # Run all queries in parallel using asyncio.gather task_results = await asyncio.gather(*[process_query(collection_name, query) for collection_name, query in tasks]) - for result, err in task_results: + for result, err, collection_name in task_results: if err is not None: - error = True + last_error = err + failed_collection_names.add(collection_name) elif result is not None: results.append(result) - if error and not results: + if failed_collection_names: + log.error( + 'query_collection_with_hybrid_search: %s collection(s) had failing queries: %s', + len(failed_collection_names), + ', '.join(sorted(failed_collection_names)), + exc_info=last_error, + ) + + if failed_collection_names and not results: raise Exception('Hybrid search failed for all collections. Using Non-hybrid search as fallback.') return merge_and_sort_query_results(results, k=k) @@ -1109,6 +1126,8 @@ def get_embedding_function( if embedding_engine == '': # Sentence transformers: CPU-bound sync operation async def async_embedding_function(query, prefix=None, user=None): + if USE_SLIM: + raise HTTPException(503, 'Configure an external embedding engine (openai, ollama, azure_openai).') # Deferred so a missing local model degrades RAG instead of crashing boot. if embedding_function is None: raise ValueError( @@ -1116,17 +1135,16 @@ def get_embedding_function( 'SentenceTransformer model name, or configure an external ' 'RAG_EMBEDDING_ENGINE (ollama, openai, azure_openai).' ) - return await asyncio.to_thread( - ( - lambda query, prefix=None: embedding_function.encode( + + def encode(): + with MPS_INFERENCE_LOCK: + return embedding_function.encode( query, batch_size=int(embedding_batch_size), **({'prompt': prefix} if prefix else {}), ).tolist() - ), - query, - prefix, - ) + + return await asyncio.to_thread(encode) return async_embedding_function elif embedding_engine in ['ollama', 'openai', 'azure_openai']: @@ -1243,6 +1261,14 @@ async def generate_embeddings( def get_reranking_function(reranking_engine, reranking_model, reranking_function, reranking_batch_size=32): + if USE_SLIM and reranking_model and reranking_engine != 'external': + + def unavailable(query, documents, user=None): + raise HTTPException( + 503, 'Configure an external reranker, or clear the reranking model to use cosine scoring.' + ) + + return unavailable if reranking_function is None: return None if reranking_engine == 'external': @@ -1250,9 +1276,14 @@ def get_reranking_function(reranking_engine, reranking_model, reranking_function [(query, doc.page_content) for doc in documents], user=user ) else: - return lambda query, documents, user=None: reranking_function.predict( - [(query, doc.page_content) for doc in documents], batch_size=int(reranking_batch_size) - ) + + def predict(query, documents, user=None): + with MPS_INFERENCE_LOCK: + return reranking_function.predict( + [(query, doc.page_content) for doc in documents], batch_size=int(reranking_batch_size) + ) + + return predict # UUIDs, SHA-256 digests, and prefixed variants thereof all fit [A-Za-z0-9_-]. @@ -1331,6 +1362,11 @@ async def filter_accessible_collections( return validated +def filter_source_metadata(metadata: dict) -> dict: + """Keep only the chunk metadata keys the operator allowed the model to see.""" + return {key: metadata[key] for key in RAG_SOURCE_METADATA_KEYS if metadata.get(key) is not None} + + async def get_sources_from_items( request, items, @@ -1683,6 +1719,8 @@ async def get_sources_from_items( def get_model_path(model: str, update_model: bool = False): + from huggingface_hub import snapshot_download + # Construct huggingface_hub kwargs with local_files_only to return the snapshot path cache_dir = os.getenv('SENTENCE_TRANSFORMERS_HOME') @@ -1728,6 +1766,17 @@ from langchain_core.callbacks import Callbacks from langchain_core.documents import BaseDocumentCompressor, Document +def cosine_similarity(query, documents) -> np.ndarray: + """Score one query against documents without loading a model runtime.""" + if len(documents) == 0: + return np.array([], dtype=float) + query = np.asarray(query, dtype=float).reshape(-1) + documents = np.asarray(documents, dtype=float) + query = query / max(np.linalg.norm(query), 1e-12) + documents = documents / np.maximum(np.linalg.norm(documents, axis=1, keepdims=True), 1e-12) + return documents @ query + + class RerankCompressor(BaseDocumentCompressor): embedding_function: Any top_n: int @@ -1763,18 +1812,18 @@ class RerankCompressor(BaseDocumentCompressor): query: str, callbacks: Callbacks | None = None, ) -> Sequence[Document]: + if not documents: + return [] reranking = self.reranking_function is not None scores = None if reranking: scores = await asyncio.to_thread(self.reranking_function, query, documents) else: - from sentence_transformers import util as st_util - query_embedding = await self.embedding_function(query, RAG_EMBEDDING_QUERY_PREFIX) doc_texts = [doc.page_content for doc in documents] document_embedding = await self.embedding_function(doc_texts, RAG_EMBEDDING_CONTENT_PREFIX) - scores = st_util.cos_sim(query_embedding, document_embedding)[0] + scores = cosine_similarity(query_embedding, document_embedding) if scores is not None: docs_with_scores = list( diff --git a/backend/open_webui/retrieval/vector/async_client.py b/backend/open_webui/retrieval/vector/async_client.py index 481f26e19f..af8df23b87 100644 --- a/backend/open_webui/retrieval/vector/async_client.py +++ b/backend/open_webui/retrieval/vector/async_client.py @@ -15,9 +15,8 @@ transparently dispatches each call to a worker thread via `asyncio.to_thread`. Async callers can `await ASYNC_VECTOR_DB_CLIENT.x(...)` in place of `VECTOR_DB_CLIENT.x(...)` and the loop stays responsive. -The original `VECTOR_DB_CLIENT` is unchanged, so callers already running -inside `run_in_threadpool` (e.g. `save_docs_to_vector_db`) are not -affected. +Client initialization and calls run in the worker thread. Synchronous callers +already inside `run_in_threadpool` use `get_vector_db_client()` directly. Thread-safety expectations -------------------------- @@ -55,7 +54,7 @@ from __future__ import annotations import asyncio from typing import Dict, List, Optional, Union -from open_webui.retrieval.vector.factory import VECTOR_DB_CLIENT +from open_webui.retrieval.vector.factory import get_vector_db_client from open_webui.retrieval.vector.main import ( GetResult, SearchResult, @@ -73,30 +72,30 @@ class AsyncVectorDBClient: typically swallowed by surrounding ``try/except``). """ - def __init__(self, sync_client: VectorDBBase) -> None: + def __init__(self, sync_client: Optional[VectorDBBase] = None) -> None: self._sync = sync_client @property def sync(self) -> VectorDBBase: """Escape hatch for code that must call the sync client directly (e.g. already inside a worker thread).""" - return self._sync + return self._sync if self._sync is not None else get_vector_db_client() @property def supports_hybrid_search(self) -> bool: - return type(self._sync).hybrid_search is not VectorDBBase.hybrid_search + return type(self.sync).hybrid_search is not VectorDBBase.hybrid_search async def has_collection(self, collection_name: str) -> bool: - return await asyncio.to_thread(self._sync.has_collection, collection_name) + return await asyncio.to_thread(lambda: self.sync.has_collection(collection_name)) async def delete_collection(self, collection_name: str) -> None: - return await asyncio.to_thread(self._sync.delete_collection, collection_name) + return await asyncio.to_thread(lambda: self.sync.delete_collection(collection_name)) async def insert(self, collection_name: str, items: List[VectorItem]) -> None: - return await asyncio.to_thread(self._sync.insert, collection_name, items) + return await asyncio.to_thread(lambda: self.sync.insert(collection_name, items)) async def upsert(self, collection_name: str, items: List[VectorItem]) -> None: - return await asyncio.to_thread(self._sync.upsert, collection_name, items) + return await asyncio.to_thread(lambda: self.sync.upsert(collection_name, items)) async def search( self, @@ -105,7 +104,7 @@ class AsyncVectorDBClient: filter: Optional[Dict] = None, limit: int = 10, ) -> Optional[SearchResult]: - return await asyncio.to_thread(self._sync.search, collection_name, vectors, filter, limit) + return await asyncio.to_thread(lambda: self.sync.search(collection_name, vectors, filter, limit)) async def hybrid_search( self, @@ -117,13 +116,7 @@ class AsyncVectorDBClient: hybrid_bm25_weight: float = 0.5, ) -> Optional[SearchResult]: return await asyncio.to_thread( - self._sync.hybrid_search, - collection_name, - query, - vectors, - filter, - limit, - hybrid_bm25_weight, + lambda: self.sync.hybrid_search(collection_name, query, vectors, filter, limit, hybrid_bm25_weight) ) async def query( @@ -132,10 +125,10 @@ class AsyncVectorDBClient: filter: Dict, limit: Optional[int] = None, ) -> Optional[GetResult]: - return await asyncio.to_thread(self._sync.query, collection_name, filter, limit) + return await asyncio.to_thread(lambda: self.sync.query(collection_name, filter, limit)) async def get(self, collection_name: str) -> Optional[GetResult]: - return await asyncio.to_thread(self._sync.get, collection_name) + return await asyncio.to_thread(lambda: self.sync.get(collection_name)) async def delete( self, @@ -143,10 +136,10 @@ class AsyncVectorDBClient: ids: Optional[List[str]] = None, filter: Optional[Dict] = None, ) -> None: - return await asyncio.to_thread(self._sync.delete, collection_name, ids, filter) + return await asyncio.to_thread(lambda: self.sync.delete(collection_name, ids, filter)) async def reset(self) -> None: - return await asyncio.to_thread(self._sync.reset) + return await asyncio.to_thread(lambda: self.sync.reset()) -ASYNC_VECTOR_DB_CLIENT = AsyncVectorDBClient(VECTOR_DB_CLIENT) +ASYNC_VECTOR_DB_CLIENT = AsyncVectorDBClient() diff --git a/backend/open_webui/retrieval/vector/dbs/chroma.py b/backend/open_webui/retrieval/vector/dbs/chroma.py index f6629164ff..2c51aeca93 100755 --- a/backend/open_webui/retrieval/vector/dbs/chroma.py +++ b/backend/open_webui/retrieval/vector/dbs/chroma.py @@ -16,6 +16,8 @@ from open_webui.config import ( CHROMA_HTTP_SSL, CHROMA_TENANT, ) +from open_webui.env import USE_SLIM +from fastapi import HTTPException from open_webui.retrieval.vector.main import ( GetResult, SearchResult, @@ -29,6 +31,8 @@ log = logging.getLogger(__name__) class ChromaClient(VectorDBBase): def __init__(self): + if USE_SLIM and not CHROMA_HTTP_HOST: + raise HTTPException(503, 'Configure CHROMA_HTTP_HOST: embedded Chroma is unavailable in slim.') settings_dict = { 'allow_reset': True, 'anonymized_telemetry': False, @@ -58,7 +62,7 @@ class ChromaClient(VectorDBBase): def has_collection(self, collection_name: str) -> bool: try: - self.client.get_collection(name=collection_name) + self.client.get_collection(name=collection_name, embedding_function=None) return True except NotFoundError: return False @@ -76,7 +80,7 @@ class ChromaClient(VectorDBBase): ) -> Optional[SearchResult]: # Search for the nearest neighbor items based on the vectors and return 'limit' number of results. try: - collection = self.client.get_collection(name=collection_name) + collection = self.client.get_collection(name=collection_name, embedding_function=None) if collection: result = collection.query( query_embeddings=vectors, @@ -105,7 +109,7 @@ class ChromaClient(VectorDBBase): def query(self, collection_name: str, filter: dict, limit: Optional[int] = None) -> Optional[GetResult]: # Query the items from the collection based on the filter. try: - collection = self.client.get_collection(name=collection_name) + collection = self.client.get_collection(name=collection_name, embedding_function=None) if collection: result = collection.get( where=filter, @@ -125,7 +129,7 @@ class ChromaClient(VectorDBBase): def get(self, collection_name: str) -> Optional[GetResult]: # Get all the items in the collection. - collection = self.client.get_collection(name=collection_name) + collection = self.client.get_collection(name=collection_name, embedding_function=None) if collection: result = collection.get() return GetResult( @@ -139,7 +143,9 @@ class ChromaClient(VectorDBBase): def insert(self, collection_name: str, items: list[VectorItem]): # Insert the items into the collection, if the collection does not exist, it will be created. - collection = self.client.get_or_create_collection(name=collection_name, metadata={'hnsw:space': 'cosine'}) + collection = self.client.get_or_create_collection( + name=collection_name, metadata={'hnsw:space': 'cosine'}, embedding_function=None + ) ids = [item['id'] for item in items] documents = [item['text'] for item in items] @@ -157,7 +163,9 @@ class ChromaClient(VectorDBBase): def upsert(self, collection_name: str, items: list[VectorItem]): # Update the items in the collection, if the items are not present, insert them. If the collection does not exist, it will be created. - collection = self.client.get_or_create_collection(name=collection_name, metadata={'hnsw:space': 'cosine'}) + collection = self.client.get_or_create_collection( + name=collection_name, metadata={'hnsw:space': 'cosine'}, embedding_function=None + ) ids = [item['id'] for item in items] documents = [item['text'] for item in items] @@ -174,7 +182,7 @@ class ChromaClient(VectorDBBase): ): # Delete the items from the collection based on the ids. try: - collection = self.client.get_collection(name=collection_name) + collection = self.client.get_collection(name=collection_name, embedding_function=None) if collection: if ids: collection.delete(ids=ids) diff --git a/backend/open_webui/retrieval/vector/dbs/milvus_multitenancy.py b/backend/open_webui/retrieval/vector/dbs/milvus_multitenancy.py index 24f1864fd4..de0a227ea3 100644 --- a/backend/open_webui/retrieval/vector/dbs/milvus_multitenancy.py +++ b/backend/open_webui/retrieval/vector/dbs/milvus_multitenancy.py @@ -24,6 +24,7 @@ from open_webui.retrieval.vector.main import ( VectorDBBase, VectorItem, ) +from open_webui.retrieval.vector.utils import process_metadata from pymilvus import DataType from pymilvus import MilvusClient as Client from pymilvus.exceptions import MilvusException @@ -191,7 +192,7 @@ class MilvusClient(VectorDBBase): 'id': item['id'], 'vector': item['vector'], 'text': text, - 'metadata': item['metadata'], + 'metadata': process_metadata(item['metadata']), RESOURCE_ID_FIELD: resource_id, } ) diff --git a/backend/open_webui/retrieval/vector/dbs/opengauss.py b/backend/open_webui/retrieval/vector/dbs/opengauss.py index cfb7166250..d8f84a2580 100644 --- a/backend/open_webui/retrieval/vector/dbs/opengauss.py +++ b/backend/open_webui/retrieval/vector/dbs/opengauss.py @@ -311,6 +311,7 @@ class OpenGaussClient(VectorDBBase): results = query.all() if not results: + self.session.rollback() return None ids = [[result.id for result in results]] @@ -333,6 +334,7 @@ class OpenGaussClient(VectorDBBase): results = query.all() if not results: + self.session.rollback() return None ids = [[result.id for result in results]] diff --git a/backend/open_webui/retrieval/vector/dbs/oracle23ai.py b/backend/open_webui/retrieval/vector/dbs/oracle23ai.py index 77408081d5..2d8fe5be3b 100644 --- a/backend/open_webui/retrieval/vector/dbs/oracle23ai.py +++ b/backend/open_webui/retrieval/vector/dbs/oracle23ai.py @@ -57,7 +57,7 @@ from open_webui.retrieval.vector.main import ( VectorDBBase, VectorItem, ) -from open_webui.retrieval.vector.utils import iter_filter_conditions +from open_webui.retrieval.vector.utils import iter_filter_conditions, process_metadata from open_webui.utils.json_codec import JSONCodec log = logging.getLogger(__name__) @@ -400,7 +400,7 @@ class Oracle23aiClient(VectorDBBase): Returns: str: JSON representation of metadata """ - return json.dumps(metadata, default=self._decimal_handler) if metadata else '{}' + return json.dumps(process_metadata(metadata), default=self._decimal_handler) if metadata else '{}' def _json_to_metadata(self, json_str: str) -> Dict: """ diff --git a/backend/open_webui/retrieval/vector/dbs/pgvector.py b/backend/open_webui/retrieval/vector/dbs/pgvector.py index 75daeccd12..04fdaed1c1 100644 --- a/backend/open_webui/retrieval/vector/dbs/pgvector.py +++ b/backend/open_webui/retrieval/vector/dbs/pgvector.py @@ -8,6 +8,7 @@ from open_webui.config import ( PGVECTOR_HNSW_M, PGVECTOR_INDEX_METHOD, PGVECTOR_INITIALIZE_MAX_VECTOR_LENGTH, + PGVECTOR_ITERATIVE_SCAN, PGVECTOR_IVFFLAT_LISTS, PGVECTOR_PGCRYPTO, PGVECTOR_PGCRYPTO_KEY, @@ -154,6 +155,7 @@ class PgvectorClient(VectorDBBase): index_method, index_options = self._vector_index_configuration() self._ensure_vector_index(index_method, index_options) self._ensure_text_search_index() + self.iterative_scan_sql = self._iterative_scan_setting(index_method) self.session.execute( text( @@ -223,6 +225,9 @@ class PgvectorClient(VectorDBBase): ) if not existing_index_def: + if index_method == 'ivfflat' and not self._has_enough_ivfflat_training_rows(): + return + index_sql = ( f'CREATE INDEX IF NOT EXISTS {index_name} ' f'ON document_chunk USING {index_method} (vector {VECTOR_OPCLASS})' @@ -237,6 +242,38 @@ class PgvectorClient(VectorDBBase): f' {index_options}' if index_options else '', ) + def _has_enough_ivfflat_training_rows(self) -> bool: + # ivfflat samples 50 rows per list to place its centroids, so recall stays poor until the table holds that many + min_training_rows = 50 * PGVECTOR_IVFFLAT_LISTS + row_count = self.session.execute( + text('SELECT count(*) FROM (SELECT 1 FROM document_chunk LIMIT :min_training_rows) AS sample'), + {'min_training_rows': min_training_rows}, + ).scalar() + + if row_count < min_training_rows: + log.info( + "Deferring vector index 'idx_document_chunk_vector' until document_chunk holds %s rows to cluster on, " + 'it has %s. Searches run as an exact scan until then.', + min_training_rows, + row_count, + ) + return False + return True + + def _iterative_scan_setting(self, index_method: str) -> Optional[str]: + if PGVECTOR_ITERATIVE_SCAN == 'off': + return None + + version = self.session.execute(text("SELECT extversion FROM pg_extension WHERE extname = 'vector'")).scalar() + version_parts = [int(part) for part in (version or '').split('.') if part.isdigit()] + if version_parts[:2] < [0, 8]: + log.info('Iterative scan needs pgvector 0.8 or newer, the server has %s.', version or 'none') + return None + + # ivfflat only accepts relaxed_order + mode = 'relaxed_order' if index_method == 'ivfflat' else PGVECTOR_ITERATIVE_SCAN + return f'SET LOCAL {index_method}.iterative_scan = {mode}' + def _ensure_text_search_index(self) -> None: if PGVECTOR_PGCRYPTO: return @@ -503,6 +540,9 @@ class PgvectorClient(VectorDBBase): .order_by(query_vectors.c.qid, subq.c.distance) ) + if self.iterative_scan_sql: + self.session.execute(text(self.iterative_scan_sql)) + result_proxy = self.session.execute(stmt) results = result_proxy.all() @@ -512,6 +552,7 @@ class PgvectorClient(VectorDBBase): metadatas = [[] for _ in range(num_queries)] if not results: + self.session.rollback() return SearchResult( ids=ids, distances=distances, @@ -631,6 +672,7 @@ class PgvectorClient(VectorDBBase): results = query.all() if not results: + self.session.rollback() return None ids = [[result.id for result in results]] @@ -670,6 +712,7 @@ class PgvectorClient(VectorDBBase): results = query.all() if not results: + self.session.rollback() return None ids = [[result.id for result in results]] diff --git a/backend/open_webui/retrieval/vector/dbs/qdrant.py b/backend/open_webui/retrieval/vector/dbs/qdrant.py index e683bb40c2..f8ed600773 100644 --- a/backend/open_webui/retrieval/vector/dbs/qdrant.py +++ b/backend/open_webui/retrieval/vector/dbs/qdrant.py @@ -22,7 +22,7 @@ from open_webui.retrieval.vector.main import ( VectorDBBase, VectorItem, ) -from open_webui.retrieval.vector.utils import iter_filter_conditions +from open_webui.retrieval.vector.utils import iter_filter_conditions, process_metadata from qdrant_client import QdrantClient as Qclient from qdrant_client.http.models import PointStruct from qdrant_client.models import models @@ -136,7 +136,7 @@ class QdrantClient(VectorDBBase): PointStruct( id=item['id'], vector=item['vector'], - payload={'text': item['text'], 'metadata': item['metadata']}, + payload={'text': item['text'], 'metadata': process_metadata(item['metadata'])}, ) for item in items ] diff --git a/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py b/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py index 00941d6bb0..870cd2b02f 100644 --- a/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py +++ b/backend/open_webui/retrieval/vector/dbs/qdrant_multitenancy.py @@ -23,7 +23,7 @@ from open_webui.retrieval.vector.main import ( VectorDBBase, VectorItem, ) -from open_webui.retrieval.vector.utils import iter_filter_conditions +from open_webui.retrieval.vector.utils import iter_filter_conditions, process_metadata from qdrant_client import QdrantClient as Qclient from qdrant_client.http.exceptions import UnexpectedResponse from qdrant_client.http.models import PointStruct @@ -182,7 +182,7 @@ class QdrantClient(VectorDBBase): vector=item['vector'], payload={ 'text': item['text'], - 'metadata': item['metadata'], + 'metadata': process_metadata(item['metadata']), TENANT_ID_FIELD: tenant_id, }, ) diff --git a/backend/open_webui/retrieval/vector/factory.py b/backend/open_webui/retrieval/vector/factory.py index 3080956163..9e17b8717c 100644 --- a/backend/open_webui/retrieval/vector/factory.py +++ b/backend/open_webui/retrieval/vector/factory.py @@ -1,8 +1,12 @@ +from threading import Lock + +from fastapi import HTTPException from open_webui.config import ( ENABLE_MILVUS_MULTITENANCY_MODE, ENABLE_QDRANT_MULTITENANCY_MODE, VECTOR_DB, ) +from open_webui.env import USE_SLIM from open_webui.retrieval.vector.main import VectorDBBase from open_webui.retrieval.vector.type import VectorType @@ -13,6 +17,11 @@ class Vector: """ get vector db instance by vector type """ + if USE_SLIM and vector_type != VectorType.PGVECTOR: + raise HTTPException( + 503, + 'Slim requires PostgreSQL/pgvector for vector storage. Set VECTOR_DB=pgvector and PGVECTOR_DB_URL, or use the standard image.', + ) match vector_type: case VectorType.MILVUS: if ENABLE_MILVUS_MULTITENANCY_MODE: @@ -88,4 +97,27 @@ class Vector: raise ValueError(f'Unsupported vector type: {vector_type}') -VECTOR_DB_CLIENT = Vector.get_vector(VECTOR_DB) +VECTOR_DB_CLIENT = None if USE_SLIM else Vector.get_vector(VECTOR_DB) +_vector_client_lock = Lock() + + +def get_vector_db_client() -> VectorDBBase: + """Initialize slim's remote client on first use so chat can start without it.""" + global VECTOR_DB_CLIENT + if VECTOR_DB_CLIENT is not None: + return VECTOR_DB_CLIENT + with _vector_client_lock: + if VECTOR_DB_CLIENT is None: + from open_webui import config + + if VECTOR_DB == VectorType.PGVECTOR and not config.PGVECTOR_DB_URL.startswith('postgres'): + raise HTTPException(503, 'Configure PGVECTOR_DB_URL for remote vector storage.') + try: + VECTOR_DB_CLIENT = Vector.get_vector(VECTOR_DB) + except HTTPException: + raise + except Exception as exc: + raise HTTPException( + 503, f'Unable to connect to configured vector database ({VECTOR_DB}): {exc}' + ) from exc + return VECTOR_DB_CLIENT diff --git a/backend/open_webui/retrieval/web/duckduckgo.py b/backend/open_webui/retrieval/web/duckduckgo.py index afd82eaf04..c9cefc25b1 100644 --- a/backend/open_webui/retrieval/web/duckduckgo.py +++ b/backend/open_webui/retrieval/web/duckduckgo.py @@ -3,7 +3,7 @@ from __future__ import annotations import logging import urllib.request -from ddgs import DDGS +from open_webui.env import USE_SLIM from open_webui.retrieval.web.main import SearchResult, get_filtered_results log = logging.getLogger(__name__) @@ -26,6 +26,13 @@ def search_duckduckgo( Returns: list[SearchResult]: A list of search results """ + if USE_SLIM: + raise ValueError( + 'DDGS is unavailable in slim. Configure another web search provider in Admin Settings > Web Search.' + ) + + from ddgs import DDGS + # The ddgs library (primp-based) does not auto-detect proxy env vars. # Resolve via stdlib getproxies() — same pattern as the other loaders. env_proxies = urllib.request.getproxies() diff --git a/backend/open_webui/retrieval/web/exa.py b/backend/open_webui/retrieval/web/exa.py index 068cc380b9..803428a342 100644 --- a/backend/open_webui/retrieval/web/exa.py +++ b/backend/open_webui/retrieval/web/exa.py @@ -1,6 +1,4 @@ import logging -from dataclasses import dataclass -from typing import Optional import requests from open_webui.retrieval.web.main import SearchResult @@ -10,18 +8,12 @@ log = logging.getLogger(__name__) EXA_API_BASE = 'https://api.exa.ai' -@dataclass -class ExaResult: - url: str - title: str - text: str - - def search_exa( api_key: str, query: str, count: int, - filter_list: Optional[list[str]] = None, + filter_list: list[str] | None = None, + max_content_length: int | None = None, ) -> list[SearchResult]: """Search using Exa Search API and return the results as a list of SearchResult objects. @@ -29,7 +21,8 @@ def search_exa( api_key (str): A Exa Search API key query (str): The query to search for count (int): Number of results to return - filter_list (Optional[list[str]]): List of domains to filter results by + filter_list (list[str] | None): List of domains to filter results by + max_content_length (int | None): Maximum characters per result; None leaves text unlimited. """ log.info('Searching with Exa for query: %s', query) @@ -39,7 +32,7 @@ def search_exa( 'query': query, 'numResults': count or 5, 'includeDomains': filter_list, - 'contents': {'text': True, 'highlights': True}, + 'contents': {'text': {'maxCharacters': max_content_length} if max_content_length is not None else True}, 'type': 'auto', # Use the auto search type (keyword or neural) } @@ -48,22 +41,13 @@ def search_exa( response.raise_for_status() data = response.json() - results = [] - for result in data['results']: - results.append( - ExaResult( - url=result['url'], - title=result['title'], - text=result['text'], - ) - ) - + results = data['results'] log.info('Found %s results', len(results)) return [ SearchResult( - link=result.url, - title=result.title, - snippet=result.text, + link=result['url'], + title=result['title'], + snippet=(result.get('text') or '')[:max_content_length], ) for result in results ] diff --git a/backend/open_webui/retrieval/web/staan.py b/backend/open_webui/retrieval/web/staan.py new file mode 100644 index 0000000000..c1af2b5b2a --- /dev/null +++ b/backend/open_webui/retrieval/web/staan.py @@ -0,0 +1,60 @@ +from __future__ import annotations + +import requests +from open_webui.retrieval.web.main import SearchResult, get_filtered_results + + +def search_staan( + api_key: str, + query: str, + count: int, + filter_list: list[str] | None = None, + market: str | None = None, + max_snippets: int | None = None, +) -> list[SearchResult]: + """Search using Staan's Web Search API and return the results as a list of SearchResult objects. + + Args: + api_key (str): A Staan API key + query (str): The query to search for + count (int): The maximum number of results to return + filter_list (list[str] | None): The domains to allow or block + market (str | None): The market to search in, e.g. 'en-us' + max_snippets (int | None): The maximum extra snippets to request per result + + Returns: + A list of SearchResult objects. + """ + url = 'https://api.staan.ai/v2/search/web' + headers = { + 'Accept': 'application/json', + 'Authorization': f'Bearer {api_key}', + } + params = {'q': query, 'market': market} + + if max_snippets: + params['extra_snippets'] = 'true' + params['max_snippets'] = max_snippets + + response = requests.get(url, headers=headers, params=params) + response.raise_for_status() + + results = response.json().get('web', {}).get('results', []) + if filter_list: + results = get_filtered_results(results, filter_list) + + return [ + SearchResult( + link=result.get('url', ''), + title=result.get('title'), + snippet=_build_snippet(result), + ) + for result in results[:count] + ] + + +def _build_snippet(result: dict) -> str: + """Combine the snippet and the extra snippets list into a single string.""" + parts = [result.get('snippet')] + parts.extend(extra.get('chunk') for extra in result.get('extra_snippets', [])) + return '\n\n'.join(part for part in parts if part) diff --git a/backend/open_webui/retrieval/web/utils.py b/backend/open_webui/retrieval/web/utils.py index a6099afe1d..c2b8be5159 100644 --- a/backend/open_webui/retrieval/web/utils.py +++ b/backend/open_webui/retrieval/web/utils.py @@ -8,6 +8,7 @@ import time import urllib.parse import urllib.request from datetime import datetime, timedelta +from importlib import import_module from typing import ( Any, AsyncIterator, @@ -29,9 +30,9 @@ import urllib3.connection import urllib3.connectionpool import validators from requests.adapters import HTTPAdapter +from fastapi import HTTPException from fastapi.concurrency import run_in_threadpool -from langchain_community.document_loaders import PlaywrightURLLoader, WebBaseLoader -from langchain_community.document_loaders.base import BaseLoader +from langchain_core.document_loaders import BaseLoader from langchain_core.documents import Document from open_webui.config import ( ENABLE_LOCAL_WEB_FETCH, @@ -58,6 +59,7 @@ from open_webui.env import ( AIOHTTP_CLIENT_SSL_CERT_FILE, AIOHTTP_CLIENT_TIMEOUT, USER_AGENT, + USE_SLIM, ) from open_webui.retrieval.loaders.external_web import ExternalWebLoader from open_webui.retrieval.loaders.microsoft_web_iq import MicrosoftWebIQLoader @@ -128,7 +130,7 @@ def _assert_addresses_allowed(addresses: Sequence[str]) -> None: def validate_url(url: Union[str, Sequence[str]]): if isinstance(url, str): - if isinstance(validators.url(url), validators.ValidationError): + if isinstance(validators.url(url, simple_host=ENABLE_LOCAL_WEB_FETCH), validators.ValidationError): raise ValueError(ERROR_MESSAGES.INVALID_URL) # Reject parser-confusing chars: urlparse and requests/aiohttp split @@ -302,6 +304,9 @@ _DROPPED_REQUEST_HEADERS = {'accept-encoding', 'connection', 'content-length', ' # The clients hand us a decoded body, so the sender's framing no longer describes it. _DROPPED_RESPONSE_HEADERS = {'connection', 'content-encoding', 'content-length', 'transfer-encoding'} +# The Playwright loader only reads the page HTML, which none of these feed. +_DROPPED_RESOURCE_TYPES = {'font', 'image', 'media'} + def _forwardable_request_headers(headers: Dict[str, str]) -> Dict[str, str]: return {name: value for name, value in headers.items() if name.lower() not in _DROPPED_REQUEST_HEADERS} @@ -643,7 +648,7 @@ class SafeMicrosoftWebIQLoader(BaseLoader, RateLimitMixin, URLProcessingMixin): raise e -class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessingMixin): +class SafePlaywrightURLLoader(BaseLoader, RateLimitMixin, URLProcessingMixin): """Load HTML pages safely with Playwright, supporting SSL verification, rate limiting, and remote browser connection. Attributes: @@ -674,6 +679,13 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing playwright_timeout: Optional[int] = 10000, ): """Initialize with additional safety parameters and remote browser support.""" + if USE_SLIM: + raise HTTPException( + 503, 'Playwright is unavailable in slim. Use basic HTTP fetching or an external web loader.' + ) + + for package in ('playwright', 'unstructured'): + import_module(package) proxy_server = proxy.get('server') if proxy else None if trust_env and not proxy_server: @@ -685,14 +697,11 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing else: proxy = {'server': env_proxy_server} - # We'll set headless to False if using playwright_ws_url since it's handled by the remote browser - super().__init__( - urls=web_paths, - continue_on_failure=continue_on_failure, - headless=headless if playwright_ws_url is None else False, - remove_selectors=remove_selectors, - proxy=proxy, - ) + self.urls = web_paths + self.continue_on_failure = continue_on_failure + self.headless = headless if playwright_ws_url is None else False + self.remove_selectors = remove_selectors or [] + self.proxy = proxy self.verify_ssl = verify_ssl self.requests_per_second = requests_per_second self.last_request_time = None @@ -700,6 +709,12 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing self.trust_env = trust_env self.playwright_timeout = playwright_timeout + @staticmethod + def _extract_html(html): + from unstructured.partition.html import partition_html + + return '\n\n'.join(str(element) for element in partition_html(text=html)) + def _request_timeout(self) -> float: # per-hop budget, since page.goto's timeout cannot reach into our own fetch and 0 disables # it. aiohttp treats it as a total where requests only caps each read, so sync runs looser. @@ -719,6 +734,9 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing def _intercept_navigation_sync(self, route, session): req = route.request + if req.resource_type in _DROPPED_RESOURCE_TYPES: + route.abort() + return hop_cookies: List[Tuple[str, str]] = [] @@ -773,6 +791,9 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing async def _intercept_navigation(self, route, session): req = route.request + if req.resource_type in _DROPPED_RESOURCE_TYPES: + await route.abort() + return hop_cookies: List[Tuple[str, str]] = [] @@ -847,12 +868,18 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing browser.new_page(service_workers='block') as page, ): page.route('**/*', lambda route: self._intercept_navigation_sync(route, session)) - page.route_web_socket('**/*', lambda ws_route: ws_route.close()) + # sync close() hangs the dispatcher; a no-op handler still never connects to the server + page.route_web_socket('**/*', lambda ws_route: None) response = page.goto(url, timeout=self.playwright_timeout) if response is None: raise ValueError(f'page.goto() returned None for url {url}') - text = self.evaluator.evaluate(page, browser, response) + for selector in self.remove_selectors: + for element in page.locator(selector).all(): + if element.is_visible(): + element.evaluate('element => element.remove()') + text = self._extract_html(page.content()) + page.unroute_all(behavior='ignoreErrors') metadata = {'source': url} yield Document(page_content=text, metadata=metadata) except Exception as e: @@ -887,7 +914,12 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing if response is None: raise ValueError(f'page.goto() returned None for url {url}') - text = await self.evaluator.evaluate_async(page, browser, response) + for selector in self.remove_selectors: + for element in await page.locator(selector).all(): + if await element.is_visible(): + await element.evaluate('element => element.remove()') + text = await asyncio.to_thread(self._extract_html, await page.content()) + await page.unroute_all(behavior='ignoreErrors') metadata = {'source': url} yield Document(page_content=text, metadata=metadata) except Exception as e: @@ -897,42 +929,58 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing raise e -class SafeWebBaseLoader(WebBaseLoader): - """WebBaseLoader with enhanced error handling for URLs.""" +class SafeWebBaseLoader(BaseLoader): + """Fetch pages with connect-time address checks and bounded concurrency.""" - def __init__(self, trust_env: bool = False, *args, **kwargs): - """Initialize SafeWebBaseLoader - Args: - trust_env (bool, optional): set to True if using proxy to make web requests, for example - using http(s)_proxy environment variables. Defaults to False. - """ - # lxml parses scraped pages far faster than the html.parser default - kwargs.setdefault('default_parser', 'lxml') - super().__init__(*args, **kwargs) + def __init__( + self, + web_paths, + verify_ssl=True, + trust_env=False, + requests_per_second=2, + continue_on_failure=False, + requests_kwargs=None, + raise_for_status=False, + default_parser='lxml', + bs_kwargs=None, + bs_get_text_kwargs=None, + ): + self.web_paths = list(web_paths) self.trust_env = trust_env - - # Propagate USER_AGENT env var so that both the sync _scrape() and - # async _fetch() paths present a real UA instead of python-requests/2.x - # which gets blocked by Cloudflare, Wikipedia, and similar bot-detection. - # _fetch() forwards self.session.headers to the aiohttp session, so - # setting it here covers both code-paths. - if USER_AGENT: - self.session.headers['User-Agent'] = USER_AGENT - - # Prevent redirect-based SSRF on the synchronous _scrape() path. - # validate_url() is called once on the originally-submitted URL, but the - # parent WebBaseLoader's _scrape() invokes self.session.get(url, **self.requests_kwargs) - # which by default follows redirects. Without the override below, an attacker - # can submit a public URL that 302-redirects to an internal address (RFC1918, - # 127.0.0.1, 169.254.169.254, etc.) and the redirected target is fetched without - # re-validation. Matches the policy enforced on the async _fetch() path below. - self.requests_kwargs = { - **(self.requests_kwargs or {}), - 'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS, + self.requests_per_second = requests_per_second + self.continue_on_failure = continue_on_failure + self.requests_kwargs = {**(requests_kwargs or {}), 'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS} + self.raise_for_status = raise_for_status + self.default_parser = default_parser + self.bs_kwargs = bs_kwargs or {} + self.bs_get_text_kwargs = bs_get_text_kwargs or {} + # Preserve the synchronous loader's environment-proxy behavior. + self.session = get_ssrf_safe_requests_session() + self.session.verify = verify_ssl + self.session.headers = { + 'User-Agent': USER_AGENT or 'DefaultLangchainUserAgent', + 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', + 'Accept-Language': 'en-US,en;q=0.5', + 'Referer': 'https://www.google.com/', + 'DNT': '1', + 'Connection': 'keep-alive', + 'Upgrade-Insecure-Requests': '1', } - self.session.mount('http://', _SSRFSafeAdapter()) - self.session.mount('https://', _SSRFSafeAdapter()) + async def fetch_all(self, urls): + semaphore = asyncio.Semaphore(self.requests_per_second) + + async def fetch(url): + async with semaphore: + try: + return await self._fetch(url) + except Exception as e: + if not self.continue_on_failure: + raise + log.warning('Error fetching %s: %s', url, e) + return '' + + return await asyncio.gather(*(fetch(url) for url in urls)) async def _fetch(self, url: str, retries: int = 3, cooldown: int = 2, backoff: float = 1.5) -> str: connector = _SSRFSafeConnector() @@ -948,10 +996,10 @@ class SafeWebBaseLoader(WebBaseLoader): else: kwargs['ssl'] = AIOHTTP_CLIENT_SESSION_SSL - async with session.get( - url, - **(self.requests_kwargs | kwargs), - ) as response: + options = self.requests_kwargs | kwargs + if isinstance(options.get('timeout'), (int, float)): + options['timeout'] = aiohttp.ClientTimeout(total=options['timeout']) + async with session.get(url, **options) as response: if self.raise_for_status: response.raise_for_status() return await response.text() @@ -963,38 +1011,24 @@ class SafeWebBaseLoader(WebBaseLoader): await asyncio.sleep(cooldown * backoff**i) raise ValueError('retry count exceeded') - def _unpack_fetch_results(self, results: Any, urls: List[str], parser: Union[str, None] = None) -> List[Any]: - """Unpack fetch results into BeautifulSoup objects.""" - from bs4 import BeautifulSoup - - final_results = [] - for i, result in enumerate(results): - url = urls[i] - url_parser = parser - if url_parser is None: - url_parser = 'xml' if url.endswith('.xml') else self.default_parser - self._check_parser(url_parser) - final_results.append(BeautifulSoup(result, url_parser, **self.bs_kwargs)) - return final_results - def lazy_load(self) -> Iterator[Document]: """Lazy load text from the url(s) in web_path with error handling.""" for path in self.web_paths: try: - soup = self._scrape(path, bs_kwargs=self.bs_kwargs) - text = soup.get_text(**self.bs_get_text_kwargs) - - # Build metadata - metadata = extract_metadata(soup, path) - - yield Document(page_content=text, metadata=metadata) + with self.session.get(path, **self.requests_kwargs) as response: + if self.raise_for_status: + response.raise_for_status() + response.encoding = response.apparent_encoding + yield self._document_from_html(response.text, path) except Exception as e: - # Log the error and continue with the next URL log.exception(f'Error loading {path}: {e}') def _document_from_html(self, html: str, url: str) -> Document: """Build one Document.""" - soup = self._unpack_fetch_results([html], [url])[0] + from bs4 import BeautifulSoup + + parser = 'xml' if url.endswith('.xml') else self.default_parser + soup = BeautifulSoup(html, parser, **self.bs_kwargs) return Document( page_content=soup.get_text(**self.bs_get_text_kwargs), metadata=extract_metadata(soup, url), diff --git a/backend/open_webui/routers/audio.py b/backend/open_webui/routers/audio.py index 9fa5d89304..b9ab84ea0e 100644 --- a/backend/open_webui/routers/audio.py +++ b/backend/open_webui/routers/audio.py @@ -46,6 +46,7 @@ from open_webui.env import ( DEVICE_TYPE, ENABLE_FORWARD_USER_INFO_HEADERS, ENV, + USE_SLIM, ) from open_webui.events import EVENTS, publish_event from open_webui.models.config import Config @@ -58,9 +59,10 @@ from open_webui.utils.session_pool import get_session from pydantic import BaseModel # pydub needs stdlib audioop (gone in 3.13); keep requires-python capped < 3.13 -from pydub import AudioSegment -from pydub.silence import split_on_silence -from pydub.utils import mediainfo +if not USE_SLIM: + from pydub import AudioSegment + from pydub.silence import split_on_silence + from pydub.utils import mediainfo log = logging.getLogger(__name__) router = APIRouter() @@ -213,6 +215,8 @@ def transcode_audio_to_mp3(audio_data: bytes, content_type_header: str, output_p def set_faster_whisper_model(model: str, auto_update: bool = False): + if USE_SLIM: + raise HTTPException(503, 'Configure an external speech-to-text engine. Local Whisper is unavailable in slim.') whisper_model = None if model: from faster_whisper import WhisperModel @@ -285,14 +289,20 @@ async def get_audio_config(request: Request, user=Depends(get_admin_user)): @router.post('/config/update') async def update_audio_config(request: Request, form_data: AudioConfigUpdateForm, user=Depends(get_admin_user)): + if USE_SLIM: + current = await Config.get_many('audio.stt.engine', 'audio.tts.engine') + if form_data.stt.ENGINE == '' and current.get('audio.stt.engine') != '': + raise HTTPException(400, 'Local Whisper is unavailable in slim. Select an external speech-to-text engine.') + if form_data.tts.ENGINE == 'transformers' and current.get('audio.tts.engine') != 'transformers': + raise HTTPException(400, 'Local TTS is unavailable in slim. Select an external text-to-speech engine.') await Config.upsert( { - **config_updates(form_data.tts.model_dump(), TTS_CONFIG_KEYS), - **config_updates(form_data.stt.model_dump(), STT_CONFIG_KEYS), + **config_updates(form_data.tts.model_dump(exclude_unset=True), TTS_CONFIG_KEYS), + **config_updates(form_data.stt.model_dump(exclude_unset=True), STT_CONFIG_KEYS), } ) - if form_data.stt.ENGINE == '': + if form_data.stt.ENGINE == '' and not USE_SLIM: request.app.state.faster_whisper_model = await asyncio.to_thread( set_faster_whisper_model, form_data.stt.WHISPER_MODEL, WHISPER_MODEL_AUTO_UPDATE ) @@ -314,6 +324,8 @@ async def update_audio_config(request: Request, form_data: AudioConfigUpdateForm def load_speech_pipeline(request): + if USE_SLIM: + raise HTTPException(503, 'Configure an external text-to-speech engine. Local TTS is unavailable in slim.') from datasets import load_dataset from transformers import pipeline @@ -328,7 +340,9 @@ def load_speech_pipeline(request): async def _raise_tts_error(exc: Exception, r=None) -> None: """Raise a standardised HTTPException from a TTS provider failure.""" - code = r.status if r is not None else 500 + if isinstance(exc, HTTPException): + raise exc + code = r.status if r is not None and r.status >= 400 else 500 # LICENSE covers this Open WebUI error identifier. # Do not alter, remove, obscure, or replace it except as LICENSE permits: # https://docs.openwebui.com/license. @@ -351,8 +365,29 @@ async def _write_tts_cache( audio: bytes, body_path: Path, payload: dict, + content_type: str = 'audio/mpeg', ) -> None: """Persist audio + request metadata to the speech cache.""" + if USE_SLIM: + mime_type = content_type.split(';')[0].strip().lower() + if mime_type not in { + 'audio/mpeg', + 'audio/mp3', + 'audio/wav', + 'audio/x-wav', + 'audio/ogg', + 'audio/opus', + 'audio/webm', + 'audio/flac', + 'audio/aac', + 'audio/mp4', + }: + raise HTTPException( + 502, + f'TTS returned unsupported format {mime_type}. Configure the provider to return MP3, WAV, Ogg, or another browser-playable audio format.', + ) + async with aiofiles.open(file_path.with_suffix('.mime'), 'w') as f: + await f.write(content_type) async with aiofiles.open(file_path, 'wb') as f: await f.write(audio) async with aiofiles.open(body_path, 'w') as f: @@ -389,6 +424,10 @@ async def _tts_openai(request, payload, file_path, file_body_path, user): audio_data = await r.read() content_type = r.headers.get('Content-Type', 'audio/mpeg') + if USE_SLIM: + await _write_tts_cache(file_path, audio_data, file_body_path, payload, content_type) + return FileResponse(file_path, media_type=content_type) + if not await asyncio.to_thread(transcode_audio_to_mp3, audio_data, content_type, file_path): async with aiofiles.open(file_path, 'wb') as f: await f.write(audio_data) @@ -430,8 +469,9 @@ async def _tts_elevenlabs(request, payload, file_path, file_body_path, user): ssl=AIOHTTP_CLIENT_SESSION_SSL, ) as r: r.raise_for_status() - await _write_tts_cache(file_path, await r.read(), file_body_path, payload) - return FileResponse(file_path) + content_type = r.headers.get('Content-Type', 'audio/mpeg') + await _write_tts_cache(file_path, await r.read(), file_body_path, payload, content_type) + return FileResponse(file_path, media_type=content_type if USE_SLIM else None) except Exception as exc: log.exception(exc) await _raise_tts_error(exc, r) @@ -465,8 +505,9 @@ async def _tts_azure(request, payload, file_path, file_body_path, user): ssl=AIOHTTP_CLIENT_SESSION_SSL, ) as r: r.raise_for_status() - await _write_tts_cache(file_path, await r.read(), file_body_path, payload) - return FileResponse(file_path) + content_type = r.headers.get('Content-Type', 'audio/mpeg') + await _write_tts_cache(file_path, await r.read(), file_body_path, payload, content_type) + return FileResponse(file_path, media_type=content_type if USE_SLIM else None) except Exception as exc: log.exception(exc) await _raise_tts_error(exc, r) @@ -474,6 +515,8 @@ async def _tts_azure(request, payload, file_path, file_body_path, user): async def _tts_transformers(request, payload, file_path, file_body_path, user): """Generate speech via the local HuggingFace SpeechT5 pipeline (thread-offloaded).""" + if USE_SLIM: + raise HTTPException(503, 'Configure an external text-to-speech engine. Local TTS is unavailable in slim.') import soundfile as sf import torch @@ -558,6 +601,8 @@ _TTS_ENGINES = { @router.post('/speech') async def speech(request: Request, user=Depends(get_verified_user)): engine = await Config.get('audio.tts.engine') + if USE_SLIM and engine in ('', 'transformers'): + raise HTTPException(503, 'Configure an external text-to-speech engine.') if engine == '': raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -572,7 +617,10 @@ async def speech(request: Request, user=Depends(get_verified_user)): body = await request.body() name = hashlib.sha256( - body + str(engine).encode('utf-8') + str(await Config.get('audio.tts.model')).encode('utf-8') + body + + str(engine).encode('utf-8') + + str(await Config.get('audio.tts.model')).encode('utf-8') + + (b':slim' if USE_SLIM else b'') ).hexdigest() file_path = SPEECH_CACHE_DIR.joinpath(f'{name}.mp3') @@ -587,7 +635,11 @@ async def speech(request: Request, user=Depends(get_verified_user)): subject_id=name, data={'engine': engine, 'cached': True}, ) - return FileResponse(file_path) + content_type = None + if USE_SLIM: + async with aiofiles.open(file_path.with_suffix('.mime')) as f: + content_type = await f.read() + return FileResponse(file_path, media_type=content_type) try: payload = JSONCodec.loads(body) @@ -960,7 +1012,11 @@ async def _transcribe_mistral(request, file_path, filename, metadata, file_dir, session = await get_session() if use_chat_completions: audio_file_to_use = file_path - if is_audio_conversion_required(file_path): + if USE_SLIM and Path(filename).suffix.lower() not in ('.mp3', '.wav'): + raise HTTPException( + 400, 'Mistral chat transcription requires MP3 or WAV in slim; local conversion is unavailable.' + ) + if not BYPASS_PYDUB_PREPROCESSING and is_audio_conversion_required(file_path): log.debug('Converting audio to mp3 for chat completions API') converted_path = await asyncio.to_thread(convert_audio_to_mp3, file_path) if converted_path: diff --git a/backend/open_webui/routers/auths.py b/backend/open_webui/routers/auths.py index e0085b73fd..16ae0422e8 100644 --- a/backend/open_webui/routers/auths.py +++ b/backend/open_webui/routers/auths.py @@ -77,8 +77,7 @@ from open_webui.utils.auth import ( from open_webui.utils.groups import apply_default_group_assignment from open_webui.utils.misc import parse_duration, validate_email_format from open_webui.utils.rate_limit import RateLimiter -from open_webui.utils.redis import get_redis_client -from pydantic import BaseModel +from pydantic import BaseModel, StrictStr, field_validator from sqlalchemy.exc import IntegrityError from sqlalchemy.ext.asyncio import AsyncSession @@ -88,12 +87,11 @@ log = logging.getLogger(__name__) # Forgive us our failed attempts, as we forgive those # who exceed their allotted rate against this gate. -signin_rate_limiter = RateLimiter(redis_client=get_redis_client(), limit=5 * 3, window=60 * 3) +signin_rate_limiter = RateLimiter(limit=5 * 3, window=60 * 3) # Best-effort throttle only: there is no caller identity before the provider answers, # and deployments may derive request.client from proxy headers. token_exchange_rate_limiter = ( RateLimiter( - redis_client=get_redis_client(), limit=OAUTH_TOKEN_EXCHANGE_RATE_LIMIT, window=OAUTH_TOKEN_EXCHANGE_RATE_LIMIT_WINDOW, ) @@ -106,6 +104,7 @@ ADMIN_CONFIG_KEYS = { 'SHOW_ADMIN_DETAILS': 'auth.admin.show', 'ADMIN_EMAIL': 'auth.admin.email', 'WEBUI_URL': 'webui.url', + 'ENABLE_LOGIN_FORM': 'ui.enable_login_form', 'ENABLE_SIGNUP': 'ui.enable_signup', 'ENABLE_API_KEYS': 'auth.enable_api_keys', 'ENABLE_API_KEYS_ENDPOINT_RESTRICTIONS': 'auth.api_key.endpoint_restrictions', @@ -113,6 +112,7 @@ ADMIN_CONFIG_KEYS = { 'DEFAULT_USER_ROLE': 'ui.default_user_role', 'DEFAULT_GROUP_ID': 'ui.default_group_id', 'DEFAULT_INTERFACE_SETTINGS': 'ui.default_interface_settings', + 'I18N': 'ui.i18n', 'JWT_EXPIRES_IN': 'auth.jwt_expiry', 'ENABLE_COMMUNITY_SHARING': 'ui.enable_community_sharing', 'ENABLE_MESSAGE_RATING': 'ui.enable_message_rating', @@ -814,7 +814,7 @@ async def signin( db=db, ) else: - if signin_rate_limiter.is_limited(form_data.email.lower()): + if await signin_rate_limiter.is_limited(request.app.state.redis, form_data.email.lower()): raise HTTPException( status_code=status.HTTP_429_TOO_MANY_REQUESTS, detail=ERROR_MESSAGES.RATE_LIMIT_EXCEEDED, @@ -1209,6 +1209,7 @@ class AdminConfig(BaseModel): SHOW_ADMIN_DETAILS: bool ADMIN_EMAIL: str | None = None WEBUI_URL: str + ENABLE_LOGIN_FORM: bool = True ENABLE_SIGNUP: bool ENABLE_API_KEYS: bool ENABLE_API_KEYS_ENDPOINT_RESTRICTIONS: bool @@ -1216,6 +1217,7 @@ class AdminConfig(BaseModel): DEFAULT_USER_ROLE: str DEFAULT_GROUP_ID: str DEFAULT_INTERFACE_SETTINGS: dict | None = None + I18N: dict[str, dict[str, StrictStr]] | None = None JWT_EXPIRES_IN: str ENABLE_COMMUNITY_SHARING: bool ENABLE_MESSAGE_RATING: bool @@ -1236,10 +1238,40 @@ class AdminConfig(BaseModel): PENDING_USER_OVERLAY_CONTENT: str | None = None RESPONSE_WATERMARK: str | None = None + @field_validator('I18N') + @classmethod + def validate_i18n(cls, value): + if value is None: + raise ValueError('I18N must be a dictionary') + unsafe_keys = {'__proto__', 'prototype', 'constructor'} + + def placeholders(text): + return {match.strip() for match in re.findall(r'\{\{\s*-?\s*([^},]+)(?:,[^}]+)?\s*\}\}', text)} + + cleaned = {} + for locale, entries in value.items(): + if not locale.strip() or locale in unsafe_keys: + raise ValueError(f'Invalid language: {locale}') + translations = {} + for key, text in entries.items(): + if not key.strip() or key in unsafe_keys: + raise ValueError(f'Invalid translation key: {key}') + if text.strip(): + if placeholders(key) != placeholders(text): + raise ValueError(f'Interpolation placeholders do not match: {locale}: {key}') + translations[key] = text + if translations: + cleaned[locale] = translations + return cleaned + @router.post('/admin/config') async def update_admin_config(request: Request, form_data: AdminConfig, user=Depends(get_admin_user)): updates = config_updates(form_data.model_dump(), ADMIN_CONFIG_KEYS) + if 'ENABLE_LOGIN_FORM' not in form_data.model_fields_set: + updates.pop('ui.enable_login_form', None) + if 'I18N' not in form_data.model_fields_set: + updates.pop('ui.i18n', None) updates['ui.default_interface_settings'] = form_data.DEFAULT_INTERFACE_SETTINGS or {} updates['folders.max_file_count'] = int(form_data.FOLDER_MAX_FILE_COUNT) if form_data.FOLDER_MAX_FILE_COUNT else '' updates['automations.max_count'] = int(form_data.AUTOMATION_MAX_COUNT) if form_data.AUTOMATION_MAX_COUNT else '' @@ -1603,8 +1635,8 @@ async def token_exchange( detail='Token exchange is disabled', ) - if token_exchange_rate_limiter and token_exchange_rate_limiter.is_limited( - request.client.host if request.client else 'unknown' + if token_exchange_rate_limiter and await token_exchange_rate_limiter.is_limited( + request.app.state.redis, request.client.host if request.client else 'unknown' ): raise HTTPException( status_code=status.HTTP_429_TOO_MANY_REQUESTS, @@ -1720,6 +1752,7 @@ async def token_exchange( user=user, user_data=user_data, provider=provider, + access_token=form_data.token, db=db, ) if await Config.get('oauth.enable_group_mapping'): diff --git a/backend/open_webui/routers/automations.py b/backend/open_webui/routers/automations.py index caf8879e28..4b44a8b8ed 100644 --- a/backend/open_webui/routers/automations.py +++ b/backend/open_webui/routers/automations.py @@ -87,7 +87,7 @@ async def check_automation_limits(request, user, rrule_str: str, db, is_create: if min_interval: min_interval = int(min_interval) if min_interval > 0: - interval = rrule_interval_seconds(rrule_str) + interval = await rrule_interval_seconds(rrule_str) if interval is not None and interval < min_interval: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -150,7 +150,7 @@ async def enrich_automation(automation: AutomationModel, db: AsyncSession, tz: s return AutomationResponse( **automation.model_dump(), last_run=last_run, - next_runs=next_n_runs_ns(automation.data['rrule'], tz=tz), + next_runs=await next_n_runs_ns(automation.data['rrule'], tz=tz), ) @@ -216,7 +216,7 @@ async def create_new_automation( await check_automation_folder_access(form_data.folder_id, user, db) await check_automation_channel_access(form_data, user, db) try: - validate_rrule(form_data.data.rrule, tz=user.timezone) + await validate_rrule(form_data.data.rrule, tz=user.timezone) except ValueError as e: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -226,7 +226,7 @@ async def create_new_automation( await check_automation_limits(request, user, form_data.data.rrule, db, is_create=True) tz = user.timezone - automation = await Automations.insert(user.id, form_data, next_run_ns(form_data.data.rrule, tz=tz), db=db) + automation = await Automations.insert(user.id, form_data, await next_run_ns(form_data.data.rrule, tz=tz), db=db) response = await enrich_automation(automation, db, tz=tz) await publish_event( request, @@ -276,7 +276,7 @@ async def update_automation_by_id( await check_automation_channel_access(form_data, user, db) try: - validate_rrule(form_data.data.rrule, tz=user.timezone) + await validate_rrule(form_data.data.rrule, tz=user.timezone) except ValueError as e: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -286,7 +286,7 @@ async def update_automation_by_id( await check_automation_limits(request, user, form_data.data.rrule, db, is_create=False) tz = user.timezone - updated = await Automations.update_by_id(id, form_data, next_run_ns(form_data.data.rrule, tz=tz), db=db) + updated = await Automations.update_by_id(id, form_data, await next_run_ns(form_data.data.rrule, tz=tz), db=db) response = await enrich_automation(updated, db, tz=tz) await publish_event( request, @@ -313,7 +313,7 @@ async def toggle_automation_by_id( await check_automations_permission(request, user) automation = await Automations.get_by_id(id, db=db) check_automation_access(automation, user) - toggled = await Automations.toggle(id, next_run_ns(automation.data['rrule'], tz=user.timezone), db=db) + toggled = await Automations.toggle(id, await next_run_ns(automation.data['rrule'], tz=user.timezone), db=db) response = await enrich_automation(toggled, db, tz=user.timezone) await publish_event( request, diff --git a/backend/open_webui/routers/calendar.py b/backend/open_webui/routers/calendar.py index f95be48e84..86d9824803 100644 --- a/backend/open_webui/routers/calendar.py +++ b/backend/open_webui/routers/calendar.py @@ -273,7 +273,10 @@ async def get_events( async def create_event(request: Request, form_data: CalendarEventForm, user: UserModel = Depends(get_verified_user)): await check_calendar_permission(request, user) await _check_calendar_access(form_data.calendar_id, user, 'write') - event = await CalendarEvents.insert_new_event(user.id, form_data) + try: + event = await CalendarEvents.insert_new_event(user.id, form_data) + except ValueError as e: + raise HTTPException(status_code=422, detail=str(e)) from e await publish_event( request, EVENTS.CALENDAR_EVENT_CREATED, @@ -325,7 +328,10 @@ async def update_event( if form_data.calendar_id is not None and form_data.calendar_id != event.calendar_id: await _check_calendar_access(form_data.calendar_id, user, 'write') - updated = await CalendarEvents.update_event_by_id(event_id, form_data) + try: + updated = await CalendarEvents.update_event_by_id(event_id, form_data) + except ValueError as e: + raise HTTPException(status_code=422, detail=str(e)) from e if not updated: raise HTTPException(status_code=500, detail='Failed to update') await publish_event( diff --git a/backend/open_webui/routers/channels.py b/backend/open_webui/routers/channels.py index 3c165f16ff..de25739c92 100644 --- a/backend/open_webui/routers/channels.py +++ b/backend/open_webui/routers/channels.py @@ -8,7 +8,7 @@ from fastapi.responses import FileResponse, Response, StreamingResponse from open_webui.config import ENABLE_ADMIN_CHAT_ACCESS, ENABLE_ADMIN_EXPORT from open_webui.constants import ERROR_MESSAGES from open_webui.events import EVENTS, publish_event -from open_webui.env import STATIC_DIR +from open_webui.env import ENABLE_PROFILE_IMAGE_URL_FORWARDING, STATIC_DIR from open_webui.internal.db import get_async_session from open_webui.models.access_grants import AccessGrants, has_public_read_access_grant, has_public_write_access_grant from open_webui.models.config import Config @@ -1102,13 +1102,12 @@ async def model_response_handler(request, channel, message, user, db=None): # Resolve model config (same path automations use) from open_webui.utils.automations import _resolve_model_defaults - tool_ids, features, filter_ids, _ = await _resolve_model_defaults(request.app, model_id) - # Build full form_data — same shape as frontend POST. # The channel: prefix routes pipeline events to the # channel emitter in socket/main.py instead of the # default chat emitter. form_data = { + **await _resolve_model_defaults(request.app, model_id), 'model': model_id, 'messages': [ system_message, @@ -1122,12 +1121,6 @@ async def model_response_handler(request, channel, message, user, db=None): } if files: form_data['files'] = files - if tool_ids: - form_data['tool_ids'] = tool_ids - if features: - form_data['features'] = features - if filter_ids: - form_data['filter_ids'] = filter_ids # Call the full chat completion pipeline — streaming, # tools, filters, RAG — everything. The pipeline runs as @@ -1823,9 +1816,15 @@ async def delete_message_by_id( @router.get('/webhooks/{webhook_id}/profile/image') -async def get_webhook_profile_image(webhook_id: str, user=Depends(get_verified_user)): +async def get_webhook_profile_image( + request: Request, + webhook_id: str, + user=Depends(get_verified_user), + db: AsyncSession = Depends(get_async_session), +): """Get webhook profile image by webhook ID.""" - webhook = await Channels.get_webhook_by_id(webhook_id) + await check_channels_access(request, user) + webhook = await Channels.get_webhook_by_id(webhook_id, db=db) if not webhook: # Return default favicon if webhook not found # LICENSE covers this Open WebUI fallback logo. @@ -1833,13 +1832,26 @@ async def get_webhook_profile_image(webhook_id: str, user=Depends(get_verified_u # https://docs.openwebui.com/license. return FileResponse(f'{STATIC_DIR}/favicon.png') + channel = await Channels.get_channel_by_id(webhook.channel_id, db=db) + if not channel: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=ERROR_MESSAGES.NOT_FOUND) + + if channel.type in ['group', 'dm']: + if not await Channels.is_user_channel_member(channel.id, user.id, db=db): + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=ERROR_MESSAGES.DEFAULT()) + else: + if user.role != 'admin' and not await channel_has_access(user.id, channel, permission='read', db=db): + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=ERROR_MESSAGES.DEFAULT()) + if webhook.profile_image_url: # Check if it's url or base64 if webhook.profile_image_url.startswith('http'): - return Response( - status_code=status.HTTP_302_FOUND, - headers={'Location': webhook.profile_image_url}, - ) + if ENABLE_PROFILE_IMAGE_URL_FORWARDING: + return Response( + status_code=status.HTTP_302_FOUND, + headers={'Location': webhook.profile_image_url}, + ) + # When forwarding is disabled, fall through to the default image to prevent client-side IP/UA/Referer leaks. elif webhook.profile_image_url.startswith('data:image'): try: header, base64_data = webhook.profile_image_url.split(',', 1) diff --git a/backend/open_webui/routers/chats.py b/backend/open_webui/routers/chats.py index 61019662de..ed339a601a 100644 --- a/backend/open_webui/routers/chats.py +++ b/backend/open_webui/routers/chats.py @@ -888,6 +888,7 @@ async def search_user_chats( created_at=chat.created_at, last_read_at=chat.last_read_at, snippet=chat_search_snippet(chat.chat, search_text), + archived=chat.archived, ) ) @@ -1592,7 +1593,7 @@ async def delete_chat_by_id( # Cancel any in-flight LLM tasks (streaming, title/tags generation) before # deleting the chat to prevent orphaned requests. await stop_item_tasks(request.app.state.redis, id) - await Chats.delete_orphan_tags_for_user(chat.meta.get('tags', []), user.id, threshold=1, db=db) + await Chats.delete_orphan_tags_for_user(chat.meta.get('tags', []), chat.user_id, threshold=1, db=db) # Cascade to internal child chats spawned from this one. for child_id in await Chats.get_internal_chat_ids_by_parent_id(id, chat.user_id): @@ -1690,15 +1691,6 @@ async def fork_chat_by_id( history = (chat.chat or {}).get('history') or {} messages_map = await Chats.get_messages_map_by_chat_id(id) or history.get('messages') or {} - if any( - message.get('role') == 'assistant' and message.get('done') is False - for message in messages_map.values() - if isinstance(message, dict) - ): - raise HTTPException( - status_code=status.HTTP_409_CONFLICT, - detail='Wait for the current response to finish before forking.', - ) source_message_id = ( (form_data.message_id if form_data else None) or chat.current_message_id or history.get('currentId') @@ -1715,6 +1707,22 @@ async def fork_chat_by_id( detail=detail, ) from exc + # An unfinished message is stale unless it is awaiting tool approval + for message in fork_history['messages'].values(): + if message.get('role') != 'assistant' or message.get('done') is not False: + continue + + output = message.get('output') + if isinstance(output, list) and any( + isinstance(item, dict) + and item.get('type') == 'function_call' + and item.get('status') in {'pending', 'queued', 'requires_approval'} + for item in output + ): + continue + + message['done'] = True + updated_chat = {**(chat.chat or {})} updated_chat.pop('currentId', None) updated_chat.update( @@ -1732,10 +1740,15 @@ async def fork_chat_by_id( 'forked_from_message_id': source_message_id, } + # The source chat's folder may no longer be writable by the caller. + folder_id = chat.folder_id + if folder_id is not None and not await has_folder_write_access(user.id, folder_id, db=db): + folder_id = None + fork = await Chats.insert_new_chat( str(uuid4()), user.id, - ChatForm(chat=updated_chat, folder_id=chat.folder_id), + ChatForm(chat=updated_chat, folder_id=folder_id), db=db, internal_meta=meta, ) diff --git a/backend/open_webui/routers/configs.py b/backend/open_webui/routers/configs.py index 3e37f05eef..38227dd2fc 100644 --- a/backend/open_webui/routers/configs.py +++ b/backend/open_webui/routers/configs.py @@ -1,8 +1,7 @@ from __future__ import annotations -import copy import logging -from typing import Optional +from typing import Any, Optional import aiohttp from fastapi import APIRouter, Depends, HTTPException, Request @@ -40,6 +39,7 @@ log = logging.getLogger(__name__) CONNECTIONS_CONFIG_KEYS = { 'ENABLE_DIRECT_CONNECTIONS': 'direct.enable', + 'ENABLE_DIRECT_INTEGRATIONS': 'direct.integrations.enable', 'ENABLE_BASE_MODELS_CACHE': 'models.base_models_cache', } CODE_EXECUTION_CONFIG_KEYS = { @@ -132,6 +132,7 @@ async def get_config_namespace(namespace: str, user=Depends(get_admin_user)): class ConnectionsConfigForm(BaseModel): ENABLE_DIRECT_CONNECTIONS: bool + ENABLE_DIRECT_INTEGRATIONS: bool = False ENABLE_BASE_MODELS_CACHE: bool @@ -146,7 +147,7 @@ async def set_connections_config( form_data: ConnectionsConfigForm, user=Depends(get_admin_user), ): - await Config.upsert(config_updates(form_data.model_dump(), CONNECTIONS_CONFIG_KEYS)) + await Config.upsert(config_updates(form_data.model_dump(exclude_unset=True), CONNECTIONS_CONFIG_KEYS)) values = await get_config_values(CONNECTIONS_CONFIG_KEYS) await publish_event( request, @@ -218,6 +219,7 @@ class ToolServerConnection(BaseModel): path: str type: str | None = 'openapi' # openapi, mcp auth_type: str | None + forward_cookies: bool = False headers: dict | str | None = None key: str | None config: dict | None @@ -307,6 +309,7 @@ class TerminalServerConnection(BaseModel): key: str | None = '' auth_type: str | None = 'bearer' + forward_cookies: bool = False config: dict | None = None @@ -808,17 +811,24 @@ class PromptSuggestion(BaseModel): class SetDefaultSuggestionsForm(BaseModel): suggestions: list[PromptSuggestion] + i18n: dict[str, Any] | None = None -@router.post('/suggestions', response_model=list[PromptSuggestion]) +@router.post('/suggestions', response_model=dict) async def set_default_suggestions( request: Request, form_data: SetDefaultSuggestionsForm, user=Depends(get_admin_user), ): data = form_data.model_dump() - await Config.upsert({'ui.prompt_suggestions': data['suggestions']}) + await Config.upsert( + { + 'ui.prompt_suggestions': data['suggestions'], + 'ui.prompt_suggestions_i18n': data.get('i18n') or {}, + } + ) suggestions = await Config.get('ui.prompt_suggestions') + suggestions_i18n = await Config.get('ui.prompt_suggestions_i18n') await publish_event( request, EVENTS.CONFIG_SUGGESTIONS_UPDATED, @@ -827,7 +837,7 @@ async def set_default_suggestions( subject_type='config', data={'count': len(suggestions or [])}, ) - return suggestions + return {'suggestions': suggestions, 'i18n': suggestions_i18n} ############################ diff --git a/backend/open_webui/routers/evaluations.py b/backend/open_webui/routers/evaluations.py index 8a6bf71f4a..00c1517c88 100644 --- a/backend/open_webui/routers/evaluations.py +++ b/backend/open_webui/routers/evaluations.py @@ -4,7 +4,10 @@ from typing import Optional from fastapi import APIRouter, Depends, HTTPException, Request, status from fastapi.concurrency import run_in_threadpool from open_webui.constants import ERROR_MESSAGES +from open_webui.env import MPS_INFERENCE_LOCK from open_webui.events import EVENTS, publish_event +from open_webui.env import USE_SLIM +from open_webui.retrieval.utils import cosine_similarity from open_webui.internal.db import get_async_session from open_webui.models.config import Config from open_webui.models.feedbacks import ( @@ -68,6 +71,8 @@ _embedding_model = None def _get_embedding_model(): global _embedding_model + if USE_SLIM: + return None if _embedding_model is None: try: from sentence_transformers import SentenceTransformer @@ -179,8 +184,9 @@ def _compute_similarities(feedbacks: list[LeaderboardFeedbackData], query: str) return {} try: - tag_embeddings = embedding_model.encode(all_tags) - query_embedding = embedding_model.encode([query])[0] + with MPS_INFERENCE_LOCK: + tag_embeddings = embedding_model.encode(all_tags) + query_embedding = embedding_model.encode([query])[0] except Exception as e: log.error(f'Embedding error: {e}') return {} @@ -215,6 +221,7 @@ class LeaderboardResponse(BaseModel): @router.get('/leaderboard', response_model=LeaderboardResponse) async def get_leaderboard( + request: Request, query: Optional[str] = None, user=Depends(get_admin_user), db: AsyncSession = Depends(get_async_session), @@ -224,7 +231,17 @@ async def get_leaderboard( similarities = None if query and query.strip(): - similarities = await run_in_threadpool(_compute_similarities, feedbacks, query.strip()) + if USE_SLIM: + tags = list({tag for feedback in feedbacks for tag in (feedback.data or {}).get('tags', [])}) + embeddings = await request.app.state.EMBEDDING_FUNCTION([query.strip(), *tags], user=user) + scores = cosine_similarity(embeddings[0], embeddings[1:]) + tag_scores = dict(zip(tags, scores.tolist())) + similarities = { + feedback.id: max((tag_scores.get(tag, 0) for tag in (feedback.data or {}).get('tags', [])), default=0) + for feedback in feedbacks + } + else: + similarities = await run_in_threadpool(_compute_similarities, feedbacks, query.strip()) elo_stats = _calculate_elo(feedbacks, similarities) tags_by_model = _get_top_tags(feedbacks) diff --git a/backend/open_webui/routers/files.py b/backend/open_webui/routers/files.py index b349eeacd5..c2a429a3db 100644 --- a/backend/open_webui/routers/files.py +++ b/backend/open_webui/routers/files.py @@ -461,7 +461,11 @@ async def upload_file_handler( log.exception(e) raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, - detail=ERROR_MESSAGES.DEFAULT('Error uploading file'), + detail=( + ERROR_MESSAGES.EMPTY_CONTENT + if isinstance(e, ValueError) and e.args == (ERROR_MESSAGES.EMPTY_CONTENT,) + else ERROR_MESSAGES.DEFAULT('Error uploading file') + ), ) diff --git a/backend/open_webui/routers/folders.py b/backend/open_webui/routers/folders.py index 7bc8641d68..046a2d8834 100644 --- a/backend/open_webui/routers/folders.py +++ b/backend/open_webui/routers/folders.py @@ -401,7 +401,7 @@ async def update_folder_parent_id_by_id( form_data.parent_id, user.id, folder.name, db=db ) - if existing_folder: + if existing_folder and existing_folder.id != id: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, detail=ERROR_MESSAGES.DEFAULT('Folder already exists'), @@ -683,7 +683,7 @@ async def delete_folder_by_id( folder_owner_id = folder.user_id folder_ids = await Folders.get_folder_ids_by_id_and_user_id_in_subtree(id, folder_owner_id, db=db) - if await Chats.count_chats_by_folder_ids_and_user_id(folder_ids, folder_owner_id, db=db): + if delete_contents and await Chats.count_chats_by_folder_ids_and_user_id(folder_ids, folder_owner_id, db=db): chat_delete_permission = await has_permission( user.id, 'chat.delete', await Config.get('user.permissions'), db=db ) @@ -704,8 +704,8 @@ async def delete_folder_by_id( for folder_id in folder_ids: if delete_contents: await Chats.delete_chats_by_user_id_and_folder_id(folder_owner_id, folder_id, db=db) - else: - await Chats.move_chats_by_user_id_and_folder_id(folder_owner_id, folder_id, None, db=db) + + await Chats.move_chats_by_folder_id(folder_id, None, db=db) # Clean up access grants for this folder await AccessGrants.revoke_all_access('folder', folder_id, db=db) diff --git a/backend/open_webui/routers/functions.py b/backend/open_webui/routers/functions.py index be3fe3edf1..212a419500 100644 --- a/backend/open_webui/routers/functions.py +++ b/backend/open_webui/routers/functions.py @@ -23,6 +23,7 @@ from open_webui.models.functions import ( ) from open_webui.utils.auth import get_admin_user, get_verified_user from open_webui.utils.plugin import ( + get_function_contents_cache, get_functions_cache, get_function_module_from_cache, load_function_module_by_id, @@ -440,6 +441,8 @@ async def delete_function_by_id( if result: FUNCTIONS = get_functions_cache(request) FUNCTIONS.pop(id, None) + FUNCTION_CONTENTS = get_function_contents_cache(request) + FUNCTION_CONTENTS.pop(id, None) await publish_event( request, EVENTS.FUNCTION_DELETED, diff --git a/backend/open_webui/routers/images.py b/backend/open_webui/routers/images.py index f08705546b..8c6e772896 100644 --- a/backend/open_webui/routers/images.py +++ b/backend/open_webui/routers/images.py @@ -329,38 +329,34 @@ def get_automatic1111_api_auth(image_config): return f'Basic {auth1111_base64_encoded_string}' -@router.get('/config/url/verify') -async def verify_url(request: Request, user=Depends(get_admin_user)): - image_config = await get_image_config() - if image_config.IMAGE_GENERATION_ENGINE == 'automatic1111': - try: - session = await get_session() - async with session.get( - url=f'{image_config.AUTOMATIC1111_BASE_URL}/sdapi/v1/options', - headers={'authorization': get_automatic1111_api_auth(image_config)}, - ssl=AIOHTTP_CLIENT_SESSION_SSL, - ) as r: - r.raise_for_status() - return True - except Exception: - raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL) - elif image_config.IMAGE_GENERATION_ENGINE == 'comfyui': - headers = None - if image_config.COMFYUI_API_KEY: - headers = {'Authorization': f'Bearer {image_config.COMFYUI_API_KEY}'} - try: - session = await get_session() - async with session.get( - url=f'{image_config.COMFYUI_BASE_URL}/object_info', - headers=headers, - ssl=AIOHTTP_CLIENT_SESSION_SSL, - ) as r: - r.raise_for_status() - return True - except Exception: - raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL) +class ConnectionVerificationForm(BaseModel): + engine: str + url: str + key: str | None = None + + +@router.post('/verify') +async def verify_connection(form_data: ConnectionVerificationForm, user=Depends(get_admin_user)): + url = form_data.url.rstrip('/') + headers = {} + if form_data.engine == 'automatic1111': + url = f'{url}/sdapi/v1/options' + if form_data.key is not None: + headers['Authorization'] = f'Basic {base64.b64encode(form_data.key.encode("utf-8")).decode("utf-8")}' + elif form_data.engine == 'comfyui': + url = f'{url}/object_info' + if form_data.key: + headers['Authorization'] = f'Bearer {form_data.key}' else: - return True + raise HTTPException(status_code=400, detail='Unsupported image engine') + + try: + session = await get_session() + async with session.get(url=url, headers=headers, ssl=AIOHTTP_CLIENT_SESSION_SSL) as r: + r.raise_for_status() + return True + except Exception: + raise HTTPException(status_code=400, detail=ERROR_MESSAGES.INVALID_URL) @router.get('/models') @@ -436,7 +432,10 @@ async def get_models(request: Request, user=Depends(get_verified_user)): ) ) except Exception as e: - log.exception(f'Failed to list image generation models: {e}') + log.error( + f'Failed to list image generation models: {str(e) or type(e).__name__}', + exc_info=not isinstance(e, (aiohttp.ClientConnectionError, TimeoutError)), + ) raise HTTPException( status_code=400, detail=ERROR_MESSAGES.DEFAULT(e, 'Failed to retrieve image generation models'), @@ -668,7 +667,9 @@ async def image_generations( if image_url := image.get('url', None): image_data, content_type = await get_image_data( image_url, - {k: v for k, v in headers.items() if k != 'Content-Type'}, + {k: v for k, v in headers.items() if k != 'Content-Type'} + if _is_same_origin(image_url, image_config.IMAGES_OPENAI_API_BASE_URL) + else None, ) else: image_data, content_type = await get_image_data(image['b64_json']) @@ -919,11 +920,8 @@ async def image_edits( if data.startswith('http://') or data.startswith('https://'): parsed = urlparse(data) - if ( - parsed.netloc == urlparse(str(request.base_url)).netloc - and parsed.path.startswith('/api/v1/files/') - and '/content' in parsed.path - ): + # Fetching /api/v1/files/{id}/content over the network would be unauthenticated. + if parsed.path.startswith('/api/v1/files/') and '/content' in parsed.path: return await load_url_image(parsed.path) # Validate URL to prevent SSRF attacks against local/private networks. @@ -1047,7 +1045,9 @@ async def image_edits( if image_url := image.get('url', None): image_data, content_type = await get_image_data( image_url, - {k: v for k, v in headers.items() if k != 'Content-Type'}, + {k: v for k, v in headers.items() if k != 'Content-Type'} + if _is_same_origin(image_url, image_config.IMAGES_EDIT_OPENAI_API_BASE_URL) + else None, ) else: image_data, content_type = await get_image_data(image['b64_json']) diff --git a/backend/open_webui/routers/knowledge.py b/backend/open_webui/routers/knowledge.py index 3b58c6f805..43ec07a53c 100644 --- a/backend/open_webui/routers/knowledge.py +++ b/backend/open_webui/routers/knowledge.py @@ -1642,13 +1642,9 @@ async def remove_file_from_knowledge_by_id( # Remove content from the vector database try: - await ASYNC_VECTOR_DB_CLIENT.delete( - collection_name=knowledge.id, filter={'file_id': form_data.file_id} - ) # Remove by file_id first - - await ASYNC_VECTOR_DB_CLIENT.delete( - collection_name=knowledge.id, filter={'hash': file.hash} - ) # Remove by hash as well in case of duplicates + await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=knowledge.id, filter={'file_id': form_data.file_id}) + if file.hash: + await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=knowledge.id, filter={'hash': file.hash}) except Exception as e: log.debug('This was most likely caused by bypassing embedding processing') log.debug(e) @@ -1988,7 +1984,8 @@ async def sync_knowledge_cleanup( try: await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=id, filter={'file_id': file_id}) - await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=id, filter={'hash': file.hash}) + if file.hash: + await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=id, filter={'hash': file.hash}) except Exception: pass @@ -2321,6 +2318,9 @@ async def delete_knowledge_directory( detail=ERROR_MESSAGES.NOT_FOUND, ) + # Collect before delete_directory drops the KnowledgeFile rows + files = [] if move_files else await Knowledges.get_files_by_id_and_directory_id(id, dir_id, db=db) + success = await Knowledges.delete_directory( directory_id=dir_id, move_files_to_parent=move_files, @@ -2331,6 +2331,23 @@ async def delete_knowledge_directory( status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail='Failed to delete directory.', ) + + for file in files: + try: + await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=id, filter={'file_id': file.id}) + if file.hash: + await ASYNC_VECTOR_DB_CLIENT.delete(collection_name=id, filter={'hash': file.hash}) + except Exception as e: + log.debug('This was most likely caused by bypassing embedding processing') + log.debug(e) + + if ( + not ENABLE_KNOWLEDGE_FILE_RETENTION + and not await Knowledges.get_knowledges_by_file_id(file.id, db=db) + and (file.user_id == user.id or user.role == 'admin') + ): + await delete_file_resource(file, db) + await publish_event( request, EVENTS.KNOWLEDGE_DIRECTORY_DELETED, diff --git a/backend/open_webui/routers/memories.py b/backend/open_webui/routers/memories.py index 7dc4e02ec6..2ed11e6094 100644 --- a/backend/open_webui/routers/memories.py +++ b/backend/open_webui/routers/memories.py @@ -241,6 +241,7 @@ async def update_memories( operations = validate_memory_operations(form_data) metadata = getattr(request.state, 'metadata', {}) or {} + model = metadata.get('model') source = form_data.source or 'tool' for operation in operations: if operation.get('action') in {'add', 'replace', 'move'}: @@ -248,7 +249,7 @@ async def update_memories( 'created_by': source, 'chat_id': metadata.get('chat_id'), 'message_id': metadata.get('message_id'), - 'model': metadata.get('model'), + 'model': model.get('id') if isinstance(model, dict) else None, } try: @@ -263,7 +264,7 @@ async def update_memories( for result in results: memory = result.get('memory') if isinstance(memory, MemoryModel): - result = {**result, 'memory': memory.model_dump()} + result = {**result, 'memory': memory.model_dump(exclude={'meta'})} if result.get('status') in {'created', 'updated'}: vector = await request.app.state.EMBEDDING_FUNCTION( memory_vector_text(memory.content, memory.path), @@ -438,7 +439,7 @@ async def read_memory_path( ) return { **result, - 'memories': [memory.model_dump() for memory in result['memories']], + 'memories': [memory.model_dump(exclude={'meta'}) for memory in result['memories']], } diff --git a/backend/open_webui/routers/models.py b/backend/open_webui/routers/models.py index a0fd97705d..f9437b5b87 100644 --- a/backend/open_webui/routers/models.py +++ b/backend/open_webui/routers/models.py @@ -12,6 +12,7 @@ from fastapi import ( APIRouter, Depends, HTTPException, + Query, Request, Response, status, @@ -19,11 +20,16 @@ from fastapi import ( from fastapi.responses import RedirectResponse, StreamingResponse from open_webui.config import BYPASS_ADMIN_ACCESS_CONTROL from open_webui.constants import ERROR_MESSAGES -from open_webui.env import ENABLE_PROFILE_IMAGE_URL_FORWARDING, PROFILE_IMAGE_ALLOWED_MIME_TYPES +from open_webui.env import ( + BYPASS_MODEL_ACCESS_CONTROL, + ENABLE_PROFILE_IMAGE_URL_FORWARDING, + PROFILE_IMAGE_ALLOWED_MIME_TYPES, +) from open_webui.events import EVENTS, publish_event from open_webui.internal.db import get_async_session from open_webui.models.access_grants import AccessGrants from open_webui.models.config import Config +from open_webui.models.files import Files from open_webui.models.groups import Groups from open_webui.models.models import ( ModelAccessListResponse, @@ -36,12 +42,14 @@ from open_webui.models.models import ( ModelResponse, Models, ) -from open_webui.utils.access_control import filter_allowed_access_grants, has_permission +from open_webui.storage.provider import Storage +from open_webui.utils.access_control import filter_allowed_access_grants, has_access, has_permission from open_webui.utils.access_control.files import has_access_to_file from open_webui.utils.auth import get_admin_user, get_verified_user from open_webui.utils.chat_variables import get_chat_variables_schema from open_webui.utils.models import get_all_models -from pydantic import BaseModel +from open_webui.utils.validate import BACKGROUND_IMAGE_MAX_BYTES, validate_background_image +from pydantic import BaseModel, Field from sqlalchemy.ext.asyncio import AsyncSession log = logging.getLogger(__name__) @@ -89,7 +97,28 @@ def _safe_static_redirect_path(url: str) -> str | None: def is_valid_model_id(model_id: str) -> bool: - return model_id and len(model_id) <= 256 + return model_id and len(model_id) <= 256 and not any(char.isspace() for char in model_id) + + +async def _verify_background_image(url: str | None, user, db, previous_url: str | None = None) -> None: + if not url or url == previous_url: + return + file_id = url.split('/')[-2] + file = await Files.get_file_by_id(file_id, db=db) + if not file or not ( + user.role == 'admin' or file.user_id == user.id or await has_access_to_file(file_id, 'read', user, db=db) + ): + raise HTTPException(status_code=403, detail='Background image is not accessible.') + try: + path = await asyncio.to_thread(Storage.get_file, file.path) + with open(path, 'rb') as image: + data = await asyncio.to_thread(image.read, BACKGROUND_IMAGE_MAX_BYTES + 1) + content_type = await asyncio.to_thread(validate_background_image, data) + except (ValueError, OSError) as error: + raise HTTPException(status_code=400, detail=str(error)) from error + if (file.meta or {}).get('content_type') != content_type: + if not await Files.update_file_metadata_by_id(file_id, {'content_type': content_type}, db=db): + raise HTTPException(status_code=500, detail='Could not validate background image.') async def _verify_knowledge_file_access( @@ -211,6 +240,11 @@ async def get_models( ########################### +@router.get('/all', response_model=list[ModelResponse]) +async def get_all_model_records(user=Depends(get_admin_user), db: AsyncSession = Depends(get_async_session)): + return await Models.get_all_models(db=db) + + @router.get('/base/tags', response_model=list[str]) async def get_base_model_tags(user=Depends(get_admin_user), db: AsyncSession = Depends(get_async_session)): tags = await Models.get_all_tags(user_id=user.id, is_admin=True, is_base_model=True, db=db) @@ -306,6 +340,8 @@ async def create_new_model( db, ) + await _verify_background_image(form_data.meta.background_image_url, user, db) + form_data.access_grants = await filter_allowed_access_grants( await Config.get('user.permissions'), user.id, @@ -336,9 +372,14 @@ async def create_new_model( ############################ -@router.get('/export', response_model=list[ModelModel]) +class ModelExportResponse(ModelModel): + background_image_data: str | None = None + + +@router.get('/export', response_model=list[ModelExportResponse]) async def export_models( request: Request, + ids: list[str] | None = Query(None), user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session), ): @@ -354,9 +395,36 @@ async def export_models( ) if user.role == 'admin' and BYPASS_ADMIN_ACCESS_CONTROL: - return await Models.get_models(db=db) + models = await Models.get_models(db=db, ids=ids) else: - return await Models.get_models(writable_by_user_id=user.id, db=db) + models = await Models.get_models(writable_by_user_id=user.id, db=db, ids=ids) + if ids is not None: + requested = set(ids) + if requested != {model.id for model in models}: + raise HTTPException(status_code=403, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) + exported = [] + for model in models: + data = model.model_dump() + url = model.meta.background_image_url + if url: + try: + file = await Files.get_file_by_id(url.split('/')[-2], db=db) + if not file: + raise ValueError('Image file is missing') + path = await asyncio.to_thread(Storage.get_file, file.path) + with open(path, 'rb') as image: + image_data = await asyncio.to_thread(image.read, BACKGROUND_IMAGE_MAX_BYTES + 1) + content_type = await asyncio.to_thread(validate_background_image, image_data) + data['background_image_data'] = f'data:{content_type};base64,' + base64.b64encode(image_data).decode( + 'ascii' + ) + data['meta']['background_image_url'] = None + except Exception as error: + raise HTTPException( + status_code=400, detail=f'Could not export background for model {model.id}.' + ) from error + exported.append(data) + return exported ############################ @@ -486,7 +554,7 @@ async def import_models( updated_model.access_grants, 'sharing.public_models', ) - await Models.update_model_by_id(model_id, updated_model, db=db) + imported_model = updated_model else: # Insert new model model_data['meta'] = model_data.get('meta', {}) @@ -530,7 +598,58 @@ async def import_models( new_model.access_grants, 'sharing.public_models', ) - await Models.insert_new_model(user_id=user.id, form_data=new_model, db=db) + imported_model = new_model + + uploaded = None + try: + encoded = model_data.pop('background_image_data', None) + if encoded is not None: + if ( + not isinstance(encoded, str) + or len(encoded) > 4 * ((BACKGROUND_IMAGE_MAX_BYTES + 2) // 3) + 64 + ): + raise ValueError('Background image must be at most 5 MiB.') + header, payload = encoded.split(',', 1) + image_data = base64.b64decode(payload, validate=True) + content_type = await asyncio.to_thread(validate_background_image, image_data) + if header != f'data:{content_type};base64': + raise ValueError('Invalid background image data URI.') + from fastapi import UploadFile + from open_webui.routers.files import upload_file_handler + + uploaded = await upload_file_handler( + request, + file=UploadFile( + file=io.BytesIO(image_data), + filename='background.' + content_type.split('/')[1], + ), + metadata=None, + process=False, + user=user, + db=db, + ) + imported_model.meta.background_image_url = f'/api/v1/files/{uploaded.id}/content' + await _verify_background_image( + imported_model.meta.background_image_url, + user, + db, + existing_model.meta.background_image_url if existing_model else None, + ) + saved = ( + await Models.update_model_by_id(model_id, imported_model, db=db) + if existing_model + else await Models.insert_new_model(user_id=user.id, form_data=imported_model, db=db) + ) + if not saved: + raise HTTPException(status_code=500, detail=f'Could not import model {model_id}.') + except Exception: + if uploaded: + try: + await Files.delete_file_by_id(uploaded.id, db=db) + await asyncio.to_thread(Storage.delete_file, uploaded.path) + except Exception: + log.exception('Could not clean up failed model background upload') + raise imported_ids.append(model_id) await publish_event( @@ -543,6 +662,10 @@ async def import_models( return True else: raise HTTPException(status_code=400, detail='Invalid JSON format') + except HTTPException: + raise + except ValueError as error: + raise HTTPException(status_code=400, detail=str(error)) from error except Exception as e: log.exception(e) raise HTTPException(status_code=500, detail=str(e)) @@ -564,6 +687,14 @@ async def sync_models( user=Depends(get_admin_user), db: AsyncSession = Depends(get_async_session), ): + existing = {model.id: model for model in await Models.get_models_by_ids([m.id for m in form_data.models], db=db)} + for model in form_data.models: + previous = existing.get(model.id) + if previous and 'background_image_url' not in model.meta.model_fields_set: + model.meta.background_image_url = previous.meta.background_image_url + await _verify_background_image( + model.meta.background_image_url, user, db, previous.meta.background_image_url if previous else None + ) models = await Models.sync_models(user.id, form_data.models, db=db) await publish_event( request, @@ -649,18 +780,37 @@ async def get_model_profile_image( profile_image_url = None updated_at = None + bypass_access_control = BYPASS_MODEL_ACCESS_CONTROL or (user.role == 'admin' and BYPASS_ADMIN_ACCESS_CONTROL) + # First, check the database for regular models model_meta = await Models.get_model_meta_by_id(id, db=db) if model_meta: - meta, updated_at = model_meta - profile_image_url = (meta or {}).get('profile_image_url') + meta, model_user_id, model_updated_at = model_meta + # Denied callers get the default image rather than an error, so model ids stay unprobeable. + if ( + bypass_access_control + or user.id == model_user_id + or await AccessGrants.has_access( + user_id=user.id, + resource_type='model', + resource_id=id, + permission='read', + db=db, + ) + ): + profile_image_url = (meta or {}).get('profile_image_url') + updated_at = model_updated_at # Fallback: check arena models stored in config (not in the DB) if not profile_image_url: arena_models = await Config.get('evaluation.arena.models', []) or [] for arena_model in arena_models: if arena_model.get('id') == id: - profile_image_url = arena_model.get('meta', {}).get('profile_image_url') + arena_meta = arena_model.get('meta', {}) + if bypass_access_control or await has_access( + user.id, permission='read', access_grants=arena_meta.get('access_grants', []), db=db + ): + profile_image_url = arena_meta.get('profile_image_url') break if profile_image_url: @@ -829,6 +979,10 @@ async def update_model_by_id( if 'profile_image_url' not in form_data.meta.model_fields_set: form_data.meta.profile_image_url = model.meta.profile_image_url + if 'background_image_url' not in form_data.meta.model_fields_set: + form_data.meta.background_image_url = model.meta.background_image_url + await _verify_background_image(form_data.meta.background_image_url, user, db, model.meta.background_image_url) + form_data.access_grants = await filter_allowed_access_grants( await Config.get('user.permissions'), user.id, @@ -855,7 +1009,7 @@ async def update_model_by_id( class ModelAccessGrantsForm(BaseModel): - id: str + id: str = Field(pattern=r'^\S+$') name: str | None = None access_grants: list[dict] diff --git a/backend/open_webui/routers/notes.py b/backend/open_webui/routers/notes.py index c1951d2097..a84dd3f94e 100644 --- a/backend/open_webui/routers/notes.py +++ b/backend/open_webui/routers/notes.py @@ -554,14 +554,15 @@ async def update_note_by_id( ): raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=ERROR_MESSAGES.DEFAULT()) - form_data.access_grants = await filter_allowed_access_grants( - await Config.get('user.permissions'), - user.id, - user.role, - form_data.access_grants, - 'sharing.public_notes', - db=db, - ) + if form_data.access_grants is not None: + form_data.access_grants = await filter_allowed_access_grants( + await Config.get('user.permissions'), + user.id, + user.role, + form_data.access_grants, + 'sharing.public_notes', + db=db, + ) try: note = await Notes.update_note_by_id(id, form_data, db=db) diff --git a/backend/open_webui/routers/ollama.py b/backend/open_webui/routers/ollama.py index e9a4685993..b283db9ae8 100644 --- a/backend/open_webui/routers/ollama.py +++ b/backend/open_webui/routers/ollama.py @@ -55,14 +55,15 @@ log = logging.getLogger(__name__) # response body. Forwarding them verbatim causes desktop / programmatic # clients to attempt decompression of an already-decoded payload, resulting # in ZlibError. See https://github.com/aio-libs/aiohttp/issues/4462. -_STRIP_PROXY_HEADERS = frozenset({'Content-Encoding', 'Content-Length', 'Transfer-Encoding'}) +# Also drop server and date: uvicorn adds its own and forwarding both duplicates them. +_STRIP_PROXY_HEADERS = frozenset({'content-encoding', 'content-length', 'transfer-encoding', 'server', 'date'}) _MODEL_LIST_TIMEOUT = aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST) BASE_MODELS_CACHE_KEY = f'{REDIS_KEY_PREFIX}:models:base' def _clean_proxy_headers(raw_headers) -> dict: - """Return a copy of *raw_headers* with stale encoding headers removed.""" - return {k: v for k, v in raw_headers.items() if k not in _STRIP_PROXY_HEADERS} + """Return a copy of *raw_headers* without the encoding, server and date headers.""" + return {k: v for k, v in raw_headers.items() if k.lower() not in _STRIP_PROXY_HEADERS} async def send_get_request( @@ -119,7 +120,7 @@ async def send_request( } if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user) + headers = include_user_info_headers(headers, user, request=request) if metadata and metadata.get('chat_id'): headers[FORWARD_SESSION_INFO_HEADER_CHAT_ID] = metadata.get('chat_id') @@ -478,13 +479,16 @@ async def get_filtered_models(models, user, db=None): @router.get('/api/tags') -@router.get('/api/tags/{url_idx}', dependencies=[Depends(get_admin_user)]) +@router.get('/api/tags/{url_idx}') async def get_ollama_tags( request: Request, url_idx: int | None = None, user=Depends(get_verified_user), ): """List Ollama model tags, optionally from a specific backend.""" + if url_idx is not None and user.role != 'admin': + raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) + if not await Config.get('ollama.enable'): raise HTTPException(status_code=503, detail=ERROR_MESSAGES.OLLAMA_API_DISABLED) @@ -541,13 +545,16 @@ async def get_ollama_loaded_models( @router.get('/api/version') -@router.get('/api/version/{url_idx}', dependencies=[Depends(get_admin_user)]) +@router.get('/api/version/{url_idx}') async def get_ollama_versions( request: Request, user=Depends(get_verified_user), url_idx: int | None = None, ): """Return the lowest Ollama version across all configured backends.""" + if url_idx is not None and user.role != 'admin': + raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) + if not await Config.get('ollama.enable'): return {'version': False} @@ -1479,7 +1486,7 @@ async def generate_responses( @router.get('/v1/models') -@router.get('/v1/models/{url_idx}', dependencies=[Depends(get_admin_user)]) +@router.get('/v1/models/{url_idx}') async def get_openai_models( request: Request, url_idx: int | None = None, @@ -1487,6 +1494,9 @@ async def get_openai_models( db: AsyncSession = Depends(get_async_session), ) -> dict: """List models in the OpenAI-compatible format.""" + if url_idx is not None and user.role != 'admin': + raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) + if url_idx is None: model_list = await get_all_models(request, user=user) raw_models = model_list['models'] diff --git a/backend/open_webui/routers/openai.py b/backend/open_webui/routers/openai.py index d7856bea48..63676d166e 100644 --- a/backend/open_webui/routers/openai.py +++ b/backend/open_webui/routers/openai.py @@ -74,15 +74,16 @@ log = logging.getLogger(__name__) # response body. Forwarding them verbatim causes desktop / programmatic # clients to attempt decompression of an already-decoded payload, resulting # in ZlibError. See https://github.com/aio-libs/aiohttp/issues/4462. -_STRIP_PROXY_HEADERS = frozenset({'Content-Encoding', 'Content-Length', 'Transfer-Encoding'}) +# Also drop server and date: uvicorn adds its own and forwarding both duplicates them. +_STRIP_PROXY_HEADERS = frozenset({'content-encoding', 'content-length', 'transfer-encoding', 'server', 'date'}) _MODEL_LIST_TIMEOUT = aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST) _UNSUPPORTED_OPENAI_MODEL_KEYWORDS = ('babbage', 'dall-e', 'davinci', 'embedding', 'tts', 'whisper') BASE_MODELS_CACHE_KEY = f'{REDIS_KEY_PREFIX}:models:base' def _clean_proxy_headers(raw_headers) -> dict: - """Return a copy of *raw_headers* with stale encoding headers removed.""" - return {k: v for k, v in raw_headers.items() if k not in _STRIP_PROXY_HEADERS} + """Return a copy of *raw_headers* without the encoding, server and date headers.""" + return {k: v for k, v in raw_headers.items() if k.lower() not in _STRIP_PROXY_HEADERS} async def send_get_request( @@ -103,7 +104,7 @@ async def send_get_request( cookies = None if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user) + headers = include_user_info_headers(headers, user, request=request) async with session.get( url, @@ -159,7 +160,7 @@ async def get_headers_and_cookies( metadata: dict | None = None, user: UserModel = None, ): - cookies = {} + cookies = getattr(request, 'cookies', {}) if config.get('forward_cookies', False) else {} headers = { 'Content-Type': 'application/json', **( @@ -176,7 +177,7 @@ async def get_headers_and_cookies( } if ENABLE_FORWARD_USER_INFO_HEADERS and user: - headers = include_user_info_headers(headers, user) + headers = include_user_info_headers(headers, user, request=request) if metadata and metadata.get('chat_id'): headers[FORWARD_SESSION_INFO_HEADER_CHAT_ID] = metadata.get('chat_id') @@ -189,11 +190,8 @@ async def get_headers_and_cookies( elif auth_type == 'none': token = None elif auth_type == 'session': - cookies = request.cookies token = request.state.token.credentials elif auth_type == 'system_oauth': - cookies = request.cookies - oauth_token = None try: if request.cookies.get('oauth_session_id', None): @@ -864,8 +862,11 @@ async def get_all_models(request: Request, user: UserModel) -> dict[str, list]: @router.get('/models') -@router.get('/models/{url_idx}', dependencies=[Depends(get_admin_user)]) +@router.get('/models/{url_idx}') async def get_models(request: Request, url_idx: int | None = None, user=Depends(get_verified_user)): + if url_idx is not None and user.role != 'admin': + raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED) + if not await Config.get('openai.enable'): raise HTTPException(status_code=503, detail='OpenAI API is disabled') @@ -1416,14 +1417,19 @@ def convert_to_responses_payload(payload: dict) -> dict: converted_tool['description'] = func['description'] if 'parameters' in func: converted_tool['parameters'] = func['parameters'] - if 'strict' in func: - converted_tool['strict'] = func['strict'] + # Responses defaults strict to true, Chat Completions to false + converted_tool['strict'] = func.get('strict', False) converted_tools.append(converted_tool) else: # Already in correct format or unknown format, pass through converted_tools.append(tool) responses_payload['tools'] = converted_tools + # Responses API expects a forced function choice as {"type": "function", "name": ...} + tool_choice = responses_payload.get('tool_choice') + if isinstance(tool_choice, dict) and isinstance(tool_choice.get('function'), dict): + responses_payload['tool_choice'] = {'type': 'function', 'name': tool_choice['function'].get('name', '')} + return responses_payload @@ -1431,17 +1437,32 @@ def convert_responses_result(response: dict) -> dict: """ Convert non-streaming Responses API result to Chat Completions format. - Extracts text from message output items so all downstream consumers + Extracts text and function calls from output items so all downstream consumers (frontend tasks, get_content_from_response) work without modification. """ output_items = response.get('output', []) content = '' + tool_calls = [] for item in output_items: if item.get('type') == 'message': for part in item.get('content', []): if part.get('type') == 'output_text': content += part.get('text', '') + elif item.get('type') == 'function_call': + arguments = item.get('arguments', '{}') + if not isinstance(arguments, str): + arguments = JSONCodec.dumps(arguments) + tool_calls.append( + { + 'id': item.get('call_id', ''), + 'type': 'function', + 'function': { + 'name': item.get('name', ''), + 'arguments': arguments, + }, + } + ) return { 'id': response.get('id', ''), @@ -1453,8 +1474,9 @@ def convert_responses_result(response: dict) -> dict: 'message': { 'role': 'assistant', 'content': content, + **({'tool_calls': tool_calls} if tool_calls else {}), }, - 'finish_reason': 'stop', + 'finish_reason': 'tool_calls' if tool_calls else 'stop', } ], 'usage': response.get('usage', {}), @@ -1566,6 +1588,20 @@ async def generate_chat_completion( is_responses = api_config.get('api_type') == 'responses' + # Explicit continuation keeps llama.cpp from echoing the prefill in streamed replies. + if ( + api_config.get('provider') == 'llama.cpp' + # These flags apply to Chat Completions, not the Responses API. + and not is_responses + # The frontend sends this ID when the user clicks Continue. + and (metadata or {}).get('assistant_message_id') + # Tool follow-ups retain the metadata but must start a new assistant turn. + and payload.get('messages') + and payload['messages'][-1].get('role') == 'assistant' + ): + payload['continue_final_message'] = True + payload['add_generation_prompt'] = False + if api_config.get('azure') or api_config.get('provider') == 'azure': # Only set api-key header if not using Azure Entra ID authentication auth_type = api_config.get('auth_type', 'bearer') diff --git a/backend/open_webui/routers/retrieval.py b/backend/open_webui/routers/retrieval.py index 6833d491db..c36b0b8a53 100644 --- a/backend/open_webui/routers/retrieval.py +++ b/backend/open_webui/routers/retrieval.py @@ -58,6 +58,7 @@ from open_webui.env import ( SENTENCE_TRANSFORMERS_CROSS_ENCODER_MODEL_KWARGS, SENTENCE_TRANSFORMERS_CROSS_ENCODER_SIGMOID_ACTIVATION_FUNCTION, SENTENCE_TRANSFORMERS_MODEL_KWARGS, + USE_SLIM, ) from open_webui.events import EVENTS, publish_event from open_webui.internal.db import get_async_db, get_async_session @@ -82,7 +83,7 @@ from open_webui.retrieval.utils import ( query_doc_with_hybrid_search, ) from open_webui.retrieval.vector.async_client import ASYNC_VECTOR_DB_CLIENT -from open_webui.retrieval.vector.factory import VECTOR_DB_CLIENT +from open_webui.retrieval.vector.factory import get_vector_db_client from open_webui.retrieval.vector.utils import filter_metadata from open_webui.retrieval.web.azure import search_azure from open_webui.retrieval.web.bing import search_bing @@ -114,6 +115,7 @@ from open_webui.retrieval.web.serphouse import search_serphouse from open_webui.retrieval.web.serply import search_serply from open_webui.retrieval.web.serpstack import search_serpstack from open_webui.retrieval.web.sougou import search_sougou +from open_webui.retrieval.web.staan import search_staan from open_webui.retrieval.web.tavily import search_tavily from open_webui.retrieval.web.utils import get_web_loader from open_webui.retrieval.web.yacy import search_yacy @@ -128,7 +130,7 @@ from open_webui.utils.misc import ( calculate_sha256_string, sanitize_text_for_db, ) -from pydantic import BaseModel +from pydantic import BaseModel, Field from sqlalchemy.ext.asyncio import AsyncSession log = logging.getLogger(__name__) @@ -150,7 +152,7 @@ def get_ef( auto_update: bool = RAG_EMBEDDING_MODEL_AUTO_UPDATE, ): ef = None - if embedding_model and engine == '': + if embedding_model and engine == '' and not USE_SLIM: from sentence_transformers import SentenceTransformer try: @@ -178,6 +180,17 @@ def get_rf( rf = None # Convert timeout string to int or None (system default) timeout_value = int(external_reranker_timeout) if external_reranker_timeout else None + if reranking_model and engine == 'external': + from open_webui.retrieval.models.external import ExternalReranker + + return ExternalReranker( + url=external_reranker_url, + api_key=external_reranker_api_key, + model=reranking_model, + timeout=timeout_value, + ) + if USE_SLIM: + return None if reranking_model: if any(model in reranking_model for model in ['jinaai/jina-colbert-v2']): try: @@ -192,55 +205,39 @@ def get_rf( log.error(f'ColBERT: {e}') raise Exception(ERROR_MESSAGES.DEFAULT(e, 'Error loading reranking model')) else: - if engine == 'external': - try: - from open_webui.retrieval.models.external import ExternalReranker + import sentence_transformers + import torch - rf = ExternalReranker( - url=external_reranker_url, - api_key=external_reranker_api_key, - model=reranking_model, - timeout=timeout_value, - ) - except Exception as e: - log.error(f'ExternalReranking: {e}') - raise Exception(ERROR_MESSAGES.DEFAULT(e, 'Error loading reranking model')) - else: - import sentence_transformers - import torch + try: + rf = sentence_transformers.CrossEncoder( + get_model_path(reranking_model, auto_update), + device=DEVICE_TYPE, + trust_remote_code=RAG_RERANKING_MODEL_TRUST_REMOTE_CODE, + backend=SENTENCE_TRANSFORMERS_CROSS_ENCODER_BACKEND, + model_kwargs=SENTENCE_TRANSFORMERS_CROSS_ENCODER_MODEL_KWARGS, + activation_fn=( + torch.nn.Sigmoid() if SENTENCE_TRANSFORMERS_CROSS_ENCODER_SIGMOID_ACTIVATION_FUNCTION else None + ), + ) + except Exception as e: + log.error(f'CrossEncoder: {e}') + raise Exception(ERROR_MESSAGES.DEFAULT(e, 'CrossEncoder error')) - try: - rf = sentence_transformers.CrossEncoder( - get_model_path(reranking_model, auto_update), - device=DEVICE_TYPE, - trust_remote_code=RAG_RERANKING_MODEL_TRUST_REMOTE_CODE, - backend=SENTENCE_TRANSFORMERS_CROSS_ENCODER_BACKEND, - model_kwargs=SENTENCE_TRANSFORMERS_CROSS_ENCODER_MODEL_KWARGS, - activation_fn=( - torch.nn.Sigmoid() - if SENTENCE_TRANSFORMERS_CROSS_ENCODER_SIGMOID_ACTIVATION_FUNCTION - else None - ), - ) - except Exception as e: - log.error(f'CrossEncoder: {e}') - raise Exception(ERROR_MESSAGES.DEFAULT(e, 'CrossEncoder error')) - - # Safely adjust pad_token_id if missing as some models do not have this in config - try: - model_cfg = getattr(rf, 'model', None) - if model_cfg and hasattr(model_cfg, 'config'): - cfg = model_cfg.config - if getattr(cfg, 'pad_token_id', None) is None: - # Fallback to eos_token_id when available - eos = getattr(cfg, 'eos_token_id', None) - if eos is not None: - cfg.pad_token_id = eos - log.debug('Missing pad_token_id detected; set to eos_token_id=%s', eos) - else: - log.warning('Neither pad_token_id nor eos_token_id present in model config') - except Exception as e2: - log.warning(f'Failed to adjust pad_token_id on CrossEncoder: {e2}') + # Safely adjust pad_token_id if missing as some models do not have this in config + try: + model_cfg = getattr(rf, 'model', None) + if model_cfg and hasattr(model_cfg, 'config'): + cfg = model_cfg.config + if getattr(cfg, 'pad_token_id', None) is None: + # Fallback to eos_token_id when available + eos = getattr(cfg, 'eos_token_id', None) + if eos is not None: + cfg.pad_token_id = eos + log.debug('Missing pad_token_id detected; set to eos_token_id=%s', eos) + else: + log.warning('Neither pad_token_id nor eos_token_id present in model config') + except Exception as e2: + log.warning(f'Failed to adjust pad_token_id on CrossEncoder: {e2}') return rf @@ -301,6 +298,7 @@ RETRIEVAL_CONFIG_KEYS = { 'ENABLE_WEB_SEARCH_CONFIRMATION': 'web.search.confirmation.enable', 'WEB_SEARCH_CONFIRMATION_CONTENT': 'web.search.confirmation.content', 'EXA_API_KEY': 'web.search.exa_api_key', + 'EXA_MAX_CONTENT_LENGTH': 'web.search.exa_max_content_length', 'EXTERNAL_DOCUMENT_LOADER_API_KEY': 'rag.external_document_loader_api_key', 'EXTERNAL_DOCUMENT_LOADER_HEADERS': 'rag.external_document_loader_headers', 'EXTERNAL_DOCUMENT_LOADER_URL': 'rag.external_document_loader_url', @@ -383,6 +381,9 @@ RETRIEVAL_CONFIG_KEYS = { 'SERPSTACK_HTTPS': 'web.search.serpstack_https', 'SOUGOU_API_SID': 'web.search.sougou_api_sid', 'SOUGOU_API_SK': 'web.search.sougou_api_sk', + 'STAAN_API_KEY': 'web.search.staan_api_key', + 'STAAN_MARKET': 'web.search.staan_market', + 'STAAN_MAX_SNIPPETS': 'web.search.staan_max_snippets', 'TAVILY_API_KEY': 'web.search.tavily_api_key', 'TAVILY_EXTRACT_DEPTH': 'web.search.tavily_extract_depth', 'TEXT_SPLITTER': 'rag.text_splitter', @@ -535,6 +536,8 @@ async def unload_embedding_model(request: Request): @router.post('/embedding/update') async def update_embedding_config(request: Request, form_data: EmbeddingModelUpdateForm, user=Depends(get_admin_user)): + if USE_SLIM and form_data.RAG_EMBEDDING_ENGINE == '': + raise HTTPException(400, 'Slim requires an external embedding engine (openai, ollama, azure_openai).') config = await get_retrieval_config() log.info('Updating embedding model: %s to %s', config.RAG_EMBEDDING_MODEL, form_data.RAG_EMBEDDING_MODEL) await unload_embedding_model(request) @@ -738,6 +741,9 @@ async def get_rag_config(request: Request, user=Depends(get_admin_user)): 'SERPLY_API_KEY': config.SERPLY_API_KEY, 'DDGS_BACKEND': config.DDGS_BACKEND, 'TAVILY_API_KEY': config.TAVILY_API_KEY, + 'STAAN_API_KEY': config.STAAN_API_KEY, + 'STAAN_MARKET': config.STAAN_MARKET, + 'STAAN_MAX_SNIPPETS': config.STAAN_MAX_SNIPPETS, 'SEARCHAPI_API_KEY': config.SEARCHAPI_API_KEY, 'SEARCHAPI_ENGINE': config.SEARCHAPI_ENGINE, 'SERPAPI_API_KEY': config.SERPAPI_API_KEY, @@ -747,6 +753,7 @@ async def get_rag_config(request: Request, user=Depends(get_admin_user)): 'BING_SEARCH_V7_ENDPOINT': config.BING_SEARCH_V7_ENDPOINT, 'BING_SEARCH_V7_SUBSCRIPTION_KEY': config.BING_SEARCH_V7_SUBSCRIPTION_KEY, 'EXA_API_KEY': config.EXA_API_KEY, + 'EXA_MAX_CONTENT_LENGTH': config.EXA_MAX_CONTENT_LENGTH, 'PERPLEXITY_API_KEY': config.PERPLEXITY_API_KEY, 'PERPLEXITY_MODEL': config.PERPLEXITY_MODEL, 'PERPLEXITY_SEARCH_CONTEXT_USAGE': config.PERPLEXITY_SEARCH_CONTEXT_USAGE, @@ -817,6 +824,9 @@ class WebConfig(BaseModel): SERPLY_API_KEY: str | None = None DDGS_BACKEND: str | None = None TAVILY_API_KEY: str | None = None + STAAN_API_KEY: str | None = None + STAAN_MARKET: str | None = None + STAAN_MAX_SNIPPETS: int | None = None SEARCHAPI_API_KEY: str | None = None SEARCHAPI_ENGINE: str | None = None SERPAPI_API_KEY: str | None = None @@ -826,6 +836,7 @@ class WebConfig(BaseModel): BING_SEARCH_V7_ENDPOINT: str | None = None BING_SEARCH_V7_SUBSCRIPTION_KEY: str | None = None EXA_API_KEY: str | None = None + EXA_MAX_CONTENT_LENGTH: int | None = Field(default=None, gt=0, strict=True) PERPLEXITY_API_KEY: str | None = None PERPLEXITY_MODEL: str | None = None PERPLEXITY_SEARCH_CONTEXT_USAGE: str | None = None @@ -952,6 +963,44 @@ class ConfigForm(BaseModel): async def update_rag_config(request: Request, form_data: ConfigForm, user=Depends(get_admin_user)): # RAG settings config = await get_retrieval_config() + if USE_SLIM: + if ( + form_data.web + and form_data.web.WEB_SEARCH_ENGINE == 'duckduckgo' + and config.WEB_SEARCH_ENGINE != 'duckduckgo' + ): + raise HTTPException( + 400, + 'DDGS is unavailable in slim. Configure another web search provider in Admin Settings > Web Search.', + ) + if ( + form_data.web + and form_data.web.WEB_LOADER_ENGINE == 'playwright' + and config.WEB_LOADER_ENGINE != 'playwright' + ): + raise HTTPException( + 400, 'Playwright is unavailable in slim. Use basic HTTP fetching or an external web loader.' + ) + if form_data.TEXT_SPLITTER == 'token_transformers' and config.TEXT_SPLITTER != 'token_transformers': + raise HTTPException( + 400, 'Transformers tokenization is unavailable in slim. Use character or token splitting.' + ) + reranker_engine = ( + form_data.RAG_RERANKING_ENGINE + if form_data.RAG_RERANKING_ENGINE is not None + else config.RAG_RERANKING_ENGINE + ) + reranker_model = ( + form_data.RAG_RERANKING_MODEL if form_data.RAG_RERANKING_MODEL is not None else config.RAG_RERANKING_MODEL + ) + if ( + reranker_engine != 'external' + and reranker_model + and (reranker_engine != config.RAG_RERANKING_ENGINE or reranker_model != config.RAG_RERANKING_MODEL) + ): + raise HTTPException( + 400, 'Slim requires an external reranker, or an empty reranking model for cosine scoring.' + ) config.RAG_TEMPLATE = form_data.RAG_TEMPLATE if form_data.RAG_TEMPLATE is not None else config.RAG_TEMPLATE config.TOP_K = form_data.TOP_K if form_data.TOP_K is not None else config.TOP_K config.BYPASS_EMBEDDING_AND_RETRIEVAL = ( @@ -1176,7 +1225,8 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend else config.RAG_RERANKING_BATCH_SIZE ) - log.info('Updating reranking model: %s to %s', config.RAG_RERANKING_MODEL, form_data.RAG_RERANKING_MODEL) + if form_data.RAG_RERANKING_MODEL is not None: + log.info('Updating reranking model: %s to %s', config.RAG_RERANKING_MODEL, form_data.RAG_RERANKING_MODEL) try: config.RAG_RERANKING_MODEL = ( form_data.RAG_RERANKING_MODEL if form_data.RAG_RERANKING_MODEL is not None else config.RAG_RERANKING_MODEL @@ -1297,6 +1347,9 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend config.SERPLY_API_KEY = form_data.web.SERPLY_API_KEY config.DDGS_BACKEND = form_data.web.DDGS_BACKEND config.TAVILY_API_KEY = form_data.web.TAVILY_API_KEY + config.STAAN_API_KEY = form_data.web.STAAN_API_KEY + config.STAAN_MARKET = form_data.web.STAAN_MARKET + config.STAAN_MAX_SNIPPETS = form_data.web.STAAN_MAX_SNIPPETS config.SEARCHAPI_API_KEY = form_data.web.SEARCHAPI_API_KEY config.SEARCHAPI_ENGINE = form_data.web.SEARCHAPI_ENGINE config.SERPAPI_API_KEY = form_data.web.SERPAPI_API_KEY @@ -1306,6 +1359,7 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend config.BING_SEARCH_V7_ENDPOINT = form_data.web.BING_SEARCH_V7_ENDPOINT config.BING_SEARCH_V7_SUBSCRIPTION_KEY = form_data.web.BING_SEARCH_V7_SUBSCRIPTION_KEY config.EXA_API_KEY = form_data.web.EXA_API_KEY + config.EXA_MAX_CONTENT_LENGTH = form_data.web.EXA_MAX_CONTENT_LENGTH config.PERPLEXITY_API_KEY = form_data.web.PERPLEXITY_API_KEY config.PERPLEXITY_MODEL = form_data.web.PERPLEXITY_MODEL config.PERPLEXITY_SEARCH_CONTEXT_USAGE = form_data.web.PERPLEXITY_SEARCH_CONTEXT_USAGE @@ -1449,6 +1503,9 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend 'SERPHOUSE_DOMAIN': config.SERPHOUSE_DOMAIN, 'SERPLY_API_KEY': config.SERPLY_API_KEY, 'TAVILY_API_KEY': config.TAVILY_API_KEY, + 'STAAN_API_KEY': config.STAAN_API_KEY, + 'STAAN_MARKET': config.STAAN_MARKET, + 'STAAN_MAX_SNIPPETS': config.STAAN_MAX_SNIPPETS, 'SEARCHAPI_API_KEY': config.SEARCHAPI_API_KEY, 'SEARCHAPI_ENGINE': config.SEARCHAPI_ENGINE, 'SERPAPI_API_KEY': config.SERPAPI_API_KEY, @@ -1458,6 +1515,7 @@ async def update_rag_config(request: Request, form_data: ConfigForm, user=Depend 'BING_SEARCH_V7_ENDPOINT': config.BING_SEARCH_V7_ENDPOINT, 'BING_SEARCH_V7_SUBSCRIPTION_KEY': config.BING_SEARCH_V7_SUBSCRIPTION_KEY, 'EXA_API_KEY': config.EXA_API_KEY, + 'EXA_MAX_CONTENT_LENGTH': config.EXA_MAX_CONTENT_LENGTH, 'PERPLEXITY_API_KEY': config.PERPLEXITY_API_KEY, 'PERPLEXITY_MODEL': config.PERPLEXITY_MODEL, 'PERPLEXITY_SEARCH_CONTEXT_USAGE': config.PERPLEXITY_SEARCH_CONTEXT_USAGE, @@ -1589,6 +1647,8 @@ def merge_docs_to_target_size( def get_transformers_tokenizer(request: Request, config: RetrievalConfig): + if USE_SLIM: + raise HTTPException(503, 'Transformers tokenization is unavailable in slim. Use character or token splitting.') if config.RAG_TOKENIZER_MODEL: from transformers import AutoTokenizer @@ -1634,6 +1694,14 @@ def get_splitter_length_function( return len +def filter_file_metadata(metadata: dict | None) -> dict: + metadata = dict(metadata or {}) + data = metadata.pop('data', None) + if isinstance(data, dict): + metadata = {**filter_metadata(data), **metadata} + return filter_metadata(metadata) + + def save_docs_to_vector_db( request: Request, docs, @@ -1665,7 +1733,7 @@ def save_docs_to_vector_db( # Check if entries with the same hash (metadata.hash) already exist if metadata and 'hash' in metadata: - result = VECTOR_DB_CLIENT.query( + result = get_vector_db_client().query( collection_name=collection_name, filter={'hash': metadata['hash']}, ) @@ -1765,11 +1833,11 @@ def save_docs_to_vector_db( ] try: - if VECTOR_DB_CLIENT.has_collection(collection_name=collection_name): + if get_vector_db_client().has_collection(collection_name=collection_name): log.info('collection %s already exists', collection_name) if overwrite: - VECTOR_DB_CLIENT.delete_collection(collection_name=collection_name) + get_vector_db_client().delete_collection(collection_name=collection_name) log.info('deleting existing collection %s', collection_name) elif add is False: log.info('collection %s already exists, overwrite is False and add is False', collection_name) @@ -1832,7 +1900,7 @@ def save_docs_to_vector_db( ] log.info('adding to collection %s', collection_name) - VECTOR_DB_CLIENT.insert( + get_vector_db_client().insert( collection_name=collection_name, items=items, ) @@ -1898,7 +1966,7 @@ async def process_file( Document( page_content=form_data.content.replace('
', '\n'), metadata={ - **file.meta, + **filter_file_metadata(file.meta), 'name': file.filename, 'created_by': file.user_id, 'file_id': file.id, @@ -1934,7 +2002,7 @@ async def process_file( Document( page_content=stored_content, metadata={ - **file.meta, + **filter_file_metadata(file.meta), 'name': file.filename, 'created_by': file.user_id, 'file_id': file.id, @@ -1967,6 +2035,7 @@ async def process_file( Document( page_content=doc.page_content, metadata={ + **filter_file_metadata(file.meta), **filter_metadata(doc.metadata), 'name': file.filename, 'created_by': file.user_id, @@ -1981,7 +2050,7 @@ async def process_file( Document( page_content=file.data.get('content', ''), metadata={ - **file.meta, + **filter_file_metadata(file.meta), 'name': file.filename, 'created_by': file.user_id, 'file_id': file.id, @@ -2636,6 +2705,19 @@ async def search_web(request: Request, engine: str, query: str, user=None) -> li ) else: raise Exception('No TAVILY_API_KEY found in environment variables') + elif engine == 'staan': + if config.STAAN_API_KEY: + return await asyncio.to_thread( + search_staan, + config.STAAN_API_KEY, + query, + config.WEB_SEARCH_RESULT_COUNT, + config.WEB_SEARCH_DOMAIN_FILTER_LIST, + market=config.STAAN_MARKET, + max_snippets=config.STAAN_MAX_SNIPPETS, + ) + else: + raise Exception('No STAAN_API_KEY found in environment variables') elif engine == 'exa': if config.EXA_API_KEY: return await asyncio.to_thread( @@ -2644,6 +2726,7 @@ async def search_web(request: Request, engine: str, query: str, user=None) -> li query, config.WEB_SEARCH_RESULT_COUNT, config.WEB_SEARCH_DOMAIN_FILTER_LIST, + max_content_length=config.EXA_MAX_CONTENT_LENGTH, ) else: raise Exception('No EXA_API_KEY found in environment variables') @@ -3026,7 +3109,7 @@ async def query_doc_handler( query_embedding = await request.app.state.EMBEDDING_FUNCTION( form_data.query, prefix=RAG_EMBEDDING_QUERY_PREFIX, user=user ) - # query_doc wraps a blocking VECTOR_DB_CLIENT.search call; + # query_doc wraps a blocking get_vector_db_client().search call; # offload so the request's event loop stays responsive. return await asyncio.to_thread( query_doc, @@ -3314,7 +3397,7 @@ async def process_files_batch( Document( page_content=text_content.replace('
', '\n'), metadata={ - **file.meta, + **filter_file_metadata(file.meta), 'name': file.filename, 'created_by': file.user_id, 'file_id': file.id, diff --git a/backend/open_webui/routers/scim.py b/backend/open_webui/routers/scim.py index a9bb999cb7..a2d093dba8 100644 --- a/backend/open_webui/routers/scim.py +++ b/backend/open_webui/routers/scim.py @@ -753,31 +753,60 @@ async def patch_user( ) update_data = {} + fields = { + 'userName': 'email', + 'displayName': 'name', + 'emails[primary eq true].value': 'email', + 'name.formatted': 'name', + } for operation in patch_data.Operations: op = operation.op.lower() path = operation.path - value = operation.value - if op == 'replace': + if op not in ('add', 'replace', 'remove'): + return scim_error(400, f'Unsupported PATCH operation: {operation.op}') + if op == 'remove': + if not path: + return scim_error(400, 'Remove requires a path', 'noTarget') + if path != 'externalId': + return scim_error(400, f'Removing {path} is not supported', 'mutability') + values = {path: None} + elif path is None: + if not isinstance(operation.value, dict) or not operation.value: + return scim_error(400, 'A pathless operation requires an attribute object', 'invalidValue') + values = operation.value + else: + values = {path: operation.value} + + for path, value in values.items(): if path == 'active': + if not isinstance(value, bool): + return scim_error(400, 'active must be a boolean', 'invalidValue') # Same guard as update_user: never demote an existing admin via SCIM. if user.role != 'admin': update_data['role'] = 'user' if value else 'pending' - elif path == 'userName': - update_data['email'] = value - elif path == 'displayName': - update_data['name'] = value - elif path == 'emails[primary eq true].value': - update_data['email'] = value - elif path == 'name.formatted': - update_data['name'] = value + elif path in fields: + if not isinstance(value, str): + return scim_error(400, f'{path} must be a string', 'invalidValue') + update_data[fields[path]] = value elif path == 'externalId': + if value is not None and not isinstance(value, str): + return scim_error(400, 'externalId must be a string or null', 'invalidValue') provider = get_scim_provider() - await Users.update_user_scim_by_id(user_id, provider, value, db=db) + scim = dict(update_data.get('scim', user.scim) or {}) + scim[provider] = {'external_id': value} + update_data['scim'] = scim + else: + return scim_error(400, f'Unsupported PATCH path: {path}', 'invalidPath') + + # Validate all operations before persisting once, and leave identical writes unchanged. + update_data = {key: value for key, value in update_data.items() if value != getattr(user, key)} + user_updated_fields = ['externalId' if field == 'scim' else field for field in update_data if field != 'role'] # Update user if update_data: + update_data['updated_at'] = int(time.time()) updated_user = await Users.update_user_by_id(user_id, update_data, db=db) if not updated_user: raise HTTPException( @@ -788,7 +817,6 @@ async def patch_user( updated_user = user role_changed = updated_user.role != user.role - user_updated_fields = [field for field in update_data.keys() if field != 'role'] if user_updated_fields: await publish_event( diff --git a/backend/open_webui/routers/terminals.py b/backend/open_webui/routers/terminals.py index a9b11f00f5..e57314bfde 100644 --- a/backend/open_webui/routers/terminals.py +++ b/backend/open_webui/routers/terminals.py @@ -33,13 +33,18 @@ from open_webui.utils.terminals import ( ) from starlette.background import BackgroundTask from starlette.requests import ClientDisconnect +from yarl import URL log = logging.getLogger(__name__) router = APIRouter() STREAMING_CONTENT_TYPES = ('application/octet-stream', 'image/', 'application/pdf') -STRIPPED_RESPONSE_HEADERS = frozenset(('transfer-encoding', 'connection', 'content-encoding', 'content-length')) +ADMIN_API_PATHS = ('api/v1/policies', 'api/v1/status', 'api/v1/terminals') +# Drop the upstream's server and date: uvicorn adds its own and forwarding both duplicates them. +STRIPPED_RESPONSE_HEADERS = frozenset( + ('transfer-encoding', 'connection', 'content-encoding', 'content-length', 'server', 'date') +) def _sanitize_proxy_path(path: str) -> str | None: @@ -62,7 +67,8 @@ def _sanitize_proxy_path(path: str) -> str | None: return None # posixpath splits on '/' only, so 'a/..\..\b' survives normpath as one component. # Upstreams that treat '\' as a separator would resolve it, so reject outright. - if '\\' in decoded: + # URL parsers also remove tabs/newlines, which can turn '.\t.' into '..'. + if any(char in decoded for char in '\\\t\r\n'): return None had_trailing_slash = decoded.endswith('/') normalized = posixpath.normpath(decoded) @@ -130,6 +136,15 @@ async def proxy_terminal( target_url = f'{base_url}/{safe_path}' + # Check the path aiohttp will send, relative to the configured server root. + base_path = URL(str(connection.get('url') or '')).path.rstrip('/') + target_path = URL(target_url).path + if any( + target_path == f'{base_path}/{prefix}' or target_path.startswith(f'{base_path}/{prefix}/') + for prefix in ADMIN_API_PATHS + ): + return JSONResponse({'error': 'Path not allowed'}, status_code=403) + if request.query_params: target_url += f'?{request.query_params}' @@ -145,16 +160,14 @@ async def proxy_terminal( return JSONResponse({'error': 'A saved chat is required for this terminal'}, status_code=409) if context_id: headers[TERMINAL_CONTEXT_HEADER] = context_id - cookies = {} + cookies = getattr(request, 'cookies', {}) if connection.get('forward_cookies', False) else {} auth_type = connection.get('auth_type', 'bearer') if auth_type == 'bearer': headers.update(bearer_auth_header(connection.get('key', ''))) elif auth_type == 'session': - cookies = request.cookies headers.update(bearer_auth_header(request.state.token.credentials)) elif auth_type == 'system_oauth': - cookies = request.cookies # Resolve the token server-side from the caller's OAuth session; never trust a client header. oauth_token = None try: @@ -188,6 +201,7 @@ async def proxy_terminal( cookies=cookies, data=body or None, ssl=AIOHTTP_CLIENT_SESSION_SSL, + allow_redirects=False, ) upstream_content_type = upstream_response.headers.get('content-type', '') @@ -326,7 +340,7 @@ async def ws_terminal( # For orchestrator-backed servers, pass user_id upstream_params['user_id'] = user.id context_id = terminal_context_id(connection, {'chat_id': chat_id}, 'chat') - upstream_headers = {} + upstream_headers = {'X-User-Id': user.id, 'X-Session-Id': chat_id} if terminal_context_config(connection, 'chat').get('context_id') == 'chat_id' and not context_id: await ws.close(code=4003, reason='A saved chat is required for this terminal') return @@ -362,6 +376,8 @@ async def ws_terminal( await upstream.send_str(_json.dumps({'type': 'auth', 'token': key})) elif auth_type == 'session' and is_terminal_orchestrator(connection): await upstream.send_str(_json.dumps({'type': 'auth', 'token': token})) + else: + await upstream.send_str(_json.dumps({'type': 'auth', 'token': ''})) await publish_event( app, diff --git a/backend/open_webui/routers/tools.py b/backend/open_webui/routers/tools.py index 2e25d28ef3..12c3771420 100644 --- a/backend/open_webui/routers/tools.py +++ b/backend/open_webui/routers/tools.py @@ -27,11 +27,12 @@ from open_webui.models.tools import ( ) from open_webui.utils.access_control import ( filter_allowed_access_grants, - has_access, + has_connection_access, has_permission, ) from open_webui.utils.auth import get_admin_user, get_verified_user from open_webui.utils.plugin import ( + get_tool_contents_cache, get_tools_cache, get_tool_module_from_cache, load_tool_module_by_id, @@ -101,7 +102,7 @@ async def get_tools( ) # OpenAPI Tool Servers - server_access_grants = {} + server_connections = {} for server in await get_tool_servers(request): server_idx = server.get('idx', 0) connections = await Config.get('tool_server.connections', []) @@ -112,10 +113,8 @@ async def get_tools( ) continue connection = connections[server_idx] - server_config = connection.get('config', {}) - server_id = f'server:{server.get("id")}' - server_access_grants[server_id] = server_config.get('access_grants', []) + server_connections[server_id] = connection tools.append( ToolUserResponse( @@ -148,10 +147,8 @@ async def get_tools( user.id, f'mcp:{server_id}' ) - server_config = server.get('config') or {} - tool_id = f'server:mcp:{info.get("id")}' - server_access_grants[tool_id] = server_config.get('access_grants', []) + server_connections[tool_id] = server tools.append( ToolUserResponse( @@ -180,12 +177,10 @@ async def get_tools( tool for tool in tools if not str(tool.id).startswith('server:') - or await has_access( - user.id, - 'read', - server_access_grants.get(str(tool.id), []), + or await has_connection_access( + user, + server_connections[str(tool.id)], user_group_ids, - db=db, ) ] @@ -339,6 +334,7 @@ async def export_tools( return await Tools.get_tools( db=db, user_id=None if bypass_access_control else user.id, + permission='write', ) @@ -680,6 +676,8 @@ async def delete_tools_by_id( if result: TOOLS = get_tools_cache(request) TOOLS.pop(id, None) + TOOL_CONTENTS = get_tool_contents_cache(request) + TOOL_CONTENTS.pop(id, None) await publish_event( request, EVENTS.TOOL_DELETED, diff --git a/backend/open_webui/routers/users.py b/backend/open_webui/routers/users.py index d736c038bc..1391727ded 100644 --- a/backend/open_webui/routers/users.py +++ b/backend/open_webui/routers/users.py @@ -21,6 +21,7 @@ from open_webui.models.chats import Chats from open_webui.models.groups import Groups from open_webui.models.oauth_sessions import OAuthSessions from open_webui.models.users import ( + InterfaceSettings, UserGroupIdsListResponse, UserGroupIdsModel, UserInfoListResponse, @@ -68,23 +69,6 @@ def merge_user_ui_settings(defaults: dict, settings: dict) -> dict: return merged -def strip_default_interface_settings(defaults: dict, settings: dict) -> dict: - stripped = {} - for key, value in settings.items(): - if value is None: - continue - - default_value = defaults.get(key) - if isinstance(default_value, dict) and isinstance(value, dict): - nested = strip_default_interface_settings(default_value, value) - if nested: - stripped[key] = nested - elif value != default_value: - stripped[key] = value - - return stripped - - ############################ # GetUsers # A house is only as strong as its care for the least of @@ -502,16 +486,37 @@ async def update_user_settings_by_session_user( user=Depends(get_verified_user), db: AsyncSession = Depends(get_async_session), ): - if user.role != 'admin' and not await has_permission( - user.id, 'settings.interface', await Config.get('user.permissions') - ): - raise HTTPException( - status_code=status.HTTP_403_FORBIDDEN, - detail=ERROR_MESSAGES.ACCESS_PROHIBITED, - ) - updated_user_settings = form_data.model_dump(exclude_unset=True) ui_settings = updated_user_settings.get('ui') + + if isinstance(ui_settings, dict): + if user.role != 'admin' and not await has_permission( + user.id, 'settings.interface', await Config.get('user.permissions'), db=db + ): + # Omitted fields are unchanged, so unauthorized Interface fields can be discarded. + for key in InterfaceSettings.model_fields: + ui_settings.pop(key, None) + + if ( + user.role != 'admin' + and 'system' in ui_settings + and ( + not await has_permission(user.id, 'chat.controls', await Config.get('user.permissions'), db=db) + or not await has_permission(user.id, 'chat.system_prompt', await Config.get('user.permissions'), db=db) + ) + ): + ui_settings.pop('system', None) + + if ( + user.role != 'admin' + and 'params' in ui_settings + and ( + not await has_permission(user.id, 'chat.controls', await Config.get('user.permissions'), db=db) + or not await has_permission(user.id, 'chat.params', await Config.get('user.permissions'), db=db) + ) + ): + ui_settings.pop('params', None) + if ( user.role != 'admin' and ui_settings is not None @@ -522,8 +527,7 @@ async def update_user_settings_by_session_user( await Config.get('user.permissions'), ) ): - # If the user is not an admin and does not have permission to use tool servers, remove the key - updated_user_settings['ui'].pop('toolServers', None) + ui_settings.pop('toolServers', None) ui_notifications = ui_settings.get('notifications') if isinstance(ui_settings, dict) else None if ( @@ -542,11 +546,6 @@ async def update_user_settings_by_session_user( if isinstance(ui_notifications, dict): ui_notifications.pop('webhook_url', None) - default_interface_settings = await Config.get('ui.default_interface_settings') - ui_settings = updated_user_settings.get('ui') - if isinstance(default_interface_settings, dict) and isinstance(ui_settings, dict): - updated_user_settings['ui'] = strip_default_interface_settings(default_interface_settings, ui_settings) - user = await Users.update_user_settings_by_id(user.id, updated_user_settings, db=db) if user: await publish_event( diff --git a/backend/open_webui/routers/utils.py b/backend/open_webui/routers/utils.py index dcad54f4b1..48e9cc3e5c 100644 --- a/backend/open_webui/routers/utils.py +++ b/backend/open_webui/routers/utils.py @@ -3,15 +3,13 @@ from __future__ import annotations import logging import black -from fastapi import APIRouter, Depends, HTTPException, Request, Response, status +from fastapi import APIRouter, Depends, HTTPException, Request, status from open_webui.config import DATA_DIR, ENABLE_ADMIN_EXPORT from open_webui.constants import ERROR_MESSAGES -from open_webui.models.chats import ChatTitleMessagesForm from open_webui.models.config import Config from open_webui.utils.auth import get_admin_user, get_verified_user from open_webui.utils.code_interpreter import execute_code_jupyter from open_webui.utils.misc import get_gravatar_url -from open_webui.utils.pdf_generator import PDFGenerator from pydantic import BaseModel from starlette.responses import FileResponse @@ -73,26 +71,6 @@ async def execute_code(request: Request, form_data: CodeForm, user=Depends(get_v ) -class ChatForm(BaseModel): - title: str - messages: list[dict] - - -@router.post('/pdf') -async def download_chat_as_pdf(form_data: ChatTitleMessagesForm, user=Depends(get_verified_user)): - try: - pdf_bytes = PDFGenerator(form_data).generate_chat_pdf() - - return Response( - content=pdf_bytes, - media_type='application/pdf', - headers={'Content-Disposition': 'attachment;filename=chat.pdf'}, - ) - except Exception as e: - log.exception(f'Error generating PDF: {e}') - raise HTTPException(status_code=400, detail=str(e)) - - @router.get('/db/download') async def download_db(user=Depends(get_admin_user)): """Download the raw SQLite database file (admin-only, SQLite deployments only).""" diff --git a/backend/open_webui/socket/main.py b/backend/open_webui/socket/main.py index a0808e1ea2..29af592458 100644 --- a/backend/open_webui/socket/main.py +++ b/backend/open_webui/socket/main.py @@ -1,10 +1,12 @@ from __future__ import annotations import asyncio +import copy import logging import random import sys import time +from contextlib import suppress from typing import Any import pycrdt as Y @@ -37,17 +39,23 @@ from open_webui.models.folders import Folders from open_webui.models.notes import Notes, NoteUpdateForm from open_webui.models.users import UserNameResponse, Users from open_webui.socket.utils import CachedRedisDict, RedisDict, RedisLock, YdocManager -from open_webui.tasks import create_task, stop_item_tasks +from open_webui.tasks import ( + REDIS_PUBSUB_MAX_RECONNECT_INTERVAL, + REDIS_PUBSUB_RECONNECT_INTERVAL, + create_task, + stop_item_tasks, +) from open_webui.utils.access_control import has_permission from open_webui.utils.auth import get_verified_user_by_token from open_webui.utils.chat_id import is_saved_chat_id -from open_webui.utils.json_codec import SOCKETIO_JSON +from open_webui.utils.json_codec import SOCKETIO_JSON, JSONCodec, dumps_bytes from open_webui.utils.misc import get_output_text from open_webui.utils.redis import ( build_sentinel_url, get_redis_connection, get_sentinels_from_env, ) +from redis.exceptions import RedisError from socketio.packet import Packet logging.basicConfig(stream=sys.stdout, level=GLOBAL_LOG_LEVEL) @@ -188,6 +196,11 @@ YDOC_MANAGER = YdocManager( redis_key_prefix=f'{REDIS_KEY_PREFIX}:ydoc:documents', ) +REDIS_EVENT_CHANNEL = f'{REDIS_KEY_PREFIX}:direct_completion' + +EVENT_QUEUES: dict[str, asyncio.Queue] = {} +EVENT_PUBLISH_LOCK = asyncio.Lock() + def get_session_pool_batches(): """All session pool entries, in bounded batches for the Redis backing.""" @@ -201,49 +214,53 @@ async def periodic_session_pool_cleanup(): retry_delay = random.uniform(WEBSOCKET_REDIS_LOCK_TIMEOUT / 2, WEBSOCKET_REDIS_LOCK_TIMEOUT) renew_interval = max(WEBSOCKET_REDIS_LOCK_TIMEOUT / 2, 0.5) while True: - if not session_aquire_func(): - log.debug('Session cleanup lock held by another node. Retrying.') - await asyncio.sleep(retry_delay) - continue - try: - while True: - if not session_renew_func(): - log.warning('Unable to renew session cleanup lock. Retrying cleanup ownership.') - break + if not session_aquire_func(): + log.debug('Session cleanup lock held by another node. Retrying.') + await asyncio.sleep(retry_delay) + continue - now = int(time.time()) - for batch in get_session_pool_batches(): - expired = [ - sid - for sid, entry in batch - if entry and now - entry.get('last_seen_at', 0) > SESSION_POOL_TIMEOUT - ] - if expired: - log.warning('Reaping %d orphaned session(s) from the session pool', len(expired)) - if WEBSOCKET_MANAGER == 'redis': - SESSION_POOL.delete_many(*expired) - else: - for sid in expired: - SESSION_POOL.pop(sid, None) - await asyncio.sleep(0) # don't hold the loop for the whole sweep - - next_cleanup_at = time.monotonic() + SESSION_POOL_TIMEOUT - lock_lost = False + try: while True: - sleep_for = min(renew_interval, next_cleanup_at - time.monotonic()) - if sleep_for <= 0: - break - await asyncio.sleep(sleep_for) if not session_renew_func(): log.warning('Unable to renew session cleanup lock. Retrying cleanup ownership.') - lock_lost = True break - if lock_lost: - break - finally: - session_release_func() + now = int(time.time()) + for batch in get_session_pool_batches(): + expired = [ + sid + for sid, entry in batch + if entry and now - entry.get('last_seen_at', 0) > SESSION_POOL_TIMEOUT + ] + if expired: + log.warning('Reaping %d orphaned session(s) from the session pool', len(expired)) + if WEBSOCKET_MANAGER == 'redis': + SESSION_POOL.delete_many(*expired) + else: + for sid in expired: + SESSION_POOL.pop(sid, None) + await asyncio.sleep(0) # don't hold the loop for the whole sweep + + next_cleanup_at = time.monotonic() + SESSION_POOL_TIMEOUT + lock_lost = False + while True: + sleep_for = min(renew_interval, next_cleanup_at - time.monotonic()) + if sleep_for <= 0: + break + await asyncio.sleep(sleep_for) + if not session_renew_func(): + log.warning('Unable to renew session cleanup lock. Retrying cleanup ownership.') + lock_lost = True + break + + if lock_lost: + break + finally: + session_release_func() + except Exception: + log.exception('Session pool cleanup failed. Retrying.') + await asyncio.sleep(retry_delay) async def periodic_usage_pool_cleanup(): @@ -822,27 +839,28 @@ async def yjs_document_update(sid, data): log.warning(f'User {user.get("id")} does not have write access to note {note_id}. Rejecting update.') return - user_id = data.get('user_id', sid) + update = data.get('update') # List of bytes from frontend - update = data['update'] # List of bytes from frontend + if update: + user_id = data.get('user_id', sid) - await YDOC_MANAGER.append_to_updates( - document_id=document_id, - update=update, # Convert list of bytes to bytes - ) + await YDOC_MANAGER.append_to_updates( + document_id=document_id, + update=update, # Convert list of bytes to bytes + ) - # Broadcast update to all other users in the document - await sio.emit( - 'ydoc:document:update', - { - 'document_id': document_id, - 'user_id': user_id, - 'update': update, - 'socket_id': sid, # Add socket_id to match frontend filtering - }, - room=f'doc_{document_id}', - skip_sid=sid, - ) + # Broadcast update to all other users in the document + await sio.emit( + 'ydoc:document:update', + { + 'document_id': document_id, + 'user_id': user_id, + 'update': update, + 'socket_id': sid, # Add socket_id to match frontend filtering + }, + room=f'doc_{document_id}', + skip_sid=sid, + ) async def debounced_save(): await asyncio.sleep(0.5) @@ -942,6 +960,62 @@ async def disconnect(sid, reason=None): # print(f"Unknown session ID {sid} disconnected") +async def redis_event_listener() -> None: + """Route events received over Redis to their local queues.""" + reconnect_interval = REDIS_PUBSUB_RECONNECT_INTERVAL + + while True: + pubsub = None + try: + # RedisCluster can't route a pubsub subscribe until initialize() fills its slot cache. + await REDIS.initialize() + + pubsub = REDIS.pubsub() + await pubsub.subscribe(REDIS_EVENT_CHANNEL) + reconnect_interval = REDIS_PUBSUB_RECONNECT_INTERVAL + + async for message in pubsub.listen(): + if message['type'] != 'message': + continue + event = JSONCodec.loads(message['data']) + queue = EVENT_QUEUES.get(event['channel']) + if queue is not None: + await queue.put(event['data']) + log.warning('Redis event listener stopped. Retrying.') + except asyncio.CancelledError: + raise + except Exception: + log.exception('Redis event listener failed. Retrying.') + finally: + if pubsub: + with suppress(Exception): + await pubsub.aclose() + + await asyncio.sleep(reconnect_interval) + reconnect_interval = min(reconnect_interval * 2, REDIS_PUBSUB_MAX_RECONNECT_INTERVAL) + + +@sio.on('*') +async def socket_event_handler(event: Any, sid: str, *args: Any) -> None: + """Route user-owned stream events to a local queue or another worker.""" + if not isinstance(event, str) or event.count(':') != 2 or not args: + return + + user = await get_socket_session_user(sid) + if not user or user.get('id') != event.split(':', 1)[0]: + return + + queue = EVENT_QUEUES.get(event) + if queue is not None: + await queue.put(args[0]) + elif WEBSOCKET_MANAGER == 'redis': + try: + async with EVENT_PUBLISH_LOCK: + await REDIS.publish(REDIS_EVENT_CHANNEL, dumps_bytes({'channel': event, 'data': args[0]})) + except RedisError as e: + log.debug('Failed to relay socket event %s: %s', event, e) + + async def _make_channel_emitter(request_info): """Event emitter that routes pipeline output to a channel message. @@ -1005,6 +1079,9 @@ async def _make_channel_emitter(request_info): if not content and not output and not done: return + if isinstance(output, list): + state['output'] = copy.deepcopy(output) + now = time.time() if done or (now - state['last_emit_at']) >= THROTTLE_INTERVAL: state['last_emit_at'] = now diff --git a/backend/open_webui/socket/utils.py b/backend/open_webui/socket/utils.py index 3ce9369910..f976a94e77 100644 --- a/backend/open_webui/socket/utils.py +++ b/backend/open_webui/socket/utils.py @@ -3,12 +3,16 @@ from __future__ import annotations import hashlib +import logging import uuid import pycrdt as Y from open_webui.env import REDIS_KEY_PREFIX from open_webui.utils.json_codec import JSONCodec from open_webui.utils.redis import get_redis_connection +from redis.exceptions import RedisClusterException, RedisError + +log = logging.getLogger(__name__) YDOC_KEY_PREFIX = f'{REDIS_KEY_PREFIX}:ydoc:documents' SCAN_BATCH_SIZE = 200 @@ -58,7 +62,10 @@ class RedisLock: return bool(self.redis.eval(self._RENEW_SCRIPT, 1, self.lock_name, self.lock_id, self.timeout_secs)) def release_lock(self): - self.redis.eval(self._RELEASE_SCRIPT, 1, self.lock_name, self.lock_id) + try: + self.redis.eval(self._RELEASE_SCRIPT, 1, self.lock_name, self.lock_id) + except (RedisClusterException, RedisError) as e: + log.warning('Failed to release lock %s; it expires on its own: %s', self.lock_name, e) class RedisDict: diff --git a/backend/open_webui/static/assets/pdf-style.css b/backend/open_webui/static/assets/pdf-style.css deleted file mode 100644 index 644dd58ae6..0000000000 --- a/backend/open_webui/static/assets/pdf-style.css +++ /dev/null @@ -1,315 +0,0 @@ -/* HTML and Body */ -@font-face { - font-family: 'NotoSans'; - src: url('fonts/NotoSans-Variable.ttf'); -} - -@font-face { - font-family: 'NotoSansJP'; - src: url('fonts/NotoSansJP-Variable.ttf'); -} - -@font-face { - font-family: 'NotoSansKR'; - src: url('fonts/NotoSansKR-Variable.ttf'); -} - -@font-face { - font-family: 'NotoSansSC'; - src: url('fonts/NotoSansSC-Variable.ttf'); -} - -@font-face { - font-family: 'NotoSansSC-Regular'; - src: url('fonts/NotoSansSC-Regular.ttf'); -} - -html { - font-family: - -apple-system, BlinkMacSystemFont, 'Segoe UI', 'NotoSans', 'NotoSansJP', 'NotoSansKR', - 'NotoSansSC', 'Twemoji', 'STSong-Light', 'MSung-Light', 'HeiseiMin-W3', 'HYSMyeongJo-Medium', - Roboto, 'Helvetica Neue', Arial, sans-serif; - font-size: 14px; /* Default font size */ - line-height: 1.5; -} - -*, -*::before, -*::after { - box-sizing: inherit; -} - -body { - margin: 0; - padding: 0; - background-color: #fff; - width: auto; -} - -/* Typography */ -h1, -h2, -h3, -h4, -h5, -h6 { - font-weight: 500; - margin: 0; -} - -h1 { - font-size: 2.5rem; -} - -h2 { - font-size: 2rem; -} - -h3 { - font-size: 1.75rem; -} - -h4 { - font-size: 1.5rem; -} - -h5 { - font-size: 1.25rem; -} - -h6 { - font-size: 1rem; -} - -p { - margin-top: 0; - margin-bottom: 1rem; -} - -/* Grid System */ -.container { - width: 100%; - padding-right: 15px; - padding-left: 15px; - margin-right: auto; - margin-left: auto; -} - -/* Utilities */ -.text-center { - text-align: center; -} - -/* Additional Text Utilities */ -.text-muted { - color: #6c757d; /* Muted text color */ -} - -/* Small Text */ -small { - font-size: 80%; /* Smaller font size relative to the base */ - color: #6c757d; /* Lighter text color for secondary information */ - margin-bottom: 0; - margin-top: 0; -} - -/* Strong Element Styles */ -strong { - font-weight: bolder; /* Ensures the text is bold */ - color: inherit; /* Inherits the color from its parent element */ -} - -/* link */ -a { - color: #007bff; - text-decoration: none; - background-color: transparent; -} - -a:hover { - color: #0056b3; - text-decoration: underline; -} - -/* General styles for lists */ -ol, -ul, -li { - padding-left: 40px; /* Increase padding to move bullet points to the right */ - margin-left: 20px; /* Indent lists from the left */ -} - -/* Ordered list styles */ -ol { - list-style-type: decimal; /* Use numbers for ordered lists */ - margin-bottom: 10px; /* Space after each list */ -} - -ol li { - margin-bottom: 0.5rem; /* Space between ordered list items */ -} - -/* Unordered list styles */ -ul { - list-style-type: disc; /* Use bullets for unordered lists */ - margin-bottom: 10px; /* Space after each list */ -} - -ul li { - margin-bottom: 0.5rem; /* Space between unordered list items */ -} - -/* List item styles */ -li { - margin-bottom: 5px; /* Space between list items */ - line-height: 1.5; /* Line height for better readability */ -} - -/* Nested lists */ -ol ol, -ol ul, -ul ol, -ul ul { - padding-left: 20px; - margin-left: 30px; /* Further indent nested lists */ - margin-bottom: 0; /* Remove extra margin at the bottom of nested lists */ -} - -/* Code blocks */ -pre { - background-color: #f4f4f4; - padding: 10px; - overflow-x: auto; - max-width: 100%; /* Ensure it doesn't overflow the page */ - width: 80%; /* Set a specific width for a container-like appearance */ - margin: 0 1em; /* Center the pre block */ - box-sizing: border-box; /* Include padding in the width */ - border: 1px solid #ccc; /* Optional: Add a border for better definition */ - border-radius: 4px; /* Optional: Add rounded corners */ -} - -code { - font-family: 'Courier New', Courier, monospace; - background-color: #f4f4f4; - padding: 2px 4px; - border-radius: 4px; - box-sizing: border-box; /* Include padding in the width */ -} - -.message { - margin-top: 8px; - margin-bottom: 8px; - max-width: 100%; - overflow-wrap: break-word; -} - -/* Table Styles */ -table { - width: 100%; - margin-bottom: 1rem; - color: #212529; - border-collapse: collapse; /* Removes the space between borders */ -} - -th, -td { - margin: 0; - padding: 0.75rem; - vertical-align: top; - border-top: 1px solid #dee2e6; -} - -thead th { - vertical-align: bottom; - border-bottom: 2px solid #dee2e6; -} - -tbody + tbody { - border-top: 2px solid #dee2e6; -} - -/* markdown-section styles */ -.markdown-section blockquote, -.markdown-section h1, -.markdown-section h2, -.markdown-section h3, -.markdown-section h4, -.markdown-section h5, -.markdown-section h6, -.markdown-section p, -.markdown-section pre, -.markdown-section table, -.markdown-section ul { - /* Give most block elements margin top and bottom */ - margin-top: 1rem; -} - -/* Remove top margin if it's the first child */ -.markdown-section blockquote:first-child, -.markdown-section h1:first-child, -.markdown-section h2:first-child, -.markdown-section h3:first-child, -.markdown-section h4:first-child, -.markdown-section h5:first-child, -.markdown-section h6:first-child, -.markdown-section p:first-child, -.markdown-section pre:first-child, -.markdown-section table:first-child, -.markdown-section ul:first-child { - margin-top: 0; -} - -/* Remove top margin of