open-webui/backend/open_webui/utils
Classic298 7fa705f3b8
feat: let operators expose chosen file metadata to the model in retrieved sources (#29696)
Custom metadata attached to a file upload now reaches the vector DB, but the model still never sees it. Both prompt-assembly paths build their output from a fixed field set: the classic RAG <source> tag carries only id, name and resource type, and the retrieval tools return only content, source and file id per chunk. A scraper that records where each document came from therefore cannot get that origin in front of the model, so answers cannot state it.

RAG_SOURCE_METADATA_KEYS names the chunk metadata keys allowed through to the model. Configured keys are emitted as extra attributes on the <source> tag and as extra fields on tool result chunks, covering both retrieval paths. It is empty by default, so nothing changes for existing deployments.

An allowlist instead of passing everything through, because chunk metadata also carries file hashes, collection names, embedding config and relevance scores, which would then be added to every retrieved chunk of every request. Values are attacker-controllable through an uploaded file, so they are escaped before they go into the tag, and a configured key can never displace a field the tag or the chunk already defines.

Reported in open-webui/open-webui#29486.
2026-09-19 17:02:59 -05:00
..
access_control refac: resolve knowledge file access from the file association (#29937) 2026-09-16 15:59:25 -04:00
images perf: build debug log messages lazily so disabled debug logs cost nothing (#27834) 2026-07-31 19:09:01 -05:00
mcp refac 2026-09-12 14:57:22 -04:00
telemetry refac(telemetry): drop deprecated semconv SpanAttributes subclass (#25784) 2026-06-29 02:05:34 -05:00
actions.py fix: enforce action availability and model access on the chat action route (#27243) 2026-07-23 12:23:05 -04:00
anthropic.py fix: drop thinking blocks when converting Anthropic Messages requests to Chat Completions (#29849) 2026-09-12 15:56:59 -05:00
asgi_middleware.py refac 2026-08-24 18:29:36 -04:00
ask_user.py fix: a rejected ask_user call ending the turn with no reply (#29252) 2026-08-31 00:17:21 -04:00
audit.py perf: build debug log messages lazily so disabled debug logs cost nothing (#27834) 2026-07-31 19:09:01 -05:00
auth.py refac 2026-09-19 17:54:09 -04:00
automations.py refac 2026-09-12 16:56:26 -04:00
calendar.py refac 2026-08-29 16:14:32 -04:00
channels.py refac 2026-03-17 17:58:01 -05:00
chat.py fix: surface upstream errors on arena models instead of crashing (#29662) 2026-09-04 17:17:02 -04:00
chat_fork.py refac 2026-07-23 02:54:56 -04:00
chat_id.py refac 2026-07-26 21:12:14 -04:00
chat_variables.py perf: stabilize the model registry signature across workers (#29264) 2026-08-30 16:12:31 -04:00
code_interpreter.py refac 2026-07-31 17:41:14 -04:00
context_compaction.py refac 2026-08-23 13:36:53 -04:00
embeddings.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
files.py refac 2026-09-13 21:37:12 -04:00
filter.py refac 2026-08-31 01:29:36 -04:00
groups.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
headers.py refac 2026-09-12 15:41:57 -04:00
json_codec.py perf: write task payloads to Redis as bytes (#28833) 2026-08-20 12:58:52 -07:00
json_response.py perf: optional orjson JSON codec behind ENABLE_ORJSON (#27583) 2026-07-27 03:45:37 -04:00
logger.py perf: stop formatting every exported log record twice under OTEL log export (#27840) 2026-08-10 23:13:52 -06:00
memory.py refac 2026-08-10 19:28:21 -06:00
middleware.py feat: let operators expose chosen file metadata to the model in retrieved sources (#29696) 2026-09-19 17:02:59 -05:00
misc.py fix: treat SVG uploads as documents instead of vision images (#30102) 2026-09-17 17:40:07 -04:00
model_ids.py refac 2026-07-26 23:09:22 -04:00
models.py refac 2026-09-07 12:40:44 -04:00
notifications.py fix: sanitize user-typed notification target ids the same way generated ones are (#29947) 2026-09-12 13:50:19 -05:00
oauth.py refac 2026-09-19 17:26:18 -04:00
payload.py refac 2026-08-16 22:56:19 -07:00
plugin.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
rate_limit.py fix: stop the sign-in rate limiter blocking the loop and leaking memory (#29977) 2026-09-13 20:28:41 -05:00
redis.py fix: Set default Redis socket timeout to None (#27104) 2026-07-27 00:30:00 -04:00
response.py refac 2026-08-17 00:57:57 -07:00
sanitize.py refac 2026-03-17 17:58:01 -05:00
security_headers.py refac 2026-08-24 18:29:36 -04:00
session_pool.py fix: long streamed lines no longer abort the response (#28114) 2026-08-25 12:16:37 -04:00
skills.py refac 2026-09-14 17:57:09 -04:00
subagents.py chore: format 2026-08-25 16:53:53 -04:00
task.py refac 2026-06-19 00:16:06 +02:00
terminals.py refac 2026-09-19 17:33:41 -04:00
timers.py fix: index the chat queries that make large SQLite instances unusable (#27663) 2026-08-23 16:11:54 -05:00
tool_approval.py chore: format 2026-08-25 16:53:53 -04:00
tools.py refac 2026-09-07 12:16:09 -04:00
validate.py refac 2026-09-15 22:47:03 -04:00
valves.py refac 2026-07-31 17:41:14 -04:00
webhook.py perf: build debug log messages lazily so disabled debug logs cost nothing (#27834) 2026-07-31 19:09:01 -05:00