open-webui/backend/open_webui/retrieval/web
Classic298 087878ce84
Match WEB_FETCH_FILTER_LIST on hostnames with label boundaries, not URL suffix (CWE-693) (#25949)
is_string_allowed does endswith() matching and was called with the full URL
(retrieval/web/utils.py) against WEB_FETCH_FILTER_LIST, so a blocklisted host with any
path (https://blocked.example/x) ended with /x, not the host, and slipped through; the
allowlist direction false-rejected legitimate URLs and admitted attacker URLs ending in
an allowed string. The same endswith caused label confusion at the hostname call site
(retrieval/web/main.py): corp.com matched evilcorp.com, 10.0.0.1 matched 110.0.0.1.

Add is_host_allowed(host, ...) matching on DNS label boundaries (host == pattern or
host.endswith('.' + pattern)), called with the parsed hostname at both web-fetch call
sites. is_string_allowed is left unchanged for the unrelated function-name filters
(utils/middleware.py, utils/tools.py).

The separate is_global guard (validate_url / _ssrf_safe_new_conn, active when
ENABLE_RAG_LOCAL_WEB_FETCH is off) already blocks RFC1918/loopback/link-local, so this
restores the admin's intended blocking of specific public hosts.

Co-authored-by: addcontent <59762500+addcontent@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:53:08 +02:00
..
azure.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
bing.py chore: format 2026-06-01 13:56:55 -07:00
bocha.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
brave.py refac 2026-05-21 16:44:36 +04:00
brave_llm_context.py feat: brave search llm context 2026-05-09 06:34:25 +09:00
duckduckgo.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
exa.py refac 2026-03-17 17:58:01 -05:00
external.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
firecrawl.py fix: handle list-shape data in Firecrawl /search response (#24712) 2026-06-01 12:07:31 -07:00
google_pse.py refac 2026-05-21 16:44:36 +04:00
jina_search.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
kagi.py Update Kagi API endpoint and request method (#25015) 2026-06-01 12:48:44 -07:00
linkup.py chore: format 2026-06-01 13:56:55 -07:00
main.py Match WEB_FETCH_FILTER_LIST on hostnames with label boundaries, not URL suffix (CWE-693) (#25949) 2026-06-16 23:53:08 +02:00
mojeek.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
ollama.py refac 2026-03-17 17:58:01 -05:00
perplexity.py feat(retrieval): add Perplexity attribution header (#24833) 2026-06-01 13:40:52 -07:00
perplexity_search.py feat(retrieval): add Perplexity attribution header (#24833) 2026-06-01 13:40:52 -07:00
searchapi.py refac 2026-03-17 17:58:01 -05:00
searxng.py refac 2026-05-21 16:44:36 +04:00
serpapi.py refac 2026-03-17 17:58:01 -05:00
serper.py refac 2026-05-21 16:44:36 +04:00
serply.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
serpstack.py refac 2026-05-21 16:44:36 +04:00
sougou.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
tavily.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
utils.py Match WEB_FETCH_FILTER_LIST on hostnames with label boundaries, not URL suffix (CWE-693) (#25949) 2026-06-16 23:53:08 +02:00
yacy.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
yandex.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
ydc.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00