open-webui/backend/open_webui/retrieval/web
Classic298 854440f703
fix: mitigate DNS rebinding in web loader fetch paths (#24759)
validate_url() resolves DNS to check IPs but discards the result; the
HTTP client resolves again independently.  Between those two lookups an
attacker can swap the DNS record from a public IP to an internal one
(DNS rebinding).

Push the IP-is-global check into the actual connection layer so the
validated resolution is the one used for the TCP connect:

- aiohttp (_fetch): _SSRFSafeResolver wraps DefaultResolver and rejects
  non-global IPs at resolve time (zero TOCTOU window).
- requests (_scrape): _SSRFSafeAdapter mounts custom urllib3 connection
  classes whose _new_conn resolves, validates, and connects to the
  validated IP in one shot (zero TOCTOU window).

Both paths respect ENABLE_RAG_LOCAL_WEB_FETCH (skip validation when on).

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-19 23:57:12 +04:00
..
azure.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
bing.py fix: pass subscription_key and endpoint in bing.py CLI search_bing() call (#24768) 2026-05-19 21:26:10 +04:00
bocha.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
brave.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
brave_llm_context.py feat: brave search llm context 2026-05-09 06:34:25 +09:00
duckduckgo.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
exa.py refac 2026-03-17 17:58:01 -05:00
external.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
firecrawl.py refactor(firecrawl): use v2 API directly (#23934) 2026-04-24 18:32:08 +09:00
google_pse.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
jina_search.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
kagi.py refac 2026-03-17 17:58:01 -05:00
linkup.py enh: linkup 2026-05-19 21:54:38 +04:00
main.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
mojeek.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
ollama.py refac 2026-03-17 17:58:01 -05:00
perplexity.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
perplexity_search.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
searchapi.py refac 2026-03-17 17:58:01 -05:00
searxng.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
serpapi.py refac 2026-03-17 17:58:01 -05:00
serper.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
serply.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
serpstack.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
sougou.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
tavily.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
utils.py fix: mitigate DNS rebinding in web loader fetch paths (#24759) 2026-05-19 23:57:12 +04:00
yacy.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
yandex.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
ydc.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00