open-webui/backend/open_webui/retrieval/web
Classic298 4ef7e35b88
fix: Playwright web loader returns only the site menu for pages with more than one <main> element (#31644)
Some pages, like the Ubiquiti tech specs pages, have more than one <main> element. The Playwright web loader only read the first one, so these pages came back as just their site menu and the actual content was lost. When a page has more than one, the loader now ignores those tags and reads the whole page. Pages with a single <main> load the same as before.

Fixes #28643
2026-09-30 21:06:01 +04:00
..
azure.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
bing.py chore: format 2026-06-01 13:56:55 -07:00
bocha.py refac 2026-07-31 17:41:14 -04:00
brave.py refac 2026-08-31 00:40:15 -04:00
brave_llm_context.py feat: brave search llm context 2026-05-09 06:34:25 +09:00
duckduckgo.py refac 2026-09-06 17:21:57 -04:00
exa.py refac 2026-09-08 23:18:48 -04:00
external.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
firecrawl.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
google_pse.py refac 2026-08-31 00:40:15 -04:00
jina_search.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
kagi.py Update Kagi API endpoint and request method (#25015) 2026-06-01 12:48:44 -07:00
linkup.py chore: format 2026-06-01 13:56:55 -07:00
main.py refac: consolidate the web fetch address checks onto the request path (#27823) 2026-08-25 11:15:48 -04:00
microsoft_web_iq.py refac 2026-08-31 00:40:15 -04:00
mojeek.py refac 2026-08-31 00:39:16 -04:00
ollama.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
openserp.py refac 2026-08-31 00:40:15 -04:00
perplexity.py refac 2026-06-17 02:52:35 +02:00
perplexity_search.py refac 2026-06-17 02:52:35 +02:00
searchapi.py fix: surface searchapi errors, news results and redirect links (#30308) 2026-09-21 10:44:06 -04:00
searxng.py refac 2026-08-31 00:40:15 -04:00
serpapi.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
serper.py refac 2026-08-31 00:40:15 -04:00
serphouse.py refac 2026-08-31 00:40:15 -04:00
serply.py refac 2026-08-31 00:39:16 -04:00
serpstack.py refac 2026-08-31 00:40:15 -04:00
sougou.py chore: format 2026-08-25 16:53:53 -04:00
staan.py fix: stop sending count to the Staan search API (#30303) 2026-09-21 08:04:26 -04:00
tavily.py feat: add a Tavily search depth setting (#31308) 2026-09-27 23:15:09 +04:00
utils.py fix: Playwright web loader returns only the site menu for pages with more than one <main> element (#31644) 2026-09-30 21:06:01 +04:00
yacy.py perf: build debug log messages lazily so disabled debug logs cost nothing (#27834) 2026-07-31 19:09:01 -05:00
yandex.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
ydc.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00