mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-16 23:43:03 +00:00
ae01ef9c95
Fetching a page with many media files through the Playwright web loader was extremely slow or timed out, while raw Playwright loaded the same page in a couple of seconds. The loader routes every request the page makes through the backend HTTP client and downloads the full body before the browser sees any of it, and the browser cancelling a media request once it has enough never reaches that download. On a page with a few dozen audio players every file was pulled in full for a text extraction that never reads it. Image, media and font requests are now aborted in the interceptor before any fetch is made. None of them feed the text extraction. Measured on the page from the report with the default timeout on the same connection: | | requests fetched | bytes downloaded | elapsed | |---|---|---|---| | before | 151 | 55.0 MB | 10.1 s | | after | 45 | 3.9 MB | 2.6 s | Fixes #29741 |
||
|---|---|---|
| .. | ||
| azure.py | ||
| bing.py | ||
| bocha.py | ||
| brave.py | ||
| brave_llm_context.py | ||
| duckduckgo.py | ||
| exa.py | ||
| external.py | ||
| firecrawl.py | ||
| google_pse.py | ||
| jina_search.py | ||
| kagi.py | ||
| linkup.py | ||
| main.py | ||
| microsoft_web_iq.py | ||
| mojeek.py | ||
| ollama.py | ||
| openserp.py | ||
| perplexity.py | ||
| perplexity_search.py | ||
| searchapi.py | ||
| searxng.py | ||
| serpapi.py | ||
| serper.py | ||
| serphouse.py | ||
| serply.py | ||
| serpstack.py | ||
| sougou.py | ||
| tavily.py | ||
| utils.py | ||
| yacy.py | ||
| yandex.py | ||
| ydc.py | ||