mirror of
https://github.com/open-webui/open-webui.git
synced 2026-09-23 00:41:56 +00:00
28a4df7921
`SafeFireCrawlLoader` inherits `RateLimitMixin` and accepts `requests_per_second` in its constructor, but neither `lazy_load` nor `alazy_load` ever invoked `_sync_wait_for_rate_limit` / `_wait_for_rate_limit`. The configured value (env `WEB_LOADER_CONCURRENT_REQUESTS`) was silently ignored: the loader fired requests as fast as the loop ran. `_safe_process_url` is not used because it's redundant for FireCrawl: FireCrawl fetches the target URL on the user's behalf, and `verify_ssl` is already honored in the FireCrawl payload (`skipTlsVerification`). |
||
|---|---|---|
| .. | ||
| testdata | ||
| azure.py | ||
| bing.py | ||
| bocha.py | ||
| brave.py | ||
| duckduckgo.py | ||
| exa.py | ||
| external.py | ||
| firecrawl.py | ||
| google_pse.py | ||
| jina_search.py | ||
| kagi.py | ||
| main.py | ||
| mojeek.py | ||
| ollama.py | ||
| perplexity.py | ||
| perplexity_search.py | ||
| searchapi.py | ||
| searxng.py | ||
| serpapi.py | ||
| serper.py | ||
| serply.py | ||
| serpstack.py | ||
| sougou.py | ||
| tavily.py | ||
| utils.py | ||
| yacy.py | ||
| yandex.py | ||
| ydc.py | ||