open-webui/backend
Álvaro Justen (@turicas) 28a4df7921 fix: honor requests_per_second in FireCrawl loader
`SafeFireCrawlLoader` inherits `RateLimitMixin` and accepts
`requests_per_second` in its constructor, but neither `lazy_load` nor
`alazy_load` ever invoked `_sync_wait_for_rate_limit` /
`_wait_for_rate_limit`. The configured value (env
`WEB_LOADER_CONCURRENT_REQUESTS`) was silently ignored: the loader fired
requests as fast as the loop ran.

`_safe_process_url` is not used because it's redundant for FireCrawl:
FireCrawl fetches the target URL on the user's behalf, and `verify_ssl`
is already honored in the FireCrawl payload (`skipTlsVerification`).
2026-04-27 19:55:01 -03:00
..
data refac: mv backend files to /open_webui dir 2024-09-04 16:54:48 +02:00
open_webui fix: honor requests_per_second in FireCrawl loader 2026-04-27 19:55:01 -03:00
.dockerignore fix: litellm config issue 2024-02-24 22:35:11 -08:00
.gitignore refac 2024-09-06 04:59:20 +02:00
dev.sh refac 2026-03-24 19:43:30 -05:00
requirements-min.txt refac 2026-04-24 18:20:10 +09:00
requirements.txt refactor(firecrawl): use v2 API directly (#23934) 2026-04-24 18:32:08 +09:00
start.sh refac 2026-03-24 19:43:30 -05:00
start_windows.bat refac 2026-04-24 15:40:02 +09:00