open-webui/backend
mia 9d3d80b9f8 fix: extract text from PDF URLs in fetch_url tool
When fetch_url fetches a PDF URL, the content was returned as garbled
binary because BeautifulSoup's HTML parser corrupted the PDF bytes.

Add three-layer PDF detection:
- Fast-path for URLs ending in .pdf (case-insensitive)
- Content-Type header detection in async SafeWebBaseLoader._fetch
- %PDF binary content fallback in get_content_from_url

Includes SSRF protection via validate_url(), streaming download with
Content-Length pre-check, 50 MB size limit, and proper session
management. No new dependencies (pypdf already in requirements.txt).

Closes #23841

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 11:07:02 +05:30
..
data refac: mv backend files to /open_webui dir 2024-09-04 16:54:48 +02:00
open_webui fix: extract text from PDF URLs in fetch_url tool 2026-04-19 11:07:02 +05:30
tests/retrieval fix: extract text from PDF URLs in fetch_url tool 2026-04-19 11:07:02 +05:30
.dockerignore fix: litellm config issue 2024-02-24 22:35:11 -08:00
.gitignore refac 2024-09-06 04:59:20 +02:00
dev.sh refac 2026-03-24 19:43:30 -05:00
requirements-min.txt refac 2026-04-13 23:40:09 -05:00
requirements.txt refac 2026-04-13 23:40:09 -05:00
start.sh refac 2026-03-24 19:43:30 -05:00
start_windows.bat refac 2026-03-24 19:43:30 -05:00