open-webui/backend/open_webui/retrieval/loaders
Classic298 c7f306031d
refactor: remove the unreachable async half of the Mistral loader and an unused Datalab helper (#28839)
The Mistral OCR loader has a full async pipeline beside its synchronous one: an async load, its own upload, signed URL, OCR, delete and retry helpers, a pooled session and a batch loader on top. The only way in was the batch loader, which nothing calls, so the entire async half was unreachable. Everything that loads documents goes through the synchronous path, and the shared loader entry point runs it in a worker thread. The Datalab loader carries a public request status poller with no caller either, since its own load inlines the polling it needs.

With the async half gone, the retry classifier's two aiohttp branches can no longer be reached, since the only retried calls are synchronous, so those go with it along with the aiohttp import that existed solely to feed them, and a timeout attribute that nothing reads any more. The class docstring loses the three bullets that only described the removed pipeline, and four docstrings stop calling themselves the sync version of something that no longer has an async counterpart.

This removes around 350 lines and leaves one code path per loader instead of one live path and one that cannot be entered.
2026-08-20 13:14:14 -07:00
..
datalab_marker.py refactor: remove the unreachable async half of the Mistral loader and an unused Datalab helper (#28839) 2026-08-20 13:14:14 -07:00
external_document.py feat: add {{USER_GROUPS}} and {{USER_GROUP_IDS}} placeholders for custom forwarded headers (#27236) 2026-07-26 23:21:26 -04:00
external_web.py refac 2026-07-27 19:39:36 -04:00
main.py refac 2026-08-10 00:26:44 -06:00
microsoft_web_iq.py refac 2026-06-29 01:52:07 -05:00
mineru.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
mistral.py refactor: remove the unreachable async half of the Mistral loader and an unused Datalab helper (#28839) 2026-08-20 13:14:14 -07:00
paddleocr_vl.py perf: build info log messages lazily so raising the log level actually saves work (#27837) 2026-08-02 15:39:10 -05:00
tavily.py refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
youtube.py fix(retrieval): report why a URL could not be read instead of blaming the knowledge base (#28362) 2026-08-10 19:18:04 -06:00