mirror of
https://github.com/zotero/zotero.git
synced 2026-10-11 03:38:25 +00:00
2568b5ccc4
Add optional pref to index and search fulltext of attachments. When enabled, attachment IDs are enqueue after regular items, notes, and annotations. Added helpers to extract outline and sections from structured text module. During indexing, the sections of the attachment are extracted, large sections are broken into chunks, and small sections are combined to fit into the context window of the model. Then, each chunk is embedded with its outline path as the prefix and added to embeddings table. Each row now contains full text of the chunk and it's path - a good amount of duplication needed to ensure that we can reliably connect the embedding of the chunk to its text for a preview. On search in Best Match mode, attachment rows get the score of the highest ranking chunk, so if there is a very relevant chunk in an attachment, the regular item with a non-relevant abstract will still rank highly. When the attachment row is selected, top 5 matching chunks appear in the new search results collapsible-section of the item pane, so one can examine matching chunks without opening the actual reader. |
||
|---|---|---|
| .. | ||
| components | ||
| content | ||
| resource | ||
| tests | ||
| chrome.manifest | ||
| runtests.sh | ||