mirror of
https://github.com/zotero/zotero.git
synced 2026-10-11 03:38:25 +00:00
628fd23069
Add ftindex.fulltextItemText, an FTS5 table (plus CJK 2-gram twin and
state table) holding each item's searchable text in per-type columns:
title and abstract for regular items, note text for notes, passage and
comment for annotations. One row per itemID; text stored normalized.
- Regular items and annotations index inline in their save transaction;
notes are written by the existing stale-flag queue in the same step
that feeds the trigram tables. Erase clears the entries.
- Backfill queue (processItemTextIndexQueue) covers pre-existing items
after the index rebuild, wired into the startup and background
drains; feed libraries are excluded. _indexDBVersion bumped to 3.
- Advanced prefs: "Items and annotations indexed" line in Index
Statistics; the item-text queue joins the progress/up-to-date logic
and drains while the pane is open.
Why an index: ranked search needs two things per query term that no
existing structure can answer -- whole-word membership ("fall" must not
match "rainfall") and per-word document counts, which drive term
weighting ("communism" outranks "fall" by rarity). The trigram note
index answers substrings, not words, and inflates counts; scanning
item text in JS costs a pass over the whole library per query and
can't prefilter without dropping diacritic matches. Word-level FTS
answers both with index probes, normalized at write time.
|
||
|---|---|---|
| .. | ||
| components | ||
| content | ||
| resource | ||
| tests | ||
| chrome.manifest | ||
| runtests.sh | ||