Commit graph

3 commits

Author SHA1 Message Date
Dan Stillman
a5bff10865 Don't use FSEvents when storage isn't on a local volume
Some checks are pending
CI / Detect changes (push) Waiting to run
CI / Test () (push) Blocked by required conditions
CI / Test (macOS NFS) (push) Blocked by required conditions
CI / Utilities Tests (push) Waiting to run
CI / Build, Upload (push) Waiting to run
FSEvents is backed by a per-volume journal that only local volumes
have. On a network mount the stream is created and started
successfully but never delivers events, so the watcher would report
that nothing had changed for as long as it was used. Check the volume
with statfs() and fall back to scanning.

(cherry picked from commit 5d20c692e8)
2026-08-26 10:37:48 -04:00
Dan Stillman
06e16c297c Fix file-change detection in all libraries after the first one synced
Some checks failed
CI / Build, Upload, Test (push) Has been cancelled
The local-file-change watcher added in f21e1b2d32 accumulated changed
item keys globally but was drained separately by each library's storage
engine, with the drained keys filtered to that library. The first
library to file-sync (normally My Library) consumed all pending events,
and keys belonging to other libraries were silently discarded, so files
modified on disk in group libraries were never marked for upload. The
initial and periodic full-scan fallbacks on Windows and Linux were
likewise global, so only the first library ever received them, and
changes made in other libraries while Zotero was closed were never
detected at all.

The sync runner now drains the watcher once per sync session and
immediately runs the modification check on the changed items across all
libraries, recording any changes in the database, and the per-library
storage engines skip the check entirely unless the watcher reports that
the library needs a full scan:

- On all platforms, a library that has never been scanned gets one full
  scan, which also gives libraries one recovery scan for changes dropped
  by affected releases.
- On Windows/Linux, where the watchers only capture events while Zotero
  is running, each library gets a full scan on its first file sync of
  the session, on every manual sync, and daily during background syncs
  (instead of the previous 3-hour interval, which dated from when scans
  were the primary detection mechanism).
- On macOS, libraries scanned since the last FSEvents journal
  discontinuity are tracked in a pref, since the journal -- and
  therefore the validity of previous scans -- survives restarts.

Also:

- Check FSEvents event flags and fall back to full scans when events
  were dropped or coalesced (MustScanSubDirs/UserDropped/KernelDropped/
  EventIdsWrapped), and skip HistoryDone sentinel events
- Disable the watcher for the session and fall back to legacy scanning
  on backend errors, including when the inotify watch limit is reached,
  instead of continuing with silently incomplete coverage
- Prune scan records for deleted libraries, since SQLite can reuse a
  deleted library's libraryID

https://forums.zotero.org/discussion/132120/
2026-06-10 22:26:13 -04:00
Dan Stillman
f21e1b2d32 Add platform-native file change watcher for storage sync
Use platform-native file change APIs to track which stored-file
attachments have been modified, avoiding expensive full scans of all
attachment files during sync.

Backends:
- macOS: FSEvents (persistent event journal, survives restarts)
- Windows: ReadDirectoryChangesW (live recursive directory watch
  via overlapped I/O polling)
- Linux: inotify (live per-directory watches)

On unsupported platforms or on error, falls back to existing scan logic.
2026-02-24 14:15:15 -05:00