The cache-existence check didn't await getCacheObject(), so the
promise was always truthy and no cache object was ever saved for
objects the server reported as unchanged. Without a cached version,
later remote changes to those objects produced avoidable conflicts.
When a user's account is deleted and then undeleted on the server, the
dataserver creates a fresh shardLibraries row at a low version, while the
client still has a much higher local library version. Subsequent syncs
then throw "_libraryVersion cannot decrease" and the user is stuck.
Tag the cannot-decrease errors from the library version setters with a
named error, catch it in Engine.start(), reset libraryVersion and
storageVersion to -1, and restart. The retry hits the existing
libraryVersion == -1 branch and runs _fullSync(), which re-uploads the
local library against the recreated server library.
If the server returns a write report with 403 for admin-only settings,
mark successful settings as synced and reset rejected settings to
their remote values by re-downloading from the server.
Needed for #5862
Deleted settings were written to syncDeleteLog but never uploaded. Add
dedicated setting deletion upload in _startUpload() using individual
DELETE requests per setting, since the API doesn't support batch setting
deletion.
resetData() clears user data and sync state using application APIs
instead of doing a full Zotero shutdown/reinit cycle, making it much
faster for tests that just need clean state between runs.
Also make Tags.init(), Creators.init(), Users.init(), and
SyncedSettings.loadAll() properly clear their caches before reloading,
so they are safe to call more than once.
Fix ghost relations to erased items reappearing after
sync. The dataserver ensures that related relations are
bidirectional. If a related item is erased, syncing
would first upload the update to the remaining related item
(where the relation is removed), followed by the deletions.
While handling update upload, the dataserver would return
the old, should-be-removed, "dc:relation" entry for that item
because it does not know that the other item is erased.
To handle this, swap the order of operations during sync:
upload deletion first and then upload updates after.
Then, when the dataserver handles the update of the non-erased item,
there is no related item to enforce bidirectional relations to.
Fixes: #5481
A test using `assert.eventually` was failing after the Bluebird removal
but worked with just `await`, and since there hasn't really been much
point to Chai as Promised since the introduction of `async`/`await` ages
ago, just remove the library instead of figuring out why.
https://forums.zotero.org/discussion/104431/syncing-problem
Replace Online Library can upload annotations created by others in a
group library, so if the upload resulted in a local write, "Cannot edit
item in library" was thrown, since annotations by others aren't
writable. This should've only been a problem if the uploaded data was
actually modified by the server, but we were also checking whether
objects were editable before checking if they had actually changed, so
it would happen for any upload of another person's annotation.
This fixes the order of checks when saving objects and makes an
edit-check exception for saving uploaded data for group annotations.
Previously only individual objects from successful requests that
couldn't be processed for some reason would be added to the queue.
`Sync.APIClient.downloadObjects()` now returns clearer and more
consistent results. It now returns an array of promises for objects with
a `keys` array of requested keys and either a `json` array of returned
API JSON objects or an `error` Error, depending on whether the request
succeeded or failed. This makes it easier to detect remotely missing
objects and request failures.
This adds a very basic reimplementation of the annotation widget for use
in the CR window. It's not pixel-perfect, but it's close enough that
most people probably won't notice. We'll want to replace this with a
real version that's shared between the PDF reader and the rest of the
codebase.
Image annotations currently show an "[image not shown]" placeholder.
Showing images is tricky: we don't have the remote cache image, so if
the remote position has changed, we could only show an image by
rendering it from the file, and only if the file itself hasn't changed.
Just for a better user experience, we could use the local image as long
as the position and file are both the same, but that would take some
rejiggering of the CR window.
Tags aren't shown because they're not shown for CR at all, though that
could be changed in the future.
Added Zotero.DataObjects.sortByParent() to sort child items immediately
after their parent items. Zotero.DataObjects.sortByLevel(), which is
used for collections, sorts each level together, but that's less
appropriate for items where, e.g., an embedded-image attachment should
immediately follow the note that depends on it.
E.g., if a local item has been modified in a way that conflicts with a
remote item that also has a new, unknown field, don't show the CR
window -- just add the item to the sync queue and show the
some-data-could-not-be-downloaded error.
Previously, if an object was uploaded but the API returned 'unchanged',
the uploaded data would be written to the sync cache, which, given that
most requests are patch requests, could result in an empty or mostly
empty object being saved to the sync cache. That would cause the next
sync to treat most/all local fields as changed and either upload them
unnecessarily or trigger a conflict instead of merging changes
automatically.
Say "Use the [local|remote] version for all remaining conflicts" for
everything instead of saying "Use [local|remote] fields for all
remaining conflicts" for some conflicts.
This also fixes a test failure after 54343c49fb.
This changes the way item types, item fields, creator types, and CSL
mappings are defined and handled, in preparation for updated types and
fields.
Instead of being predefined in SQL files or code, type/field info is
read from a bundled JSON file shared with other parts of the Zotero
ecosystem [1], referred to as the "global schema". Updates to the
bundled schema file are automatically applied to the database at first
run, allowing changes to be made consistently across apps.
When syncing, invalid JSON properties are now rejected instead of being
ignored and processed later, which will allow for schema changes to be
made without causing problems in existing clients. We considered many
alternative approaches, but this approach is by far the simplest,
safest, and most transparent to the user.
For now, there are no actual changes to types and fields, since we'll
first need to do a sync cut-off for earlier versions that don't reject
invalid properties.
For third-party code, the main change is that type and field IDs should
no longer be hard-coded, since they may not be consistent in new
installs. For example, code should use `Zotero.ItemTypes.getID('note')`
instead of hard-coding `1`.
[1] https://github.com/zotero/zotero-schema
Check file-editing access for the group from the API before offering to
reset, update the filesEditable setting properly, and restart the sync
automatically after resetting.
Restores the "Restore to Zotero Server" functionality, now using the
API:
1. Get all remote keys and send `DELETE` for any that don't exist
locally.
2. Upload all local objects in full (non-patch) mode using only library
version so that the remotes are overwritten.
3. Reset file sync history, causing all files to be uploaded (or, more
likely, reassociated with existing remote files).
Since these are treated as regular updates on the server, they'll sync
down to other clients normally. Unsynced changes by other clients might
still trigger conflicts.
This and Reset File Sync History can also now be run on group libraries,
with a library selector in the Reset pane (which I forgot to do with
React).
The full sync option is now removed from the Reset pane, since there
wasn't ever really a reason to run it manually.
We should be able to reimplement Restore from Online Library (#1386)
using the inverse of this approach.
Closes#914
While objects in the sync queue that fail to save should remain in the
queue, objects that just don't exist remotely need to be removed, or
else they'll be retried forever.
