Items are added to the collection created for an import as they're
saved, before the imported collection hierarchy exists, so an item only
in a subcollection ended up in the top collection as well as its own.
Remove those once the hierarchy has been created.
When everything imported belongs to a single top-level collection, that
collection is used for the import rather than being nested inside a
collection named after the file.
https://forums.zotero.org/discussion/133174/
Exporting a collection included only its subcollections, so an item
directly in the collection came through with no collection at all, and
an item in both the collection and a subcollection came through in only
the subcollection.
Exporting a selection of multiple collections exported a flat list of
items with no collections at all.
We now include the selected collection(s), with one exception: if a
saved search is also selected, we export a flat item list, since a
search can't be exported as a collection.
Mozilla's OSKeyStore.encrypt() encodes the string as UTF-8 before
encrypting, but its decrypt() returns the decrypted bytes as a binary
string without decoding them, so a WebDAV password containing non-ASCII
characters came back mojibake and authentication failed.
https://forums.zotero.org/discussion/133465/problem-login-into-webdav-server-with-10-0-1
Operators can be typed as the Advanced Search shows them, which every
locale already translates, and new keyword messages cover the join
words, "no"/"has", the units of a relative date, and the range forms.
Each range form is given as an example with its two ends filled in, so
that a locale can say it its own way and each form keeps its own words
(e.g., no "between 1970 to 2000").
"year is between 1970 and 2000", "year:1970-2000", "1970..2000", and
"1970 to 2000" all match values within the range, inclusive of both
ends. Ends can be a year, a month ("added between 2024-02 and
2024-06"), a day ("date:2020-03-01..2020-03-15"), or a count ("number
of tags between 2 and 5").
* Fix js-ctypes-based symlinking on Linux by using `libc.so.6` instead of `libc.so` in `OS.File.unixSymlink()` and `Zotero.File.createSymlink()`
* Use that instead of `/bin/ln`, which doesn't exist on NixOS
* Replace `/bin/ln` with `Zotero.File.createSymlink()` in symlinked-database test
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
FSEvents is backed by a per-volume journal that only local volumes
have. On a network mount the stream is created and started
successfully but never delivers events, so the watcher would report
that nothing had changed for as long as it was used. Check the volume
with statfs() and fall back to scanning.
Since f21e1b2d32, a full local file scan no longer runs periodically and
on every manual sync, so locally missed attachments stayed marked for
upload and were skipped as unavailable instead of being downloaded.
"Reset File Sync History" marked every attachment for upload, including
files that had never been downloaded, so the forced download check added
in 404fc41b88 found nothing to download.
Missing files are now marked for download when the upload queue is
filled, and in at-sync-time mode they're downloaded in the same sync.
The reset marks them for download directly, and downloads are no longer
skipped just because there were no remote storage changes.
https://forums.zotero.org/discussion/133414/
Any error while saving a note prompted the user to restart Zotero, even
a transaction timeout caused by a long-running operation elsewhere.
Nothing has been written when the wait times out, so retry, unless newer
note content has been handed to the editor in the meantime.
https://forums.zotero.org/discussion/133298/
Editing a note flagged it stale, and the background drain indexed it and
then ran an FTS5 'optimize' -- a single statement that rewrites the
content index and can hold the shared database for over a minute --
because the queue was empty again. Note saves waiting on the connection
hit the transaction timeout and told the user to restart Zotero.
Merge the index in bounded steps instead, so no statement runs long
enough to keep other queries waiting, and only after enough items have
been indexed to be worth it.
https://forums.zotero.org/discussion/133298/
Any Field expands to a generic 'field' condition, which the cross-level
code treated as matching only on top-level items. At an attachment
result level it therefore matched attachments whose parent item had the
value, instead of attachments with the value themselves. Since the
condition stands in for every field, it's now treated as matching at any
level those fields live at.
Zotero.SearchQuery turns a query like `by:smith after:2020 tag:"to read"
crispr` or `creator is smith and (tag is foo or bar)` into a
Zotero.Search, matching whatever text is left over using the current
search mode. Anything that doesn't look like a clause is free text, so a
DOI or a title with a colon in it is matched literally.
The search box syntax-highlights the parts of recognized conditions and
offers autocomplete for condition names, for the values of conditions
that have a fixed set of them (like item type), and for tags and
creators from the selected libraries.
Values without a parsable year are stored with a 0000-00-00 sort key,
and the guard meant to exclude them from comparisons checked the full
multipart value, which is always greater than '0000-00-00'.
Update the global schema to 45, resolve a field's date type through its
base-field mapping in ItemFields.isDate() (to cover priorityDate), and
convert stored values of date-type fields to multipart dates on schema
upgrade.
Only Date and Accessed were routed to the datefield condition, so other
date fields (e.g., Filing Date) offered only text operators and
couldn't be compared as dates. Text operators are still accepted on
date fields so existing saved searches keep loading and running.
CIFS mounts can't create real symlinks without special mount options,
so skip the symlinked-database and broken-symlink tests when a created
symlink doesn't exist or isn't visible as one.
On macOS, SQLite chooses locking methods based on the filesystem
containing the database, and network filesystems (e.g., SMB, NFS),
read-only volumes, and filesystems without byte-range locking get
methods without shared-memory support, which WAL requires. Opening a
database with an adjacent WAL file on those crashes -- Mozilla's VFS
wrapper hides the missing shared-memory methods from SQLite's WAL
support check -- so the first Zotero 10 run converted the database to
WAL and every launch after that segfaulted during connection
initialization.
On macOS, mirror SQLite's method selection and use a rollback journal
when shared memory isn't available, converting an existing WAL database
before opening it: an empty WAL by reverting the header format versions
in place, and a non-empty WAL by replaying it into a temporary copy on
local disk that replaces the database file only after passing an
integrity check. A WAL file next to an already-converted database
(e.g., from an interrupted conversion) goes through the same
conversion. Also use openNotExclusive during integrity checks and
corruption recovery, which otherwise fail on SMB shares with an I/O
error from the exclusive open lock.
This corrects 22055d92b7, which passed openNotExclusive on all
platforms for an open failure described as affecting macOS and Linux,
and expected locking_mode=EXCLUSIVE to keep the WAL index in heap
memory with no -shm file. Neither claim held up: mozStorage opens the
WAL while initializing the connection, before any pragma can run, so
the index uses shared memory unless the exclusive VFS is in use, and
the exclusive open works on Linux CIFS mounts -- where non-exclusive
access instead made SQLite's lock-upgrade sequence trip over the SMB
byte-range lock mapping, leaving startup hung and the database never
created. So pass openNotExclusive only on macOS. On Linux this restores
unix-excl, which performs all locking under a single held lock and
keeps the WAL index in heap memory; Windows has no distinct exclusive
VFS and is unaffected.
https://forums.zotero.org/discussion/133258/
The traversal waits on focus/blur events, which Gecko fires only while
the window is active, so a window that can't be activated -- another
app focused during a local run, or intermittent deactivation in CI --
timed out after 10 seconds instead of reporting that it couldn't run.
Tests asserted exact search results for words like "one" and "two", so
a leftover item from an earlier test whose random title contained one
of the words as a substring could match and fail the assertion. Use
distinctive random strings instead.
EDTF dates -- ranges ("2021/2026"), uncertain/approximate dates
("2004-06~"), and BCE dates ("-0429") -- and common equivalent notations
("1995-1996", "2021-22", "~1995", "ca. 1995", "429 BCE") are now passed
to citeproc-js as CSL date ranges, circa flags, and negative years.
Previously, such dates were mangled or dropped entirely unless entered as
CSL variables in Extra.
CSL date variables in Extra get the same parsing.
Other date handling in the client doesn't understand EDTF yet: the y/m/d
indicator in the date field doesn't reflect EDTF parsing, date searches
only match a range by its start date, and BCE dates still can't be
sorted.
Addresses #637
The test relied on the automatic selection of a newly created item,
which can lose the race with a view switch (e.g., from another library
selected by a previous test), leaving the info pane empty.
Notifier events that added or removed rows (e.g., items downloaded
during a sync) didn't update the no-selection message in the item pane,
so the count went stale until the selection changed. The row provider
now emits a rowCountChange event, and the pane re-renders the count,
debounced, when nothing is selected.
Zotero.Users.setName() updated the in-memory cache even if the
transaction was later rolled back, so a retry (e.g., of a failed sync
download batch) would skip rewriting the users row.
An error thrown from a commit callback rejected executeTransaction()
even though the transaction had been committed, so callers would treat
saved data as rolled back, and rollback callbacks (e.g., the notifier
reset) ran against committed data. Commit-callback errors are now
logged instead, and any error after a successful commit skips rollback
callbacks and is marked with 'committed' on the error object.
If a transaction was rolled back, callbacks queued via
addCurrentCallback('commit') stayed queued and ran after the next
successful commit, potentially applying in-memory cache updates for
rolled-back data.
Runs a function within a transaction that's rolled back at the end,
with rollback callbacks guaranteed to have run by the time it returns.
Replaces try/catch blocks in existing tests that threw and caught a
marker string from executeTransaction() to force a rollback.
the all-creators mode (fieldmode 2) concatenated
firstname || ' ' || lastname, so a two-field creator without a first
name produced ' last', which advanced search fields inserted with the
space.
Binding is meaningful for a condition that matches at every level -- a
tag bound to an attachment means the tag is on the attachment -- but a
group carrying one lost the binding as soon as the search was
serialized, so "items with an attachment tagged foo" couldn't be built.
An exact-match condition compared with SQLite's case-sensitive '=' and
skipped the normalized shadow columns, so 'publication is "review of
finance"' missed "Review of Finance" while every other kind of search
matched it.
Bug 2037682 removed nsIPK11TokenDB in favor of getting the internal key
token directly from @mozilla.org/security/internalkeytoken;1, and bug
2043434 replaced initPassword()/needsUserInit with changePassword().
Bug 2008041's change to boolean attributes also covers hidden and
collapsed, whose UA selectors became [hidden] and [collapsed], so
setAttribute('hidden', false) now hides the element. Switch the setters
that can be passed a falsy value to toggleAttribute(), read them with
hasAttribute(), and match the [collapsed=true] selectors in our own
stylesheets to the new presence-only form.
Bug 2008041 made them html-style boolean attributes, so presence alone
means true and setAttribute('disabled', false) now disables the element.
Switch the sites that can be passed a falsy value to toggleAttribute(),
read them with hasAttribute(), and drop a disabled="false" from markup.
Sites that only ever pass true are left alone.
Bug 2009806 moved XUL checkbox click handling into C++ and dropped the
CheckboxStateChange event, so listen for command instead. The event now
only fires on user interaction, so the tests have to click the checkboxes
rather than assign to .checked.
findLogins() now throws, and removeLogin()/modifyLogin()/removeAllLogins()
are Async-suffixed and return promises. Without this, Zotero couldn't read
the stored API key and showed the corrupted-logins-database error on
startup.
There's no synchronous read left, so Zotero.Sync.Runner.enabled -- which is
consulted from places that can't await -- now reads a value that
hasCredentials() caches.
Closes#5974.
Additional fixes for broken item tree behaviour when multiple items are
selected, and changing focus with ctrl/cmd-arrow keys.
Aligned Collection Tree/Virtualized Tree collapse/expand behaviour when
multiple containers are selected, one of them is focused, and arrow key
left-right is pressed, to the behaviour in Item Tree - now all of them
are collapsed/expanded.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
The PMC lookup in #5906 was added before the OA lookup, but the OA
lookup often returns a PMC URL of its own (including, at least now, for
the DOI given in the PR), with a direct file URL rather than a page to
scrape, so try it first and fall back to the PMCID-derived URL only when
it turns up nothing.
Since #5906, the 'oa' resolver list can start with a plain URL object
for items with a PMCID, but _getOpenAccessPDFURLs() and
saveOpenAccessAttachment() called resolvers[0](), which threw for those
items -- skipping the lookup silently in the first case and failing the
save in the second.
Show File triggered a file 'open' notification, so revealing a file
moved its item to the top of Recently Read. Use a separate 'reveal'
event, which file syncing still observes to queue a modification check.
https://forums.zotero.org/discussion/133006/
- Fix multiple potential scenarios causing a template engine crash
- Add support for specifying string literals in the template engine
- Validate `if/else/elseif` order and matching clause closures
- Validate to ensure every `{{` is properly closed with a matching `}}`
- When a template is invalid, display a warning, do not offer batch-renaming tools, do not update synced setting
- When a template is invalid, prompt the user to fix or reset the template when
Closes#5965
