The OS.File.makeDir() shim swallowed IOUtils errors, so a failure to
create the data directory showed as a generic "database cannot be
opened" error. For the default location, just explain and quit instead
of offering to choose another directory.
https://forums.zotero.org/discussion/134124/zotero-not-working-on-a-mac
MDPI serves a JS proof-of-work interstitial in place of the article
page, so Find Full Text found nothing. Run the challenge in a hidden
browser when a page's meta refresh points to a registered challenge
host, then retry the page.
Also match meta refresh URL= case-insensitively, since MDPI's is
uppercase, and bound meta refreshes by the redirect limit.
https://forums.zotero.org/discussion/132837/https://forums.zotero.org/discussion/134079/
An hour of retries makes sense for syncing, which runs automatically and
can just spin during server maintenance rather than showing errors that
send people to the forums, but it was also inherited by foreground
requests, where it stalled operations the user was waiting on. Sync and
file syncing now ask for the long window explicitly.
A file URL returning a server error was retried for up to an hour inside
the download, and a 429 or Retry-After was waited out there too. Since
the queue processes one item at a time, that blocked every other
selected item. Throttling now goes to Find Full Text's own per-domain
handling, as it did before 10.0, which also needed to read Retry-After
from a fetch Response and parse HTTP-date values.
https://forums.zotero.org/discussion/133703/
Retry-After was honored unconditionally with no attempt limit, so a server
returning 429 or 503 with the header on every request retried forever.
Retry-After waits and backoff intervals now share the errorDelayMax
budget, with each Retry-After counted as at least a second so that a
value of 0 can't loop forever.
This runs during Find Full Text and connector saves, where the default
30-second timeout and hour of 5xx retries are far longer than a user
wants to wait.
A custom resolver returning a 429/5xx inherited Zotero.HTTP's default
retry policy, which could cause the whole queue to stall for up to an
hour -- or indefinitely for a Retry-After -- on a dead resolver, despite
the 5-second timeout on the request.
https://forums.zotero.org/discussion/133642/
Since the switch to fetch() in 0fe31b0f04 (Zotero 10.0.0), download
requests didn't use cookies, and the Referer header was silently dropped
as a forbidden header. Sites that check either -- e.g., IEEE Xplore,
which returns a 502 -- failed during Find Full Text.
https://forums.zotero.org/discussion/133703/
As of Firefox 153.3.0esr, Services.scriptloader refuses jar:file: and file:
URIs unless allowUnsafeURL is passed (Mozilla bug 1974213), so no plugin
loads: bootstrap.js fails with "Trying to load untrusted URI", then
"Plugin ... is missing bootstrap method 'startup'".
Pass allowUnsafeURL when loading a plugin's bootstrap.js and prefs.js,
and preference pane scripts, and temporarily enable
security.allow_unsafe_subscript_loads, which covers loads from plugin
code outside the bootstrap scope.
Also add a test that installs a fixture plugin that loads a script from its XPI
and sets a default pref.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Switching to a view that has the same tags but with different types
(manual vs. automatic) kept the previous list, so with "Show Automatic"
off a tag could show or hide incorrectly.
If a view contained manual and automatic tags with the same name, only
one was kept, and it would disappear with "Show Automatic" off.
https://forums.zotero.org/discussion/133869/
As of Firefox 140.17/153.4 (Bug 2068406), setting nsIFilePicker's
displayDirectory to a directory that doesn't exist or isn't readable
throws instead of being ignored. This broke callers that pass a saved
path that may be stale (e.g., choosing a PDF/EPUB handler after the old
app's folder was removed, or a missing Scaffold translators directory).
The action column's width was given as "32px", which VirtualizedTable
now turns into an invalid CSS width, collapsing the column and hiding
the buttons needed to resolve unmapped and ambiguous citations. Also
restore the accept-match icon on ambiguous-citation candidates.
https://forums.zotero.org/discussion/133740/
A condition that matched a child item in the trash rolled up to its
parent when the search had a top-level (or other ancestor) result level,
or when "Include parent and child items of matching items" was checked.
https://forums.zotero.org/discussion/133836/
A standalone PDF recognized by ISBN wasn't moved under the new parent
item, because the move was attempted before the item had been saved
and had an ID.
https://forums.zotero.org/discussion/133957/
A userdata upgrade that took more than 5 minutes (e.g., dropping a large
legacy word index) was rolled back by Sqlite.sys.mjs, while its remaining
statements autocommitted and marked the database as upgraded, leaving
steps 126 and 127 missing.
Disable the transaction timeout and replace steps 122-129 (added in
Zotero 7, 9, and 10) with step 130, which checks for each change before
making it. To speed up the upgrade, drop the legacy word index without
overwriting the freed pages, since the full text already exists on disk
and we're just rewriting it in fulltext.sqlite.
https://forums.zotero.org/discussion/133859/zotero-connector-in-chrome-not-workinghttps://forums.zotero.org/discussion/133965/citation-saving-does-not-work-with-zotero-connector-firefox
Firefox has AppKit draw the menulist's capsule and chevron, but it sizes
the box and positions the label itself, using a fixed dropdown border
and the label's CSS margins. On macOS 27 (and probably 26), that left the
label closer to the edges and the chevron than in a native NSPopUpButton.
It also made the box 26.5px tall. Firefox assumes regular pop-up buttons
are 22px tall and only draws them unscaled in boxes up to 2px taller, so
it drew the control at 22px and scaled the image up, enlarging the
capsule and chevron.
Add padding to match native label insets, and trim the label's vertical
margins to keep the box at 24px, so Firefox draws the control unscaled.
The result measures the same as a native NSPopUpButton.
Build the launcher with the macOS 26.5 SDK. AppKit chooses control metrics
based on the main executable's SDK, so with the old 15.5 launcher, native
buttons and menulists used pre-Tahoe metrics with built-in margins while
Mozilla's XUL (built with 26.5) stopped compensating for them on macOS 26+
(bug 1992898), leaving labels cramped and controls indented.
Also add custom libmozglue.dylib (#6056) and set source info in mozconfig,
which official builds require and which Mozilla only detects automatically
from Mercurial checkouts.
Mozilla's helper apps (GPU, content, etc.) use the hardened runtime and
Mozilla's Team ID, so in unsigned builds they couldn't load our custom
libmozglue.dylib and failed to launch. Re-sign them ad hoc.
Firefox stopped inflating native controls on Tahoe (bug 1992898), but
still uses the old widget border sizes, so labels nearly touch the edges
of the new capsule-shaped buttons and run into menulist arrows.
curl is already required, so drop wget as a build requirement.
build_autoupdate.sh now checks the HTTP status code for its ETag cache
instead of relying on wget not creating the file on a 304.
If NOTARIZATION_PROFILE is set, notarize_mac_app and notarization_info
authenticate with that stored notarytool profile (e.g., an App Store
Connect API key) instead of an Apple ID and app-specific password,
unlocking the keychain first if needed.
153.3.0esr updated the Chromium sandbox, changing the TargetConfig
interface that xul.dll calls into the launcher's sandbox broker
through. The stubs were still built from 153.0esr, so the 153.3.0esr
xul.dll called the wrong methods and crashed on startup with
"config->SetProcessMitigations(initialMitigations) failed".
The release script never set SAFARI_APP_EXTENSION, so 10.0 through
10.0.3 shipped with only the web extension, which doesn't run on Big Sur
or Monterey. Only beta builds have been including both.
https://forums.zotero.org/discussion/133871/
This showed up as devtools failing to start in a -d build on some machines, but any Subprocess.call() could fail because of it.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
The login manager can fail to store a value even when the OS keystore
works -- e.g., if key4.db is read-only, which Firefox 153 triggers
because NSS now creates a new AES key on the first write. Show the
existing corrupted-logins instructions instead of the keystore alert or
the unencrypted-storage offer, neither of which can help.
Also update the corrupted-logins dialog to add an "Open Profile
Directory" button
Allowing only 'copy' for file attachment drags kept File Explorer from
moving files out of 'storage' but locked the cursor at '+' even for
moves within Zotero. Instead, provide the file via a flavor data
provider that copies it to the temp directory when a target asks for it,
so the drag can allow 'copyMove' again and Explorer's move only touches
the copy. A copy's directory is removed once Explorer has moved the file
out of it, at the next drag, or with the temp directory at shutdown.
Since 56eb77b704, drags of file attachments allow only 'copy' so that
File Explorer doesn't move the file out of storage, but the trees set
dropEffect to 'move' in onDragOver() for drops within Zotero, and OLE
refuses a drop whose dropEffect isn't among the drag's allowed effects.
Have setDropEffect() fall back to an allowed effect and have onDrop()
act on the effect the tree chose, kept in
Zotero.DragDrop.currentDropEffect, rather than on the drop event's
dropEffect.
https://forums.zotero.org/discussion/133765/
The drop indicator span was inserted before the cells, so the first
cell stopped matching :first-child and picked up inline-start padding.
Insert it after the cells instead.
Regression from 5ca1fbb167, which replaced the .first-column class with
:first-child.
fd812070b6 made _parseURI() decode the URL credentials so that
download() could build its own Basic Auth header from the decoded
values, but request() passed the decoded credentials to xmlhttp.open(),
which percent-decodes them again, so a password like "example%41pass"
was sent as "exampleApass". Re-encode the credentials before passing
them to open().
Fixes#6048
Gecko 140.15 made nsIExternalProtocolService.loadURI()'s triggering
principal mandatory, so Zotero.launchURL() threw NS_ERROR_ILLEGAL_VALUE
for any scheme handled by an external app.
https://forums.zotero.org/discussion/133709/
relinkAttachment() calls getClosestDirectory() before showing the file
picker, and a too-long filename caused the OS.File.stat() in
getClosestDirectory() to throw, which prevented the file picker from
appearing after clicking Locate.
https://forums.zotero.org/discussion/133685/
Firefox ESR 140.15 rejects loads from a content process for URLs that
process couldn't load on its own, including blob: URLs created by chrome
code. Full-text indexing loads HTML attachments through such a URL, so
indexing crashed Zotero with "Illegal load attempt of blob: URL from
web". Loads started by the parent are exempt, so start the load there
and use the child actor only to disable content retargeting.
https://forums.zotero.org/discussion/133661/
SpiderMonkey stacks contain only frames, so the startup error messages
that showed just the stack didn't say what had failed. 9b3d7a32e3 added
the message to one of the three, where a ternary-precedence bug then
dropped the surrounding text instead. Format all three the same way.
We only ever checked for corruption errors from transactions in
queryAsync(), so a corruption error raised by the COMMIT that mozStorage
runs itself bypassed the check. A 10.0 schema upgrade -- which heavily
exercises the database -- that failed due to corruption showed "Database
upgrade error" and a single Sqlite.sys.mjs frame instead of the prompt
offering to restore from a backup.
Two users reported this, but it's not clear what triggered it --
corruption usually occurs during a statement, which we did catch. There
may have been some statement transaction whose corruption error was
caught and ignored rather than being left to abort the transaction,
causing SQLite to then block the commit. That's what the test does, and
it fails without the fix.
https://forums.zotero.org/discussion/133611/
If the keystore's secret is replaced -- say, after a keyring reset --
the stored value won't be able to be decrypted again, so tell them to
try logging in again instead of trying to fix their keyring.
https://forums.zotero.org/discussion/133603/
unload() closes and re-adds the tab, and the close recorded an undo-close
history entry, so Undo Close Tab could open a duplicate of a tab that was
still open.
_addEntry() wrote the row with REPLACE INTO, listing only itemID and
data, so flag reverted to its default of 0. It's called for every match
on every update, not just new ones, so a retraction the user had hidden
came back as soon as the list changed and Zotero restarted.
A window is marked as closed before its unload handlers run, so the tab
events fired from ZoteroPane.destroy() warned about every observer that
the closing window hadn't torn down yet. The leak check now also requires
the window to be detached from its docShell, and it covers functions. It
caught an observer that outlived its window -- the sync-reminder API key
observer, which is now unregistered on teardown.
Listeners added to the item and collection trees' event bindings
survived unregister() and held ZoteroPane and the views for as long
as the tree objects did. makeClassEventDispatcher() now provides
clearEventListeners(), which the trees call on unregister, so callers
don't need to remove their own listeners. The collection tree focus
listener, a plain DOM listener, is removed in destroy().
Hiding the tag selector destroyed the TagSelectorContainer but left
its media-query listener registered, keeping the destroyed instance
alive for the lifetime of the window and firing setState() on the
unmounted component when the display density changed. The fontSize
pref observer id was also overwritten by the uiDensity registration,
so it was never unregistered.
Selecting a collection doesn't wait for the tag selector to re-render,
so waitForTagSelector() could be resolved by the initial render instead
of the awaited change, and the assertions could run against a stale tag
list.
The cached retractions list is saved with the current data version from
the code, which we can bump to make clients discard their cached data
and refetch it. We've never actually done that, but if we did, it
would've hit a bug: init() would've started the refetch but still sent
the cached ETag, so if the list itself hadn't changed, the server
would've returned 304 and nothing would have been updated. The stale
version then would have survived, so the refetch would have been retried
on every startup.
Retraction Watch has renamed, merged, and added reason terms since these
were added, leaving only 65 of the 102 entries matching a term still in
use. 90% of retracted items had at least one reason that didn't show a
description, and 21% showed none at all. Rebuilt from the current
Appendix B list, which covers every reason in the data.
Windows returns this for ERROR_FILE_CORRUPT/ERROR_DISK_CORRUPT. It's not
in xpc.msg, so the exception has an empty name and isn't in
Components.results, and it was falling through to the generic file sync
error instead of the file access error with the path. Show a message
about the reported corruption in place of the usual permissions advice.
https://forums.zotero.org/discussion/133592/
The shims from 5697ee0af7 keep old plugins working for users, but
beta/dev/source builds should still throw to get the attention of
developers and (hopefully breakage-tolerant) beta users.
Status messages were left aligned for the multi-paragraph intro text,
but in a narrow window, that made single lines that wrapped sit
off-center, so center those instead.
Plugins written for earlier versions call methods like
CollectionTree#getSelectedSearch(), which now throw. We did this
intentionally to make old code more obviously broken, even with single
selection, so developers would fix their code, but we're not yet
blocking plugins that illegitimately declared compatibility with a
future version, so some haven't been updated and are breaking Zotero.
With a single row selected, the pre-10 functions can technically
continue to work, so for now, restore them and just warn and name the
plugin instead of throwing. They still need to throw if multiple rows
are selected, since plugins that haven't been updated can't safely act
on a selection that might span collections or libraries.
https://forums.zotero.org/discussion/133565/
A plugin that monkey-patches a method in the item list's load path --
say, Zotero.CollectionTreeRow.prototype.getItems() -- can throw and
leave the pane showing only "Error loading items list", with nothing in
the error report to identify it. Update the message to name the culprit.
https://forums.zotero.org/discussion/133565/
The row values were computed with `row.isItem && …`, so collection and
search rows in the trash got the boolean `false`, which the table then
rendered as the text "false".
https://forums.zotero.org/discussion/133560/
Zotero.Session.save() runs from a quit-application-granted observer
registered before the database checks, but session.json is only read
after them, so a startup error -- an incompatible database from a newer
version, say -- meant that quitting overwrote the file with an empty
state and all open tabs were lost.
https://forums.zotero.org/discussion/133542/
A stored-file path of 'storage:/' -- left behind by the 128 schema step,
which skipped paths with no basename -- triggered an
NS_ERROR_FILE_UNRECOGNIZED_PATH that aborted the whole
checkForUpdatedFiles() loop, so no files synced in the library. Skip an
attachment that throws instead of failing the whole library.
Also apply the setter's directory-path rule in getFilePath[Async]() to
avoid errors elsewhere.
https://forums.zotero.org/discussion/133523/synchronize-issue
Collections are now tracked as they're selected, added to, or dropped
on, and the five most recent usable targets are listed above the
full collection hierarchy by full path.
transformToDocument() needs a load group, which it takes from the source
document or from the window that created the XSLTProcessor. Neither of
those has existed since 0f2690eb75 in Zotero 8 stopped taking
XSLTProcessor from the hidden window, so the CSL 0.8 → 1.0 upgrade threw
NS_ERROR_FAILURE. In Word, this showed as "Zotero encountered an error
while updating your document."
https://forums.zotero.org/discussion/133543/
citeproc-rs is no longer maintained, and people who had enabled the
hidden pref were hitting errors.
This also drops the free() call on CSL engines, which only existed to
free the citeproc-rs wasm driver.
https://forums.zotero.org/discussion/133515/
If storing the API key failed, the pane kept showing the spinner and
"Waiting for login…" until the user clicked Cancel, even though the
login attempt was already over.
https://forums.zotero.org/discussion/133418/
Every failure shows "User canceled OS unlock entry" no matter what went
wrong. Test the store for the actual state, and give callers a message
describing what couldn't be accessed.
Both the keystore fallback prompt and the migration alert are reachable
from the credential read paths that a sync uses, so don't show if it's
an automatic sync.
Some Linux systems have no Secret Service running, and if users can't
change that (e.g., a managed system), storing an API key fails and login
never completes. Offer to store credentials unencrypted instead, and try
to encrypt them on a later read if the keystore becomes usable.
https://forums.zotero.org/discussion/133418/
Firefox 153 renders a menulist's popup as a native macOS menu, which
can't be opened to a submenu, so revealSelectedCondition() has done
nothing on macOS since the upgrade. Opt this popup out of native
rendering.
The conditions menu had its own find-as-you-type, which
Utilities.Internal.addMenuFindAsYouType() now provides for any menu.
Matching walks the menu rather than a cached list of every condition, so
typing can no longer select a condition the menu doesn't offer --
Collection and Saved Search are removed when the search spans multiple
libraries.
An attachment or annotation condition is shown with a short label inside
its submenu, so pass the full name to match on, which is what the
menulist shows once the condition is selected.
The Collection value menu was a flat list of every collection in the
library, with subcollections set apart by an indent (or hyphens before
606d8f19ba). Build it with Utilities.Internal.createMenuForTarget()
instead, using the new 'filter' feature to limit to collections. The new
FAYT helper preserves matching on subcollections.
If a subcollection is selected, open the path down to the subcollection
when opening the menu. (As of fx153, macOS renders popups as native
menus, which can't be opened to a submenu, so revert to non-native menus
there.) A collection in a submenu can't be a menulist's selected item,
so the condition holds the value and sets the menulist's label and icon
itself, with the collection's path as a tooltip.
A menulist's built-in find-as-you-type searches only the direct children
of its menupopup, so an item in a submenu can't be reached from the
keyboard. addMenuFindAsYouType() matches on every item in the menu
instead.
customElements.js opens a focused menulist when a space is pressed,
which would end a search partway through a name, so defer that while a
search is in progress.
The target menu isn't rebuilt between openings, so the checkmark stayed
on the target the menu was built with. In the New Collection dialog,
whose menu belongs to a menulist, that would leave two checkmarks if you
clicked a different collection and then reopened the menu.
On macOS, the current target in the New Collection dialog and the Add To
menus showed a checkmark in place of its folder icon -- type="checkbox"
makes nsMenuItemX::SetupIcon() skip the item. The `checked` attribute
alone marks it without suppressing the icon. Windows and Linux draw the
check in place of the icon either way.
Items are added to the collection created for an import as they're
saved, before the imported collection hierarchy exists, so an item only
in a subcollection ended up in the top collection as well as its own.
Remove those once the hierarchy has been created.
When everything imported belongs to a single top-level collection, that
collection is used for the import rather than being nested inside a
collection named after the file.
https://forums.zotero.org/discussion/133174/
Exporting a collection included only its subcollections, so an item
directly in the collection came through with no collection at all, and
an item in both the collection and a subcollection came through in only
the subcollection.
Exporting a selection of multiple collections exported a flat list of
items with no collections at all.
We now include the selected collection(s), with one exception: if a
saved search is also selected, we export a flat item list, since a
search can't be exported as a collection.
Forcing a Gatekeeper assessment seems to fix the extension when it's
broken. General theory: the system does an assessment while the app is
doing an in-place update, calculates a signature mismatch between the
parent app and the appex (or within one of the bundles?), and caches
that forever, so forcing a reassessment fixes it.
Mozilla's OSKeyStore.encrypt() encodes the string as UTF-8 before
encrypting, but its decrypt() returns the decrypted bytes as a binary
string without decoding them, so a WebDAV password containing non-ASCII
characters came back mojibake and authentication failed.
https://forums.zotero.org/discussion/133465/problem-login-into-webdav-server-with-10-0-1
addItems() passes its options to Zotero.Item::save(), but removeItems()
dropped everything but skipEditCheck, so callers couldn't batch the
resulting notifications.
Operators can be typed as the Advanced Search shows them, which every
locale already translates, and new keyword messages cover the join
words, "no"/"has", the units of a relative date, and the range forms.
Each range form is given as an example with its two ends filled in, so
that a locale can say it its own way and each form keeps its own words
(e.g., no "between 1970 to 2000").
"year is between 1970 and 2000", "year:1970-2000", "1970..2000", and
"1970 to 2000" all match values within the range, inclusive of both
ends. Ends can be a year, a month ("added between 2024-02 and
2024-06"), a day ("date:2020-03-01..2020-03-15"), or a count ("number
of tags between 2 and 5").
* Fix js-ctypes-based symlinking on Linux by using `libc.so.6` instead of `libc.so` in `OS.File.unixSymlink()` and `Zotero.File.createSymlink()`
* Use that instead of `/bin/ln`, which doesn't exist on NixOS
* Replace `/bin/ln` with `Zotero.File.createSymlink()` in symlinked-database test
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
FSEvents is backed by a per-volume journal that only local volumes
have. On a network mount the stream is created and started
successfully but never delivers events, so the watcher would report
that nothing had changed for as long as it was used. Check the volume
with statfs() and fall back to scanning.
Since f21e1b2d32, a full local file scan no longer runs periodically and
on every manual sync, so locally missed attachments stayed marked for
upload and were skipped as unavailable instead of being downloaded.
"Reset File Sync History" marked every attachment for upload, including
files that had never been downloaded, so the forced download check added
in 404fc41b88 found nothing to download.
Missing files are now marked for download when the upload queue is
filled, and in at-sync-time mode they're downloaded in the same sync.
The reset marks them for download directly, and downloads are no longer
skipped just because there were no remote storage changes.
https://forums.zotero.org/discussion/133414/
getFilePath() and getFile() return a path whether or not the file
exists, so dragging an attachment that hadn't been downloaded handed
the drop target a path that didn't exist. Missing files also went
unreported by the drag data provider.
Firefox 140.14 in Zotero 10.0 made the drag transferable's data
principal null for chrome-initiated drags, so the file-promise stream
that File Explorer used couldn't be created and the drop failed with
"Unspecified error". On Windows the promise was just a file:// URL for
the attachment itself, so hand over the file directly instead, and
force a copy so that File Explorer doesn't move it out of storage.
https://forums.zotero.org/discussion/133399/https://bugzilla.mozilla.org/show_bug.cgi?id=2054665
If an update had already finished downloading when the download page was
shown, the page waited for an update-staged notification even when
staging wasn't possible -- e.g., a default Windows installation in
Program Files, which isn't writable -- so it never advanced past
"Applying update…". It now checks whether staging is actually in
progress.
Not yet tested in an updatable build
rollbackAllTransactions() called transactionInProgress() and
rollbackTransaction(), neither of which has existed since nested
transaction support was removed in 14d435b8d8, so it would have thrown
had either of its two callers still been reachable. Both are in code
long since replaced: Zotero.Sync.Server, which uses the synchronous
Zotero.DB.columnQuery(), and an error handler in Zotero.Sync.Storage
that nothing calls.
Drop it along with _transactionNestingLevel, _transactionRollback, and
_shutdown, which nothing reads.
Any error while saving a note prompted the user to restart Zotero, even
a transaction timeout caused by a long-running operation elsewhere.
Nothing has been written when the wait times out, so retry, unless newer
note content has been handed to the editor in the meantime.
https://forums.zotero.org/discussion/133298/
Editing a note flagged it stale, and the background drain indexed it and
then ran an FTS5 'optimize' -- a single statement that rewrites the
content index and can hold the shared database for over a minute --
because the queue was empty again. Note saves waiting on the connection
hit the transaction timeout and told the user to restart Zotero.
Merge the index in bounded steps instead, so no statement runs long
enough to keep other queries waiting, and only after enough items have
been indexed to be worth it.
https://forums.zotero.org/discussion/133298/
_getConnectionAsync() checked for an existing connection and then awaited
several filesystem operations before assigning one, so callers arriving in
that window each opened their own. Only the last was kept, and the rest
stayed open and unreachable, holding a mozStorage thread apiece until
shutdown.
A permanently closed connection kept its idle observer, so it went on
being notified and attempting backups for the life of the process. This
affects plugin databases, which are closed permanently when the plugin
shuts down.
The observer was added on every open with no matching removal, so each
reopen left behind another registration that received its own idle
notification. On macOS, where the periodic backup closes and reopens the
connection, the registrations accumulated and multiplied the work done
on each idle.
Addresses #6027
currentDBTime and lastBackupTime became Date objects when the backup was
made asynchronous in 2014, so comparing them with == has tested object
identity and returned false ever since. The interval check that follows
measures from the backup file's mtime, which a copy inherits from the
database file, so an unchanged database was backed up on every idle.
Fixes#6027
Any Field expands to a generic 'field' condition, which the cross-level
code treated as matching only on top-level items. At an attachment
result level it therefore matched attachments whose parent item had the
value, instead of attachments with the value themselves. Since the
condition stands in for every field, it's now treated as matching at any
level those fields live at.
Zotero.SearchQuery turns a query like `by:smith after:2020 tag:"to read"
crispr` or `creator is smith and (tag is foo or bar)` into a
Zotero.Search, matching whatever text is left over using the current
search mode. Anything that doesn't look like a clause is free text, so a
DOI or a title with a colon in it is matched literally.
The search box syntax-highlights the parts of recognized conditions and
offers autocomplete for condition names, for the values of conditions
that have a fixed set of them (like item type), and for tags and
creators from the selected libraries.
Searching was deferred whenever the query contained a double quotation
mark, with an items-pane message to press Enter. Quoted phrases are no
longer expensive now that Everything mode uses full-text search, so
defer only while an unclosed quote means the phrase is still being
typed, and search as soon as it's closed. Enter still forces a search
mid-quote.
Values without a parsable year are stored with a 0000-00-00 sort key,
and the guard meant to exclude them from comparisons checked the full
multipart value, which is always greater than '0000-00-00'.
Update the global schema to 45, resolve a field's date type through its
base-field mapping in ItemFields.isDate() (to cover priorityDate), and
convert stored values of date-type fields to multipart dates on schema
upgrade.
Only Date and Accessed were routed to the datefield condition, so other
date fields (e.g., Filing Date) offered only text operators and
couldn't be compared as dates. Text operators are still accepted on
date fields so existing saved searches keep loading and running.
The GitHub runner image installs the x64 build of Git for Windows, so
its bash and everything run from it execute under x64 emulation on
Windows on ARM, where uname -m reports x86_64 and the staged build got
the wrong architecture. uname -s still reports the machine, as
MINGW64_NT-10.0-26200-ARM64, so use that instead. If the image ever
installs a native arm64 Git for Windows, uname -m will be correct and
this can go.
The ZIP format requires forward slashes, and nsIZipWriter stores
whatever it's given, so entries for files in subdirectories were
written with backslashes.
zotero.exe is the Mozilla launcher process, which spawns the browser
process and exits, so the test runner checked for the success file and
cleaned up the profile while the app was still starting, and passing
runs were reported as failures that ended at arbitrary points.
Neither is reliably available on Windows, so copy the source tree with
Python when rsync is missing and delete the CSL locale support files
with a directory walk.
Boots the Windows x64 and arm64 builds, which use custom-built Firefox
components rather than stock Mozilla binaries, and runs the DB and file
tests. A Gecko bump commit updates the custom-component hashes in
config.sh along with the version, so this tests each new set of custom
components, which previously required manual Windows testing.
Run the smb/nfs test shards and a new macOS NFS job only when the
changes touch the Gecko version or the network-filesystem database
handling, plus on manual and monthly scheduled runs, via a 'changes'
job feeding the test matrix. The macOS job uses the native NFS server
with a loopback mount and runs the real code path that crashed on
network data directories on macOS.
Since 30b5bf4bb9, the popup opens as a dependent window of the main
window, so it appeared behind the Style Preview window and, on Windows,
brought the main window to the front.
https://forums.zotero.org/discussion/133296/
CIFS mounts can't create real symlinks without special mount options,
so skip the symlinked-database and broken-symlink tests when a created
symlink doesn't exist or isn't visible as one.
The online backup API writes the backup file through SQLite's default
VFS, whose locking can hang on network filesystems -- its lock upgrades
conflict with the SMB byte-range lock mapping on CIFS mounts and retry
forever -- so fall back to an offline backup, which copies the file
with the connection closed. Extends getFileSystemInfo() to Linux using
statfs f_type magic numbers.
Adds 'smb' and 'nfs' entries to the test matrix that run the DB and
file test suites with the test data directory on a loopback CIFS or
NFS mount, covering the network-filesystem database handling from
bba85a3939. The test harness places the data directory under TMPDIR,
so connection initialization itself runs against the share, which is
where network-filesystem failures (startup crashes and hangs) occur.
The step timeout turns a startup hang into a failure.
On macOS, SQLite chooses locking methods based on the filesystem
containing the database, and network filesystems (e.g., SMB, NFS),
read-only volumes, and filesystems without byte-range locking get
methods without shared-memory support, which WAL requires. Opening a
database with an adjacent WAL file on those crashes -- Mozilla's VFS
wrapper hides the missing shared-memory methods from SQLite's WAL
support check -- so the first Zotero 10 run converted the database to
WAL and every launch after that segfaulted during connection
initialization.
On macOS, mirror SQLite's method selection and use a rollback journal
when shared memory isn't available, converting an existing WAL database
before opening it: an empty WAL by reverting the header format versions
in place, and a non-empty WAL by replaying it into a temporary copy on
local disk that replaces the database file only after passing an
integrity check. A WAL file next to an already-converted database
(e.g., from an interrupted conversion) goes through the same
conversion. Also use openNotExclusive during integrity checks and
corruption recovery, which otherwise fail on SMB shares with an I/O
error from the exclusive open lock.
This corrects 22055d92b7, which passed openNotExclusive on all
platforms for an open failure described as affecting macOS and Linux,
and expected locking_mode=EXCLUSIVE to keep the WAL index in heap
memory with no -shm file. Neither claim held up: mozStorage opens the
WAL while initializing the connection, before any pragma can run, so
the index uses shared memory unless the exclusive VFS is in use, and
the exclusive open works on Linux CIFS mounts -- where non-exclusive
access instead made SQLite's lock-upgrade sequence trip over the SMB
byte-range lock mapping, leaving startup hung and the database never
created. So pass openNotExclusive only on macOS. On Linux this restores
unix-excl, which performs all locking under a single held lock and
keeps the WAL index in heap memory; Windows has no distinct exclusive
VFS and is unaffected.
https://forums.zotero.org/discussion/133258/
rpl 2.1.0 guesses each file's character encoding instead of assuming
UTF-8, and aborts on some locale files, leaving the script's later
steps unrun. Pin the encoding, and use -exec + so a failure in one of
these passes actually stops the script.
Also drop the ⏎ replacement, which hasn't matched anything since 2013
and would now insert real newlines rather than literal \n, truncating
.properties entries.
The traversal waits on focus/blur events, which Gecko fires only while
the window is active, so a window that can't be activated -- another
app focused during a local run, or intermittent deactivation in CI --
timed out after 10 seconds instead of reporting that it couldn't run.
Firefox 153 removed the wizard's extra1 and extra2 buttons, so onLoad()
threw while caching their labels and never got to starting the update
check or adding the page handlers. Create the buttons ourselves instead.
i686 builds are gone as of Zotero 11, so a full build failed trying to
package a linux-i686 MAR from a staging directory that no longer exists.
Skip the architecture entirely, and don't download it for older FROM
versions when building incrementals.
Tests asserted exact search results for words like "one" and "two", so
a leftover item from an earlier test whose random title contained one
of the words as a substring could match and fail the assertion. Use
distinctive random strings instead.
EDTF dates -- ranges ("2021/2026"), uncertain/approximate dates
("2004-06~"), and BCE dates ("-0429") -- and common equivalent notations
("1995-1996", "2021-22", "~1995", "ca. 1995", "429 BCE") are now passed
to citeproc-js as CSL date ranges, circa flags, and negative years.
Previously, such dates were mangled or dropped entirely unless entered as
CSL variables in Extra.
CSL date variables in Extra get the same parsing.
Other date handling in the client doesn't understand EDTF yet: the y/m/d
indicator in the date field doesn't reflect EDTF parsing, date searches
only match a range by its start date, and BCE dates still can't be
sorted.
Addresses #637
The debounced count update from fe35127f50 went through itemSelected()
and fired on any notifier change, so a pending timer could bump the
itemSelected spy after a test reset it, failing "shouldn't select a
modified item" in CI. Emit rowCountChange only when the row count
actually changed, and re-render the item pane message directly.
The build-and-deploy scripts run scripts and config from the local
checkout but build source files from the tip of the remote branch, so
a stale or wrong-branch checkout could silently build with the wrong
Gecko version or omni patches.
md5Async() read and hashed the entire file synchronously on the main
thread. Read the file via an input stream pump instead, with file I/O
off the main thread and hashing done a segment at a time.
The test relied on the automatic selection of a newly created item,
which can lose the race with a view switch (e.g., from another library
selected by a previous test), leaving the info pane empty.
Notifier events that added or removed rows (e.g., items downloaded
during a sync) didn't update the no-selection message in the item pane,
so the count went stale until the selection changed. The row provider
now emits a rowCountChange event, and the pane re-renders the count,
debounced, when nothing is selected.
Zotero.Users.setName() updated the in-memory cache even if the
transaction was later rolled back, so a retry (e.g., of a failed sync
download batch) would skip rewriting the users row.
Saving each downloaded object in its own transaction made large
download syncs extremely slow, since every object required a separate
disk commit. Objects are now saved in batches sized by the existing
notifier batch ramp, and a failed batch is reprocessed individually so
that an error still rolls back only that object's save. The cached
parent item is reloaded on rollback, since it would otherwise keep
collections added via newParentItemCollections in memory and the retry
would skip writing them.
An error thrown from a commit callback rejected executeTransaction()
even though the transaction had been committed, so callers would treat
saved data as rolled back, and rollback callbacks (e.g., the notifier
reset) ran against committed data. Commit-callback errors are now
logged instead, and any error after a successful commit skips rollback
callbacks and is marked with 'committed' on the error object.
If a transaction was rolled back, callbacks queued via
addCurrentCallback('commit') stayed queued and ran after the next
successful commit, potentially applying in-memory cache updates for
rolled-back data.
Runs a function within a transaction that's rolled back at the end,
with rollback callbacks guaranteed to have run by the time it returns.
Replaces try/catch blocks in existing tests that threw and caught a
marker string from executeTransaction() to force a rollback.
the all-creators mode (fieldmode 2) concatenated
firstname || ' ' || lastname, so a two-field creator without a first
name produced ' last', which advanced search fields inserted with the
space.
purgeOrphanedContent() ran on every startup, unlike the other
data-object purges, which are gated on the purge.* prefs.
startQueueDrain() also ran an FTS5 'optimize' even when the queues were
empty.
Gate the former on purge.items and the latter on having indexed
something.
The menu labeled Annotation Type listed "Highlight annotation" and "Image
Annotation", from the strings the reader announces annotations with. Use
the short names, which existed for two of the six.
mozinference (which is currently CPU-only) is better for many tasks, but
plugins may want WebGPU for some features (e.g., chatbots), so just
follow Firefox, which currently enables it by default for Windows and
Apple Silicon macOS. Adds 5.8 MB compressed to the Windows installer.
Binding is meaningful for a condition that matches at every level -- a
tag bound to an attachment means the tag is on the attachment -- but a
group carrying one lost the binding as soon as the search was
serialized, so "items with an attachment tagged foo" couldn't be built.
The row label builder skipped values that weren't strings, so a numeric
cell -- a note count, or a pre-1000 year (which is parsed with
parseInt()) -- announced only its column
name.
Also switch to formatColumnName() while we're here, so row labels
resolve the column name the same way the header does.
The guard tested the newly clicked column's settings entry while the
delete targeted the previously sorted column's, so the old direction
survived when the new column had no entry, and clicking threw when the
old column had none.
Skip crashhelper (all platforms -- the crash reporter is disabled) and
desktop-launcher and nmhproxy.exe (Windows).
Also skip dxcompiler.dll (Windows, 22 MB), which is loaded only by
WebGPU, now also disabled via pref. Nothing renders with WebGPU, and
local ML -- including any future plugin API -- should go through
Firefox's inference libraries (mozinference/onnxruntime), which are
compiled CPU-only.
XULButtonElement only opens a type=menu button's menu when the button
is the event's original target, which worked before because xul.css
gave .toolbarbutton-menu-dropmarker 'pointer-events: none'. That rule
was removed along with the dropmarker element, so clicks on our
recreated dropmarker did nothing.
The menu icon element is now an <html:img> that renders the 'image'
attribute or 'content: var(--menuitem-icon)', and 'list-style-image' is
ignored, so iconic menuitems showed broken-image placeholders on
Windows and Linux. list-style-image is kept for native macOS menus,
which still read it.
Firefox 153 removed the chrome://browser/skin/window-controls/ SVGs and
now draws the caption buttons with Segoe Fluent Icons glyphs (falling
back to Segoe MDL2 Assets on Windows 10), which broke the Windows build.
Port the new browser.css button styling and stop keeping the removed
directory in build.sh.
An exact-match condition compared with SQLite's case-sensitive '=' and
skipped the normalized shadow columns, so 'publication is "review of
finance"' missed "Review of Finance" while every other kind of search
matched it.
Better fix than d5aacafe3f -- we don't need source-repo info in our
builds at all, so just disable MOZ_INCLUDE_SOURCE_INFO rather than
synthesizing it.
Firefox's official-build source-repo.h generator only auto-detects
Mercurial, so building from our git checkout left the file empty and
packaging failed with "no preprocessor directives found". Derive
MOZ_SOURCE_REPO and MOZ_SOURCE_CHANGESET from git in the mozconfig so
configure records them for any invocation, including a manual
./mach package.
app/win/build added the cross-compilation target to whatever toolchain
was default before switching to RUST_VERSION, so after a Rust version
bump the new toolchain lacked the aarch64/i686 target and the arm and
win32 builds failed at configure. Set the default first.
Bug 1974578 added oncommand to GlobalEventHandlers for the Invoker
Commands API, so the assignment in _initImportedNodesPostInsert() -- an
inert expando write until now -- started setting the WebIDL event
handler, and [LegacyTreatNonObjectAsNull] turns a string into null. That
wiped the handlers compiled when the pane fragment was imported, leaving
every inline oncommand handler in the preferences panes dead.
importNode() compiles the attribute handlers itself, so the loop can
just be removed.
Bug 1980856 removed both wantdropmarker support and the rule that gave
.toolbarbutton-menu-dropmarker its image, once Firefox's last consumer
stopped using it, so the New Item, Add Attachment, and New Note buttons
lost their chevrons, along with the ones in advanced search, the feed
settings dialog, and Scaffold.
Draw it with a background, since a <dropmarker> doesn't render
list-style-image, and use chevron-6 to match the quick search dropmarker
and the menu marker on our own buttons. The rule also moves out of the
toolbarbutton block, since a plain <button> can host one too.
Firefox now creates a clear button for <input type="search"> in chrome
documents regardless of layout.forms.input-type-search.enabled (bug
1655503), which doubled up with the one we draw ourselves.
There's no mac wizard.css -- the shared one from the global list is all
macOS needs, as on Linux -- so this only produced a missing-file warning
whenever a <wizard> was created.
Without -f, curl saves the error page and the download appears to succeed,
so a missing build fails later with a confusing "Unrecognized archive
format" from tar.
The sidebar customizations all broke: the category buttons are
moz-page-nav-buttons now rather than button[name="..."], and the Settings
and Support links moved out of <sidebar-footer> into a secondary nav. The
sidebar also has a heading now that duplicates the main pane's "Manage
Your Plugins", so hide the whole column instead of the pieces.
-moz-mac-help-button draws the question mark itself, and the Help label is
now painted over it at the bottom of the settings panes. Firefox hides the
label the same way for the Page Info help button (bug 2009625).
Babel strips the trailing newline from defaults/preferences/zotero.js, so
the platform-specific prefs appended after it shared a line with the last
Zotero pref.
SelectableProfile.sys.mjs now statically imports BackupService, which
statically imports chrome://browser/content/backup/backup-constants.mjs,
so startup failed once SelectableProfileService was instantiated as a
command-line handler.
nsMenuBarX now looks for a menuitem with the id menu_settings on Ventura
and later, falling back to menu_preferences only on older versions, so
declare both the way Firefox does.
Bug 2037682 removed nsIPK11TokenDB in favor of getting the internal key
token directly from @mozilla.org/security/internalkeytoken;1, and bug
2043434 replaced initPassword()/needsUserInit with changePassword().
Bug 2017957 made Cu.Sandbox freeze built-ins by default for
system-principal sandboxes, so the translation framework's assignments to
Promise silently failed and loading the framework threw on
Zotero.Promise.method.
Bug 2008041 made disabled, checked, hidden, collapsed, and selected
boolean attributes, so their value is empty and [disabled="true"] no
longer matches. Nothing sets any of them to "false", so matching on
presence alone is equivalent.
Bug 2008041's change to boolean attributes also covers hidden and
collapsed, whose UA selectors became [hidden] and [collapsed], so
setAttribute('hidden', false) now hides the element. Switch the setters
that can be passed a falsy value to toggleAttribute(), read them with
hasAttribute(), and match the [collapsed=true] selectors in our own
stylesheets to the new presence-only form.
Bug 2008041 made them html-style boolean attributes, so presence alone
means true and setAttribute('disabled', false) now disables the element.
Switch the sites that can be passed a falsy value to toggleAttribute(),
read them with hasAttribute(), and drop a disabled="false" from markup.
Sites that only ever pass true are left alone.
Bug 2009806 moved XUL checkbox click handling into C++ and dropped the
CheckboxStateChange event, so listen for command instead. The event now
only fires on user interaction, so the tests have to click the checkboxes
rather than assign to .checked.
Bug 2011326 moved remote type prediction into C++ and removed
E10SUtils.predictOriginAttributes() and getRemoteTypeForURI(). The
resulting TypeError meant nothing ever loaded in the hidden browser.
Translators are evaluated with the system principal, which worked until
now only because nsContentSecurityUtils::IsEvalAllowed() exempted any
profile with "JS hacks" present -- which for us meant
xpinstall.signatures.required being false. Bug 2038660 dropped that
exemption, so ask for eval explicitly.
findLogins() now throws, and removeLogin()/modifyLogin()/removeAllLogins()
are Async-suffixed and return promises. Without this, Zotero couldn't read
the stored API key and showed the corrupted-logins-database error on
startup.
There's no synchronous read left, so Zotero.Sync.Runner.enabled -- which is
consulted from places that can't await -- now reads a value that
hasCredentials() caches.
Bug 2043845 put the FORCE_ALLOWED_DTD content policy type behind a pref
that's off for everything but Thunderbird, so NS_NewChannel() refused the
load with NS_ERROR_CONTENT_BLOCKED and the XML parser silently skipped the
entity. Custom elements whose markup uses DTD entities then failed to
parse with "not well-formed XML".
Bug 2038660 applies "script-src chrome: resource: moz-src:" to every
chrome: document, which blocks inline <script>s and inline event
handlers, so the main window loaded but ran none of its scripts.
We should move our inline scripts and event handlers into separate files
so that we can drop the pref.
Mozilla removed the widget and its stylesheet in bug 1968916 without a
drop-in replacement, so bring both into Zotero. The two rules that lived
in Firefox's UA sheets (text-shadow and -moz-window-dragging) move to
_search.scss, and the aria-autocomplete removal that fetch_xulrunner
used to patch in is now part of the widget itself.
aboutaddons.js and aboutaddonsCommon.js were split into
aboutaddons-utils.mjs plus per-component modules under components/, and
the addon details template moved out of aboutaddons.html into
addon-details.mjs.
The unsigned-addon warning is now suppressed through Firefox's own
isUnsignedWarningMessageDisabled(), which is otherwise gated on
MOZILLA_OFFICIAL.
AboutTranslations and the PictureInPicture actors moved out of the
JSWINDOWACTORS literal into conditional assignments after it.
Anchor every pattern to the start of the line while we're here, since
'PictureInPicture: {' matched the MozTogglePictureInPicture event name in
the new file and silently deleted everything through the next ' },'.
The gap after the icon came from whitespace baked into the legacy
cross.png, so it disappeared when the icon switched to the
edge-to-edge cross.svg and the title abutted the icon.
While trying to [fix](https://github.com/NixOS/nixpkgs/pull/519431) another issue in the Zotero package for nixpkgs, I discovered a new bug. When setting a non-`source` update channel, the build fails on MacOS with x86 architectures.
The build fails with:
```
source not found twice in ChannelPrefs
source
stringWithCString:encoding:
```
Which definitely means it is caused by the `strings` command [here](https://github.com/zotero/zotero/blob/main/app/mac/set-channel-prefs-channel#L29).
When manually executing the strings command on the file, I can repeat getting only one `source` result on x86 mac, while getting two on arm64 or linux. After digging around, I tried with the flag `-arch all`, and both `source` values are found on x86 mac. Full command: `strings -n 3 -arch all $binary`.
Closes#5974.
Additional fixes for broken item tree behaviour when multiple items are
selected, and changing focus with ctrl/cmd-arrow keys.
Aligned Collection Tree/Virtualized Tree collapse/expand behaviour when
multiple containers are selected, one of them is focused, and arrow key
left-right is pressed, to the behaviour in Item Tree - now all of them
are collapsed/expanded.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
The PMC lookup in #5906 was added before the OA lookup, but the OA
lookup often returns a PMC URL of its own (including, at least now, for
the DOI given in the PR), with a direct file URL rather than a page to
scrape, so try it first and fall back to the PMCID-derived URL only when
it turns up nothing.
Since #5906, the 'oa' resolver list can start with a plain URL object
for items with a PMCID, but _getOpenAccessPDFURLs() and
saveOpenAccessAttachment() called resolvers[0](), which threw for those
items -- skipping the lookup silently in the first case and failing the
save in the second.
readAsText() already reads the charset from the blob's type when no
encoding is passed, so pulling it out first had no effect. The regex
was also missing a quantifier and captured a single character, which
readAsText() discarded as an unrecognized label.
Show File triggered a file 'open' notification, so revealing a file
moved its item to the top of Recently Read. Use a separate 'reveal'
event, which file syncing still observes to queue a modification check.
https://forums.zotero.org/discussion/133006/
- Fix multiple potential scenarios causing a template engine crash
- Add support for specifying string literals in the template engine
- Validate `if/else/elseif` order and matching clause closures
- Validate to ensure every `{{` is properly closed with a matching `}}`
- When a template is invalid, display a warning, do not offer batch-renaming tools, do not update synced setting
- When a template is invalid, prompt the user to fix or reset the template when
Closes#5965
Menu plugins reading it acted on one arbitrary row of the selection.
Reading it now throws and names collectionTreeRows, which the context
has already provided since multi-collection selection landed.
The context now copies property descriptors rather than values, since
copying values would evaluate the throwing collectionTreeRow getter for
every menu.
Collections and saved searches can be selected together, and the
search's items don't need to be in any of the collections, which would
result in search-only items being moved to the trash.
The view-wide branches (trash, duplicates, feeds, Recently Read) read
the first selected row, which multi-collection selection preserved with
a getter rather than updating. setCollectionTreeRows() now derives the
kind of view the selection adds up to, throwing if the rows disagree,
and those branches test it.
Multi-collection selection left .collectionTreeRow and similar in place
to reduce breakage, but that would just leave plugins and other callers
potentially broken when multiple rows were selected. All singular
getters now throw and say what to use instead. getSelectedLibraryIDs()
was added to replace getSelectedLibraryID().
Collections and saved searches can be selected together, so a
collection-item change called getDescendents() on rows that don't have
it and compared search IDs against collection IDs.
Recently Read rows can span libraries, but read attachments were marked
as matches for the first row's library only, leaving the rest as grayed
context rows.
Code walking the items list assumed every row was an object, so the
headers and spacers shown whenever more than one row is selected got
picked up as items: getSortedItems() passed them to export and report
generation, restoring from the trash called item methods on them, and
the item pane counted them in "N items in this view".
The requireServerIDOnWrite flag was added to the base endpoint class but
never set on the upload receiver, so uploads sent without the header
were still rejected with 428.
The cache-existence check didn't await getCacheObject(), so the
promise was always truthy and no cache object was ever saved for
objects the server reported as unchanged. Without a cached version,
later remote changes to those objects produced avoidable conflicts.
lastRead conflicts were auto-resolved only when a cached version of
the object was available, so differing read times could still trigger
the conflict resolution window.
The citation dialog's + column now has a tooltip that reflects how
many items a click will add based on the current selection, e.g.,
"Add 3 Items to Citation".
Clicking the + button on a row that is part of a multi-item selection
now adds all selected items to the citation, matching the existing
list-mode behavior. Clicking + on an unselected row still adds only
that item.
With focus moving back to the input after a multi-item add, a typed
number was applied as a page locator to every just-added item. It now
starts a search instead. Single-item adds still enable the typed-locator
shortcut.
_scrollItemTreeToRow() parsed the row index from the wrong rowID
segment, so it threw on every call, and adding items from the items list
never returned focus to the input as intended.
Every local API response includes a stable per-database server ID so
that clients can partition cached data, especially local object
versions, by Zotero instance. Write requests must include the ID, and
when it's provided on any request it must match the current server.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Add support for item, collection, and search writes (POST/PUT/PATCH/
DELETE), tag deletion, full-text writes, and the three-phase file
upload flow. Writes require a local API key granted by the user via a
confirmation dialog (POST /api/local/authorize) and follow the web
API's concurrency model, with If-Unmodified-Since-Version and
per-object version preconditions checked against local versions.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Add a clientVersion column for items, collections, searches, and
libraries, incremented once per library per transaction on every
object save or deletion. The local API reports these versions instead
of synced versions -- in object JSON, format=versions, since=
filtering, and Last-Modified-Version -- since synced versions don't
reflect local changes and are 0 for unsynced objects. Group metadata
responses keep reporting the synced group version, which has no local
counterpart.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Developer ID-signed Safari web extensions load only in Safari 18.4 and
later -- older Safari blocks them at the code-signing layer -- so macOS
11 and 12 users, whose Safari versions top out at 16.6 and 17.6, lost
the connector with the web extension conversion.
Embed the prebuilt legacy App Extension ($SAFARI_APP_EXTENSION, set by
the deploy scripts) at Contents/PlugIns/ZoteroSafariExtension.appex with
the historical bundle identifier, and move the web extension to
Contents/PlugIns/ZoteroSafariWebExtension.appex with a new
.SafariWebExtension identifier. The web extension's
SFSafariAppExtensionBundleIdentifiersToReplace key causes Safari
versions that can load it to hide the App Extension and migrate its
enabled state, while older Safari shows only the App Extension.
Also remove the removed-files entries for the App Extension's
resources, which would otherwise delete them on update.
https://forums.zotero.org/discussion/132925/
hdiutil makehybrid gives every file on the image non-empty Finder info
(it sets an icon location), which appears as a com.apple.FinderInfo
extended attribute. codesign --verify --strict rejects FinderInfo as
detritus, and it can cause Safari to fail to load the web extension.
(Reported by a user with Safari 17.6 on macOS 12. No other reports from
the beta, so maybe not universal.) The Safari App Extension apparently
wasn't affected, since it shipped with the same attributes for years.
Convert the hybrid image to a read-write image, mount it, strip the
attributes from its files, and compress from that. The volume header is
untouched, so the open-folder flag that makes Finder open the volume
window on mount is preserved.
Only fresh installs from the DMG were affected. The updater writes fresh
files without the attributes, so copies updated in place were clean.
(Notably, this is the reverse of the post-update extension breakage for
which the standard advice has always been to delete Zotero.app and
redownload. Following that advice is exactly what resulted in a broken
copy here.)
https://forums.zotero.org/discussion/132925/
The toolkit's default menuitem padding is em-based, so it balloons
with the UI font size in menulist dropdowns (e.g., the item type
menu), which inherit that font, forcing long lists to scroll. macOS
uses a fixed value on all menus and is unaffected. Match that on
Windows and Linux, slightly roomier, so every menu stays compact
regardless of font size.
https://forums.zotero.org/discussion/132797/request-for-compact-view-in-dropdown-menus
allowJavaScript: false didn't disable scripts for a system-principal
document, such as a blob: URL created from chrome code, which runs
scripts regardless of the allowJavascript flag. Sandbox the document
instead, forcing a null principal and blocking script execution.
The retraction checker fired a separate 'refresh' notification for
every matched item on each check, including already-known retractions
that hadn't changed. At startup this produced a burst of events that
forced observers like the item tree to rebuild repeatedly. Batch each
operation's changes into a single notification, and notify only for
items whose retraction state actually changed.
The candidate scan counted items whose Extra content parsed as fields
but couldn't actually be migrated (e.g., fields already set on the
item), so the window could appear on every global schema update without
changing anything.
Also use a dedicated headline instead of "Upgrading database…", and show
the window only if the migration is still running after 500 ms.
A stored file's path is normally 'storage:<filename>', which is parsed
as a string, but some ancient libraries apparently have relative paths
('../.../foo.pdf') that fell through to PathUtils.filename() and threw
NS_ERROR_FILE_UNRECOGNIZED_PATH, breaking syncing.
Resolve stored-file leaves as strings regardless of prefix and reserve
PathUtils.filename() for linked files, whose paths are genuine absolute
paths.
https://forums.zotero.org/discussion/132861/zotero-sync-issue-id-1987012678
The connector is now a Safari web extension, so the bundled appex is a
static wrapper stub whose placeholder resources are replaced at build
time with a connector build via $SAFARI_EXT_RESOURCES. The stub only
needs to be rebuilt when the native wrapper project changes, not for
connector releases.
ee68452b treated any backslash as a directory separator, in both the
attachmentPath setter and the userdata 128 migration. But backslashes
are technically valid on Linux/macOS and appear in real filenames (due
to LaTeX in titles, etc.), so that would wrongly throw on or truncate
them. Only a forward slash (never valid in a filename) or a Windows
absolute path (drive-letter or UNC prefix) reliably indicates a leaked
directory path, so treat just those as paths and leave bare backslashes
alone. We should make sure we're running getValidFileName() everywhere
to avoid saving those.
These conditions expanded into an OR-group across their underlying
fields, so a "does not contain"/"is not" operator matched almost every
item: any item missing one of the fields satisfied that field's negated
condition. Use an AND-group for negative operators, so the value must be
absent from every field.
https://forums.zotero.org/discussion/132835/
Selecting all libraries pulled in feed rows, which belong to a different
visibility group and so can't be shown together, collapsing the
selection back to a single row. Constrain the all-libraries scope to the
current selection's visibility group. Also preventDefault() the shortcut
so it no longer falls through to the cmd_selectAll key command, which
threw NS_ERROR_FAILURE.
https://forums.zotero.org/discussion/132843/beta-cannot-select-all-libraries
The download-rename test used a backslash as its sample invalid
character, which now throws in the attachmentFilename setter. Use a
colon instead, which still tests download-time filtering.
Someone ended up (via a plugin, presumably) with stored-file attachments
with a full path after 'storage:', which broke file syncing. Throw when
setting a stored-file path containing a slash, and strip paths from
existing filenames in a schema update step. No particular reason to
think that the file with that basename will exist in the storage dir,
but at least it will be looking for the right file and not be totally
broken.
Separately, the dataserver will clean up filenames with full paths and
block going forward.
https://forums.zotero.org/discussion/132822/reference-sychronization-error
Accented stop-words weren't matched by the existing unaccented
entries (e.g., "fur"), so automatic journal abbreviation kept and
capitalized them: "Jahrbuch für Heimatkunde" became "Jahrb. Für
Heimatkunde" instead of "Jahrb. Heimatkunde".
https://groups.google.com/g/zotero-dev/c/uP18QEKe2JU/m/AsoGWxd9AwAJ
If queued items can't be indexed and removed for any reason, the
startup and background drains would otherwise run forever. Stop after
three passes without the queues shrinking; the items are picked up
again on the next trigger.
If a queued attachment was missing its cache file and re-extraction
produced no text, nothing was recorded in the index state, so the
content-index queue selected the same items forever and the startup
drain never finished.
https://forums.zotero.org/discussion/132808/
Say "Attachments indexed"/"Partially indexed" instead of
"Indexed"/"Partial", reverse "File or full-text content not
available", and list notes last so the attachment rows are contiguous.
The progress popup was opened with alwaysontop, which is system-wide,
so a long-running popup (e.g., full-text indexing) stayed on top of
other apps' windows after switching away from Zotero. Open it as a
dependent window instead, which keeps it above only the Zotero window.
A term mixing CJK with non-ASCII words (e.g., Cyrillic plus Japanese)
was routed to the CJK index with only its CJK characters, matching
every document that contained those. Treat any non-CJK letter or digit
as making a term mixed-script, which falls back to a literal scan of
the cached text.
Replace the trigram FTS5 index for attachment content with a unicode61
word index, so terms match whole words with the final token as a prefix
("archive" matches "archives", but "ion" doesn't match "condition"), as
in the pre-FTS5 word index. A multi-word phrase gets adjacent-token
candidates from the index and is then verified against the cached text
of just those items, since FTS5 ignores what separates adjacent tokens;
the verification treats whitespace and hyphen runs as equivalent
(they're frequently extraction layout or styling) but requires other
punctuation to match literally. Notes keep the trigram index and CJK
matching is unchanged; the index database version is bumped so the
index is rebuilt.
Follow-up to #5979
JNA 5.x ships per-architecture macOS native libraries instead of a
single com/sun/jna/darwin/ directory, so the notarization re-signing
step failed with "filename not matched".
This step can be removed entirely once the plugin ships a jna.jar
without the macOS natives, which are never loaded.
SQLite replays a leftover -wal file (e.g., from a force-quit) into
whatever file next occupies the database path, so copying a backup
over zotero.sqlite produced a corrupted-database error, and the
automatic restore recreated the same mismatch and failed every time.
When corruption is detected, check if the database file is valid without
its journal files, and if so, save a verified copy and restart, swapping
the copy in at the next startup before the database is reopened. (The
copy is made before shutdown because SQLite automatically checkpoints
the WAL into the database file when the last connection closes, which
would write the stale WAL data into the file.) Otherwise, move journal
files along with the .damaged file to clear them from the main path
before restoring from the automatic backup or creating a new database.
Since a mismatched WAL can cause subtle data damage without errors, also
run a full integrity check at startup after an unclean shutdown. Skip
the explicit close-time WAL checkpoint once corruption has been flagged,
and truncate the WAL during idle maintenance to limit stale WAL data.
showZoteroPaneProgressMeter() now returns a token capturing the
previous message, meter mode and position, and owner, and
restoreZoteroPaneProgressMeter() restores that state if no other
operation has changed the display since, with nested tokens restoring
in reverse order.
The CSL locales repository replaced locales-sr-RS.xml with script
variants, so citing with a stored or default 'sr-RS' locale loaded no
terms and failed with an et-al error. Resolve unavailable locales to
the closest available CSL locale (sr-Cyrl-RS for sr-RS) when rendering
and in locale selectors.
https://forums.zotero.org/discussion/comment/515598/#Comment_515598
Backup copies preserve the database file's mtime, and the backup
interval is measured from the backup file's mtime, so a forced
pre-update backup of a long-idle database could be rotated out a day
early. Regular backups still keep the database mtime, which the
unchanged-database check relies on.
The backup before repairing integrity-check errors from the Advanced
pane has thrown a TypeError since backUpDatabase() started requiring an
options object in 8.0, aborting the repair. Also force the backup so a
recent automatic backup can't cause it to be skipped.
If a schema update committed the userdata upgrade but kept failing in a
later step, each restart forced another rotation backup of the
now-upgraded database, deleting the pre-upgrade backup after numBackups
restarts. Record the pending target versions in the database when
making a forced backup and skip the backup while they're unchanged, so
the pre-update backup survives the normal rotation period.
Replace the manually maintained minorUpdateFrom flag, which was easy to
forget to update when adding a slow upgrade step and couldn't account
for database size or hardware. The message is now shown for any upgrade
still running after 500 ms.
Versioned backups (zotero.sqlite.<version>.bak) were kept until the
next userdata upgrade, potentially bloating the data directory by
gigabytes, while minor upgrades made no backup at all. Now any userdata
upgrade, integrity check, or global schema update forces a rotation
backup. Since versioned backups ignored backup.numBackups, setting that
to 0 now fully disables backups, including before upgrades.
Show indexing progress as a bar (items in the search index out of those
that will be) that resolves to "up to date", alongside
Indexed/Partial/Notes counts and a "File or full-text content not
available" count for attachments with no local file or full-text
content. While the pane is open, trigger indexing directly so it
advances as the user watches.
Remove the Rebuild Index and Clear Index buttons. Rebuild Index marked
all content unsynced and re-uploaded it, triggering a server reindex and
re-download on other devices, and the per-item reindex paths now cover
what needed it -- raising a length limit re-extracts only the affected
items, and unindexed items are indexed automatically. Clear Index wiped
the local index and extracted-text cache files, a destructive action
that shouldn't be exposed.
Note content is indexed into fulltext.sqlite, making note searches
accent- and case-insensitive and matching the note's plain text rather
than its HTML markup. To avoid re-indexing on every auto-save, a save
flags the note for background indexing, and searches match a flagged
note from its normalized text in memory until it's indexed.
Closes#378
The condition `required` flag was removed in #5962, but the column was
kept so older clients could still read the database. The full-text
search changes bump the userdata compatibility version, locking out
those clients, so the column can now be dropped.
Index attachment content into a contentless trigram FTS5 table in a
separate, attached fulltext.sqlite, normalized so matching is accent-
and case-insensitive. For content containing CJK characters, a companion
'ascii'-tokenized table holds bigrams so 1-2 character CJK queries, which
the trigram tokenizer can't match, still work. The extracted text still
lives in the .zotero-ft-cache files, so the index is fully derived and
rebuildable.
Use the FTS index for the fulltextContent condition, falling back to the
cached-text scan for queries too short to index, and point quick
search's content matching at the FTS index in place of the now-removed
word index. (One side effect: quick search now matches attachment
content by substring rather than by word.)
Already-extracted content is migrated into the index at startup, slowing
down on active usage. A background queue then extracts not-yet-indexed
attachments gradually when Zotero is idle. Attachments with no local
file or full-text content are recorded as missing. Content downloaded
via sync is processed into the index immediately when the sync finishes,
rather than waiting for idle like it did before, so it's searchable
immediately in on-demand file-download mode.
The index DB is tied to the main DB via the local user key and rebuilt
if they don't match (e.g., after a delete-and-resync). We compact it by
running FTS5's 'optimize' command once the indexing queue drains, and we
vacuum the attached database when necessary to reclaim disk space.
Closes#2038, #2044
Addresses #1595
Code that attaches its own database to the main connection needs to hook
into that connection's lifecycle. Add onIdle() to run maintenance (e.g.,
vacuuming) during the main database's idle maintenance, and
addCorruptionHandler() to handle a corruption error for the attached
database (after confirming that the main database is actually corrupt,
so a corrupt attached database doesn't trigger main-database recovery).
Load a bundled SQLite extension (e.g., FTS5) by name. mozStorage
disables generic extension loading but allows specific bundled ones.
Extensions are registered per connection, so a loaded extension is
remembered and re-loaded automatically after a reconnect, before
onConnect() callbacks run.
Search now ignores accents, so "seance" matches "séance" and vice versa.
Text is normalized with Unicode NFKD compatibility decomposition (which
also handles typographic ligatures, superscripts, full-width forms,
etc.) plus a small map for letters NFKD leaves alone (ø, œ, æ, ß, ...)
and the fraction slash, via Z.Utilities.Internal.normalizeForSearch().
The HTML tags we support in item fields are stripped, so markup isn't
matched (#81). Typographic quotes (#29, #1876) and dashes are folded to
ASCII.
Each searchable column gets a normalized shadow column --
itemDataValues.valueNormalized, tags.nameNormalized,
creators.firstNameNormalized/lastNameNormalized, and
itemAnnotations.textNormalized/commentNormalized -- populated at write
time and matched via COALESCE(normalized, raw) LIKE. NULL is stored when
normalizing only changes case, so plain-ASCII values are only stored
once. This covers the contains/doesNotContain/beginsWith operators in
both quick search and Advanced Search.
The new columns are local-only derived data and aren't synced. Older
clients will ignore them, so this doesn't break DB compatibility.
Existing rows are backfilled after the startup sync by
Zotero.Schema.populateNormalizedSearchColumns(), which should only take
a few seconds on most databases.
Closes#29, #81, #1300, #1876
Rename the sync-delivered full-text content processor's methods and
state to "sync content" (registerSyncContentProcessor,
processSyncedContent, indexSyncedContent, getSyncedContentCacheFile,
etc.), so it reads distinctly from the local index-building queues added
for content search. No behavior change.
The document-worker bump raised SDT_PROCESSOR_VERSIONS.pdf from 2 to 3,
so the test's "valid" pack fixture (embedding version 2) was rejected as
stale-processor during generation, failing "should generate the pack when
missing."
viewAttachment() fired the 'open' notification before launchFile() saved
a sniffed content type, so AttachmentReadObserver's concurrent lastRead
save could reload the item and discard the pending change, leaving the
attachment with a blank or incorrect content type.
The #focus() Tab/Shift-Tab tests wait on focus/blur events, which fire
only while the window is active. It intermittently loses activation in
CI's Xvfb, so call win.focus() in beforeEach when the window isn't
active.
Temporary diagnostic for the intermittent shift-tab focus-test timeout.
A central afterEach in runtests.js tracks when
Services.focus.activeWindow stops being the main window; the shift-tab
test logs the last-active and first-inactive test in its timeout
message, to identify what leaves the pane inactive in CI.
This reverts commit 03a610e696.
activate() foregrounds a window via the window manager, which CI's Xvfb
doesn't have, so activateZoteroPane() in beforeEach just hangs there
instead of fixing anything. Restore the temporary open-window logging to
try to catch what leaves the pane window inactive.
An image pasted into a note is stored as a hidden attachment, so a
matching search condition (e.g., "# of Tags" is 0) would pull the
image's parent item into the results with no visible match.
Notes and attachments are counted on regular items, and annotations on an
attachment or across a regular item's attachments; other rows are excluded
rather than always matching with a count of 0. Trashed children aren't
counted.
The front-most tab's item appeared under Selected Items instead of Open
Documents, and the library tab's actual selection wasn't shown. Selected
Items now always reflects the item tree selection, and an item that's both
selected and open stays under Open Documents (sorted after the front-most
tab) so that suggestions specific to open documents (e.g., the current page
number, planned in #6001) can apply to it. Notes open in tabs now appear
under Open Documents when adding a note.
Opening the Advanced Search from the quick search cleared the search
text only on the first selected row, while the quick search had applied
it to all of them, so with a multi-collection selection the other
collections stayed filtered by the now-invisible text -- both while the
advanced search was open and after closing it.
With the saved-search editor open, Cmd/Ctrl-Shift-F replaced it with the
temporary Advanced Search pane and the search field's close button
closed it, in both cases discarding the editor's working copy with no
way to recover the edits. Move the collection-switch save prompt into a
shared helper and show it before replacing or closing the editor.
Choosing Save on a collection switch now continues the switch directly
instead of relying on the save's modify notification to re-sync the view.
onConditionSelected() populates the Author value menu asynchronously,
and until it finished, the row's default textbox was still the visible
value control, so serializing the row (running or saving the search)
replaced the loaded author with an empty value. Serialize the stored
value while the menu is pending, and invalidate the population if a
newer condition selection takes over meanwhile.
removeRow() remembered the removed row's index within its own group,
but when the removal emptied the group and pruned it, the index was
applied to the ancestor's children, focusing an unrelated row. Track the
pruned group's slot in its parent instead.
The pane-level Enter handler ran the default action (save in the saved
editor, submit in the temporary pane) regardless of the event target, so
Enter with focus on the Cancel button saved and closed the editor, and
Enter on Clear ran the search. Click the focused button instead, which
the platform otherwise only does for Space.
The "in the same annotation" binding menu was hidden as soon as a group
no longer had two conditions at the bound level, but the group's stored
binding was still saved and applied. After switching one of two bound
annotation conditions to another type, the group looked like a plain
"Match all of the following" group but still required a single
annotation to satisfy all of its conditions, and no control was left to
see or clear the binding.
Show the menu whenever the group is bound, drop the binding only once no
condition at its level remains, and refresh the menus before
collectGroup() in updateSearch() so the first save after such an edit
doesn't serialize stale state.
setAdvancedSearch() invalidated the row unconditionally, unlike
setSearch()/setTags(), so merely opening or closing the Advanced Search
pane (which applies a null filter) re-queried and rebuilt the items
list. A passed search is still always applied, since its conditions may
have been edited in place, and setFilter() still waits for a refresh
already in flight so that callers see the settled view.
_rollUpAnyToLevel() only followed an annotation's parent when the
attachment itself had a parent, so in a search for top-level items, a
tag on a standalone attachment's annotation matched nothing.
The condition seeded by _loadConditions() used `mode: undefined`, but
toJSON() only omits the "/mode" suffix when the mode is exactly false
(what parseCondition() returns), so a saved search migrated from the
obsolete childNote condition serialized -- and synced -- the condition
as "resultLevel/undefined".
_buildQuery() already uses the first top-level joinMode for the
full-text post-filter; with multiple top-level joinMode conditions (only
creatable via the API), combineConditions() used the last.
Extends existing handling for tag selection and quicksearch to handle
advanced search as well.
Fixes#5996 (not just for zotero://select but for all selectItems()
callers)
Instead of just prefixing the labels with "-", indent the whole row,
including the icon. This looks better and fixes FAYT on subcollection
names.
https://forums.zotero.org/discussion/132561/
The #focus() Tab/Shift-Tab tests dispatch synthetic keydowns and wait on
a focusin event to sync with the collections search field's async
(setTimeout) focus. Gecko only fires focus/blur events while the window
is active, so when the pane window wasn't active -- it opens in the
background on macOS/Windows and can lose activation on Linux/CI -- the
focusin never fired and the test hung until Mocha's timeout. Activate
the window in beforeEach, as the item pane and tags box focus tests
already do.
Also removes the temporary open-window logging added in e1f19f34f5.
Cache patches and compressed files by content hash so that they're reused
across FROM versions and releases. Cache entries are touched when used and
expire after 60 days.
The cache key includes a tag covering the xz/mbsdiff versions and the
compression options, so entries are invalidated when the toolchain changes.
Run the per-file mbsdiff/xz work in make_incremental_update.sh and
make_full_update.sh through a parallel job pool, largest files first, and
assemble the manifests serially afterward, producing byte-identical MARs.
Set UPDATE_PACKAGING_JOBS to override the number of parallel jobs.
Tests that shrink the window to test visibility-based rendering restore the
height on their last line without waiting for it to apply, so the next test
could start rendering while the window was still 100px tall. The attachments
box's async render is skipped for panes outside the viewport and only retried
on scroll, so the pane never rendered and assertions on it failed or timed
out.
waitForPreviewBoxRender() without an item id returns immediately if the box
still holds the previous item's completed render, so on a slow machine the
row assertions could run before the new item's async render started.
EPUB recognition translates the EPUB's RDF metadata, and
Zotero.Translators.save() requires translators to be loaded. The full test
run relied on an earlier test file having initialized translators.
Tests restore the stub at the end of the test body, so a failure left
Zotero.Translate.Search.prototype.translate wrapped and retries failed with
"Attempted to wrap translate which is already wrapped".
The reporter only showed the final failure, so if a retry failed differently
(e.g., from a stub left wrapped by the first failure), the original error was
lost.
Zotero.Cite.getLocatorString() requires Zotero.Styles.init() to have run.
The full test run relied on an earlier test file (bibliographyTest) having
initialized styles.
Pushes to branches in this repo already run CI, so run pull_request jobs only
for PRs from forks, which don't trigger push runs here. The push run's checks
still appear on the PR, since checks are attached to the head commit. (This
skips merge-commit testing for same-repo PRs -- the push run tests the branch
as-is.)
Each shard job builds and runs a quarter of the test files via runtests.sh -p.
The deployment ZIP upload and utilities tests move to separate jobs. With the
default fail-fast matrix behavior, a failure in one shard cancels the others.
The sorted file list is split into contiguous chunks of roughly equal total
file size, using size as a stand-in for run time, so new test files are
included automatically and slow test files are spread across shards rather
than landing in the same shard by chance. Contiguous chunks preserve the
alphabetical run order of a full run, so files keep the same preceding files
as in a full run except at chunk starts, and a shard can be reproduced locally
by passing its first and last files to -s and -e.
After a full build, dir_build saves a manifest of build/ files to
staging/.build-manifest. On subsequent runs, if all changed files are
ones that build.sh copies into omni.ja unmodified (chrome/, components/,
resource/, and test/ when tests are staged), zip just those files into
the staged omni.ja instead of rebuilding, taking rebuilds from ~15
seconds to ~0.3 seconds on an M1 Mac. Files are prescreened by size and
mtime so that only changed files need to be hashed. Zotero .ftl files
are also updated at their localization/<locale>/ paths, and test files
are also copied to the staged tests/ directory.
Any other change triggers an automatic full rebuild: files transformed
by build.sh (defaults/, chrome.manifest, version, translators/, styles/,
mozilla .ftl files, CSL locales), removed files, changes to build inputs
in app/ (detected via a size/mtime fingerprint, with xulrunner runtimes
covered by the hash-* files written by fetch_xulrunner), or requesting
tests or devtools that the staged build doesn't include.
Other changes:
- dir_build no longer takes -q and always skips omni.ja compression and
optimization, which only matter for distribution builds made via
build.sh. Use -f (dir_build or build_and_run) to force a full rebuild.
- build_and_run now always rebuilds. -r is deprecated, and -n skips the
rebuild and just launches the app.
- build_and_run no longer passes -purgecaches. Startup caches are
invalidated automatically when the BuildID changes, which now happens
whenever omni.ja is modified (including via add_omni_file), so
relaunching an unchanged build can use the startup cache.
- build_and_run and runtests.sh invoke js-build directly instead of via
'npm run', which saves ~270ms of npm overhead per build.
- The Word integration dylib is now ad-hoc-signed by dir_build, and only
on full rebuilds, since incremental updates don't invalidate the
existing signature. This also covers test builds, which were never
signed before.
- dir_build removes broken symlinks left in build/ when source files are
deleted, which previously broke rsync in prepare_build.
The reader now releases its rendered pages while its document is
hidden, which requires document.visibilityState to be truthful: it's
driven by docShellIsActive, which also controls event-loop throttling
for background tabs. The flag was maintained by scattered one-off
writes and went stale on the transitions they didn't cover: a tab
opened in the background ran at foreground priority (unthrottled rAF
and timers) with a 'visible' document until the next tab switch, and a
hidden tab kept an active docShell forever after its Read Aloud
playback was paused or stopped — in both cases keeping the reader from
ever releasing memory.
Replace the writes with a single _updateDocShellActivity() that
re-derives the state — active only while the tab is selected or
playing Read Aloud — called from the three places its inputs change:
tab creation, tab selection, and Read Aloud status changes.
If the preview fails, log the error and show "Preview unavailable"
instead of breaking the item-add flow with an unhandled rejection.
If io.sort() fails during accept, log and continue unsorted -- the
insert runs the same processor operation, so a real failure triggers the
document-update error dialog instead of a stuck progress window.
Skipping the sort doesn't affect the inserted citation, which the
processor sorts itself. It only determines the stored item order, and
with it the initial bubble order if the citation is edited later -- the
reopened dialog re-sorts once cited data has loaded.
Deleting a search in the collections pane moves it to the trash rather
than erasing it, so the check that closes the saved-search editor when
the edited search is deleted didn't catch it. The save-changes prompt
then appeared on the next selection change, and canceling couldn't
restore the removed row, so the prompt reappeared on every selection
until choosing Don't Save. Treat a trashed search like a deleted one
and close the editor without prompting.
Previously a search could be saved only at a library root. Now the Save
Search button is enabled when collections and/or saved searches within
a single editable library are selected, and the selection is added to
the saved search as collection/savedSearch conditions -- an 'any' group
of them when more than one row is selected. If the search's own join
mode is 'any', its existing conditions move into an 'any' group of
their own so the scope conditions apply to every result instead of
joining the OR. A 'recursive' condition is included per the
recursiveCollections pref.
https://forums.zotero.org/discussion/132528/beta-cannot-create-a-saved-search-from-a-collection
With "Hide Non-Matching Annotations" enabled, an attachment displayed as
empty and non-expandable if no annotations matched the search, so
searching by any non-annotation condition made it impossible to expand
attachments to browse their annotations. Now only hide the non-matching
annotations when the attachment actually has a matching one.
https://forums.zotero.org/discussion/132519/beta-advanced-search-cannot-expand-annotations-of-search-results
Previous command only worked with OpenSSL's labeled output where the hash is the second field; the update also handles LibreSSL (macOS), which prints just the bare hash.
When the item pane is dragged wide, the items pane is squeezed and the
quick search wrapper kept its intrinsic width and overflowed, pushing the
trailing Advanced Search button out under the item pane. Let the wrapper
shrink so the button stays within the pane.
Fixes#5982
The 'items in this view' count failed to update when the result set
changed without a selection change (e.g., quick search, tag filter, or
sort with nothing selected). The listener that refreshes the item pane
on refresh was registered in 8277277948 to fire only once, on initial
load, so later refreshes never recomputed the count.
This is all a regression from the items tree refactor (cbbff600a6),
which stopped running itemSelected() on every refresh.
- Fix a bug where focusing on a read-only field with a common value across items would clear the field and display a "Multiple" placeholder instead
- Fix read-only date fields always showing "Multiple" in batch edit mode, even when values are genuinely different
- Read-only fields are now focusable in batch edit mode
A field that exists at more than one level (e.g., Title, on both
top-level items and attachments) targeting a descendant result level
only matched the closest ancestor, so searching annotations by Title
found nothing, since it checked the parent attachment's title rather
than the top-level item's. Map down from each ancestor level and union
them, testing the predicate once so its bound parameters aren't
duplicated.
Addresses #5978
Saved searches were folded into the "Collection" condition's value menu
in 2016 (9c52ebdf8b), for reasons I can't totally remember. Give Saved
Search its own condition again.
Add annotationType, annotationColor, and annotationAuthor conditions,
each tagged `level: 'annotation'` so the cross-level search logic maps
and negates them correctly. The value fields are drop-down menus: the
annotation types, the reader's color palette, and the library's
existing annotation authors.
Ported from #5839. The PR kept a negated annotation condition (e.g.,
"Annotation Color" "is not" "yellow") from matching every non-annotation
item by checking whether the condition name contained "annotation". This
does the same using the condition's level, which the cross-level logic
already handles, so the existing annotationText and annotationComment
conditions are covered too.
Fixes#5837
The flag forced a condition to be ANDed even in "any" mode, but
condition groups now express that directly. It was never exposed in the
search UI and nothing seems to have been using it.
addCondition()/updateCondition() now throw if passed a truthy
`required`. Not dropping the column now to preserve DB compatibility.
Opening Advanced Search from a non-empty quick search reproduces the
current quick search mode as editable conditions, one per word (or quoted
phrase) joined with "all":
- Title/Creator/Year: a single Title, Creator, Year condition per word,
with the result level set to item (the mode matches only top-level items)
- All Fields & Tags: a single Any Field condition per word
- Everything: Any Field + Full Text Content as an "any" group per word
(relies on grouped full-text composing correctly in SQL)
The Title/Creator/Year mode needs a condition to map to, so add a "Title,
Creator, Year" search condition that expands to the same field set as the
quick search mode (title, publication title, short title, court, year,
citation key, creator), mirroring how Any Field matches All Fields & Tags.
Like Any Field, it expands at query-build time, so the saved search stores
a single condition and its sub-fields don't need their own entries in the
condition menu.
An Any Field condition expands into field/tag/note/creator conditions for
the term. Splice the expansion in right after the condition so it stays at
the same nesting depth, rather than appending it to the end of the
processing queue, which would emit it at the top level instead of within
its group. (Top-level Any Field is unaffected.)
Add annotation text and comments to the conditions the Any Field
condition expands to, matching the fields covered by the All Fields &
Tags quick search mode, as the comment already says is intended. Key
detection and quoted-phrase splitting still differ, since those depend
on the quick search string parsing that Any Field doesn't do.
- Reword the header as one sentence with a result-level menu ("Find
[attachments] matching [all] of the following:")
- Provide a per-group menu to bind the group's descendant conditions to
the same attachment, note, or annotation (e.g., one annotation that is
both red and contains a given word, not two different ones)
- Show a hint that offers to group ungrouped sibling conditions (e.g.,
two annotation conditions at the top level, to bind them to one
annotation)
- Show a warning when conditions can't combine at the chosen result
level (e.g., an annotation condition with a note result level)
- Remove the two legacy checkboxes:
- "Show top-level items" becomes result level = top-level item and is
migrated on save
- "Include parent and child items", which has no result-level
equivalent, keeps working, stays editable, and round-trips on
searches that already have it, but it isn't offered on new searches
and is removed on save if unchecked
Give a search a result level -- top-level item, attachment, note, or
annotation -- and map every condition to that level, so one search can
mix conditions that match at different levels of the item hierarchy
(e.g., a top-level item with a given author and a red annotation on one
of its PDFs). Each condition carries the level(s) it matches at: a match
is mapped up to an ancestor or down to a descendant, level-agnostic
conditions (tags) roll up to the result level, and fields that exist on
both items and attachments (title, url, accessDate) match natively at
either. The result level is stored as a `resultLevel` marker condition
alongside the join mode.
This also removes the temporary annotation-parent hacks, which the
general cross-level mapping replaces.
A fulltextContent condition was evaluated as a global post-filter keyed
on the search's top-level join mode -- correct for a top-level
condition, but not for one inside a group, which must combine with its
siblings under the group's own join mode. The new condition grouping UI
allows fulltextContent to be placed within groups, and we need to do so
to prefill the advanced-search pane from an "Everything" quicksearch.
Materialize a grouped fulltextContent into an itemID set and emit it as an
ordinary itemID IN/NOT IN predicate, so combineConditions composes it
under the group's join mode. Top-level fulltextContent keeps the existing
post-filter unchanged.
This also removes the quicksearch full-text post-filter special case. A
quick search puts its full-text in per-word "any" groups, so the
_hasQuicksearch flag was needed to make the post-filter union those
matches rather than intersect them under the top-level "all" join. Now
that the grouped full-text is composed in SQL it never reaches the
post-filter, so the flag is gone.
With condition groups, the position of conditions and the pairing of
groupStart/groupEnd markers are meaningful, but conditions were diffed
as an unordered member set (compared by value, with additions appended).
A sync-conflict merge could reorder conditions or add/drop group
markers, corrupting the group structure.
Diff them as a single ordered unit instead, like creators. This changes
how concurrent edits to a search are reconciled: rather than merging the
two sides' conditions member by member, a conflicting edit now resolves
wholesale -- the remote condition list replaces the local one (searches
auto-merge to the remote version); a one-sided change still applies that
side's full list. Discarding one side of a rare simultaneous edit is
acceptable and avoids silently corrupting a grouped search's structure.
This was the only user of SearchConditions.equals(), so remove it.
Render the search as a tree of groups: a root group plus nested
search-condition-group elements, each with its own join-mode menu and a
remove control. Each condition row gets a "( )" button that wraps it in
a new group in place, so further conditions can be added to combine with
it under a separate join mode. Switch the builder to rebuild-from-tree --
the DOM is the source of truth and the search's flat conditions (with
groupStart/joinMode/groupEnd markers) are regenerated on each edit, so
the old conditionID-as-index tracking is gone.
Replace the flat anySQL/quicksearch-block assembly in _buildQuery with a
recursive tree of AND/OR groups, built and reduced by a new
Zotero.Search.combineConditions() helper. groupStart/groupEnd markers
delimit nested groups and a joinMode marker sets each group's mode, so a
saved search can combine conditions with arbitrary nesting and per-group
join modes. The per-condition SQL generation is unchanged.
For example, a search built as
search.addCondition('joinMode', 'all');
search.addCondition('title', 'contains', 'foo');
search.addCondition('groupStart', 'true', '');
search.addCondition('joinMode', 'any');
search.addCondition('tag', 'is', 'x');
search.addCondition('tag', 'is', 'y');
search.addCondition('groupEnd', 'true', '');
means "title contains 'foo' AND (tag is 'x' OR tag is 'y')". The 'true'
operator on the group markers is an unused placeholder -- they carry no
value, but a condition's operator can't be empty.
The quick search (matching multiple words) and the Any Field condition
previously had their own special handling in the query builder; they now
use the same grouping as everything else, so that code is gone. Behavior
for existing non-grouped searches is unchanged; new tests cover nested
groups and combineConditions() directly.
A plain select-all would mix uncombinable rows, so scope it to the
current selection's type: a library selection expands to all library
roots; a collection selection to every collection sharing a parent with
a selected one (so multi-level/multi-parent selections expand within
each branches); and a Recently Read selection to Recently Read in every
library. Other rows -- saved searches, Unfiled, Trash, etc. -- have no
useful expansion, so the tree's key handler leaves the selection
untouched instead of clearing it.
Version Read Aloud cache keys with the server-provided cacheVersion so
a version bump misses stale entries and re-fetches correct audio, prune
obsolete entries once per session, and skip caching responses sent with
Cache-Control: no-store.
Regressed by f4adb452ee, which dropped the items-tree body's top padding
to 0 for sticky section headers; the scroll container's overflow then
clipped the top of the first row's focus ring, which drew outside the
row box.
The pinned sticky section header was opaque to paint but had
pointer-events: none, so clicks, drag-starts, and drops fell through to
the item occluded underneath it.
Make the opaque content capture pointer events instead. Clicks no-op
(header rows aren't selectable), drags don't start (header rows aren't
draggable), and drops on the header are rejected so they're a no-op
rather than acting on the list underneath.
Also drop the index > 0 exception when skipping non-selectable rows in
_onSelection(), so clicking a non-selectable row at the top (i.e., the
first library's header) is a true no-op instead of scrolling to the top.
Also skip non-selectable rows in handleActivate(), since the header can
now be double-clicked; without this it would try to open the library as
an item.
Fixes#5960
Arrow-key navigation scrolled the newly selected row flush with the top
of the view, leaving it hidden behind the pinned sticky section header.
Reserve a row's worth of space at the top so the row lands below the
pinned header.
Fixes#5959
#5658 added includeDeleted to the Advanced Search outside the trash, so
trashed items kept matching: trashing a result removed its row, but
re-running the search brought it back. Exclude deleted items by default,
the same as a quick search, and only include them when viewing the trash,
where the scope returns only deleted items.
Fixes#5956
When viewing the trash, trashed collections and saved searches were appended
to the items list unconditionally, so every advanced search (and quick search)
in the trash matched all of them. Now they're filtered by name during a quick
search and excluded entirely when an advanced search or tag filter is active,
since they can't match item-level conditions or tags.
Fixes#5957
The refactor dropped the 8px (COLUMN_PADDING / 2) offset that makes up
for the inline-start padding the first cell omits, so the header label
sat 8px too far left of the item titles.
Pin the header inside the scrolling body as a zero-height position:
sticky element so its width tracks the body's content box and it lines
up with the rows without any JS geometry. Reserve a scrollbar gutter so
the macOS overlay scrollbar doesn't float over the content (the opaque
header must paint above the rows to occlude them, and so above the
scrollbar, so it can't be put under it). Drop the body's top padding for
the item tree so rows clip exactly where the header pins, and keep the
gap below the column header as a margin outside the scroll area.
Fixes#5958
Route each item by its own library -- items already in the target
library are added directly (or skipped, for a library root), while
items from other libraries are copied in -- instead of attempting an
invalid cross-library insert. Disallow a move of such a selection
rather than silently copying.
Fixes#5961
The collections passed through Notes.open into the note window were never
read -- the note editor has no collections setter and EditorInstance.collection
is never assigned on this path -- so they had no effect. New notes are still
added to the selected collection(s) directly in newNote().
When multiple collection-list rows are selected, group the combined
items by library under sticky headers (e.g., "My Library", "Group X (2
collections selected)"), or show a single summary header for a multi-row
selection within one library, with blank spacer rows separating
libraries.
When stickySectionHeaders is enabled, the header of the section at the
top of the view is pinned in an overlay that the rows scroll under,
pushed up as the next section's header arrives. Consumers identify
header rows via isSectionHeader. Row striping restarts at each section
header so every section's first row is the same shade.
stickySectionHeaders defaults to false, so existing tables are
unaffected.
When the items list contains items from more than one library, group
them by library -- in collections-list order, independent of the active
sort -- with a section heading above each library's items.
Grouping is triggered automatically by an items list spanning more than
one library, not the kind of selection behind it, so any future source
of multi-library items would be separated the same way. Today the
cross-library collection selection is the only such source.
Allow selecting multiple collections, saved searches, or library roots in the
collection tree -- within a library or across libraries -- and show the union
of their items. The selection is threaded through the pane as an array
(getCollectionTreeRows(), changeCollectionTreeRows(), etc.); the item pane, tag
selector, reports, and export operate on all selected rows.
Adding items (new items and notes, drag-and-drop, the attachment dialog,
import, Add by Identifier) targets every selected collection. Only rows that can
share an items view may be combined: collections, saved searches, and library
roots mix freely, and multiple Recently Read rows can be combined across
libraries, but other special views (Trash, Duplicates, etc.) and rows from
different visibility groups can't be shown together, so a selection mixing them
keeps only the focused row. In-window advanced search runs across all selected
collections.
Advanced search value autocomplete is now scoped to the searched library, fixing
a long-standing TODO where suggestions were drawn from all libraries regardless
of the search scope; for a cross-library selection it spans the selected
libraries.
For a cross-library selection, the tag selector shows the union of tags
(colored tags only when a single library is in scope, since colors are
per-library), and deleting a tag spans all selected libraries while
rename/color/split are disabled. If advanced search is open, the
collection and saved-search conditions are omitted, since each is scoped
to a single library.
On a fresh profile, the default sort column had no explicit sortDirection,
so the first header click set the default direction instead of reversing
the sort, and no sort indicator was shown.
In other words, don't show the children that make the parent "recently
read" as context rows.
Children that don't match the Recently Read condition (such as notes,
as well as other child attachments that weren't read since the cutoff)
are still shown as context rows.
And add tests for the new behavior, and clean up an unused local var.
- Include the Advanced Search button in the quick-search tab sequence
- Tab: quick-search textbox -> Advanced Search button -> item tree
- Shift-Tab: Advanced Search button -> quick-search textbox
- Shift-Tab: item tree -> Advanced Search button (when the deck is
closed)
- Shift-Tab: Advanced Search collapse button -> New Item button
- Fix Shift-Tab from the item tree doing nothing when the Advanced
Search deck is open; now moves focus to the deck's last focusable
element
- add tooltiptext to expand/collapse and close btns
PubMed saves can include a PMCID even when DOI-based full-text lookup
fails to produce a PDF. In that case the connector asks the desktop
attachment resolver flow for fallback sources, but getFileResolvers()
only considered DOI and URL data, so Zotero never tried the matching
PMC article URL.
Treat PMCID as a valid signal for file lookup eligibility and add a
PubMed Central OA resolver before DOI-based OA lookup. Cover the new
fallback path in attachment resolver tests and the connector
hasAttachmentResolvers endpoint.
1. 862573eab added references to fileRenaming.ftl strings but didn't
add a <link> to it, so document.l10n.formatMessages() returned
nothing. It fell back to Zotero.getString(), which threw an error
due to a missing string in en-US. We didn't catch that, so the entire
search process aborted every time.
2. message.attributes is an optional array of { name, value } objects,
not an object itself, so we were never successfully adding FTL .title
and .label attributes to the parsed strings.
This fixes the 862573eab regression, and makes search match FTL
attributes as it was always intended to.
Fixes#5947
(Confirmed on macOS, but probably Linux too)
Pass openNotExclusive to avoid acquiring an exclusive lock at the OS level
at open time, which fails with an I/O error on network shares. We still set
locking_mode=EXCLUSIVE, so the connection holds an exclusive SQLite lock for
its lifetime and keeps the WAL index in heap memory (no -shm file).
Regression in Zotero 7 (Fx115)
Fixes#4860
The temporary advanced search saved silently under an auto-generated
"Untitled" name. Prompt for a name instead (defaulting to that name),
and label its button "Save Search…" to signal the dialog. The saved-search
editor's button, which saves changes directly, is now just "Save".
https://github.com/zotero/zotero/pull/5658#issuecomment-4696398750
Deleting the saved search being edited triggered onCollectionSelected's
save-changes prompt, which doesn't make sense for a search that's gone.
Detect the deleted search and close the editor without prompting.
The menu was focused programmatically (in response to the filter-button
click), so :focus-visible wasn't set and no focus ring showed. Pass
focusVisible to force the ring.
The remove button on the sole remaining condition is now enabled once a
value is entered, and clicking it replaces the condition with the default
empty one shown when the pane is first opened, instead of staying
disabled.
macOS-normalize-controls zeros margins on inputs and checkboxes but not
menulists or buttons, so their native platform margins threw off the
spacing. Zero them and restore spacing via the containers' gaps.
https://github.com/zotero/zotero/pull/5658#issuecomment-4696398750
The inner <deck> stacks both panes in one grid cell, so the area was
always sized to the taller (saved-search) pane and never shrank back.
The non-selected pane is now removed from layout entirely.
https://github.com/zotero/zotero/pull/5658#issuecomment-4696398750
When conditions are removed, shift conditionIDs so that
conditionIDs always go in increments of 1 (0, 1, 2 ...).
It prevents conditionIDs from conflicting with each other
when conditions are rearranged.
Fixes: #3434
After the first bubble is added, the focused input gets a placeholder
indicating that typing a number will add it as a page to the just-added
bubble. The placeholder is truncated if it's too close to the edge in
multi-item citations.
Also add a tip to the item details popup explaining that locators can
be typed into the main input field, with a link to the documentation.
The tip stops appearing once a typed locator has been used.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
- allow to accept the dialog via cmd/ctrl+Enter from
inside of a panel
- enable click-through on item details panel, so
one can click on the "Accept" button without having
to close the popup first
These two measures allow one to add a bubble, modify
the citation via item details popup, and accept it
with one less step (without having to close the popup first)
Zotero never sets a primary password, so if one is set on the NSS key
database, it was either corrupted or copied in from a Firefox profile,
and stored logins can never be decrypted, since there's no
primary-password prompt. This previously made it impossible to save
credentials without manually deleting cert9.db, key4.db, and
logins.json from the profile directory.
If reading or saving credentials fails and a primary password is set,
clear stored logins and reset the key database so that credentials can
be saved again. Saving the API key or WebDAV password is retried
automatically, so logging in completes without a manual fix, and if
previously stored credentials are lost to a reset, show a one-time
alert prompting the user to log in again.
The once-per-minute rate limit from e40fff6a7d (5.0.91) was inverted,
and the last-error time was never initialized, so the alert hasn't
appeared since 2020.
Wraps the values stored in nsILoginManager with OSKeyStore, which derives
its master key from Keychain on macOS, DPAPI on Windows, and libsecret on
Linux. A copy of the profile alone is no longer enough to extract these
credentials.
Existing plaintext entries are mirrored once per session to a new
"(encrypted)" realm but preserved in the original realm so a user can
still downgrade to a release that doesn't know about encryption. Active
credential changes (sign in, sign out, password change) write to the
encrypted realm only and remove the legacy entry. A future version can
clear any remaining legacy entries on startup.
Patches MOZ_APP_BASENAME in the bundled runtime so the keychain master
key is labeled "Zotero Encrypted Storage" rather than "Firefox Encrypted
Storage", with a check_line guard so a future Mozilla change to the
OSKeyStore label format fails the build instead of silently rebranding
the entry. Also fixes check_line to take an explicit file argument.
The local-file-change watcher added in f21e1b2d32 accumulated changed
item keys globally but was drained separately by each library's storage
engine, with the drained keys filtered to that library. The first
library to file-sync (normally My Library) consumed all pending events,
and keys belonging to other libraries were silently discarded, so files
modified on disk in group libraries were never marked for upload. The
initial and periodic full-scan fallbacks on Windows and Linux were
likewise global, so only the first library ever received them, and
changes made in other libraries while Zotero was closed were never
detected at all.
The sync runner now drains the watcher once per sync session and
immediately runs the modification check on the changed items across all
libraries, recording any changes in the database, and the per-library
storage engines skip the check entirely unless the watcher reports that
the library needs a full scan:
- On all platforms, a library that has never been scanned gets one full
scan, which also gives libraries one recovery scan for changes dropped
by affected releases.
- On Windows/Linux, where the watchers only capture events while Zotero
is running, each library gets a full scan on its first file sync of
the session, on every manual sync, and daily during background syncs
(instead of the previous 3-hour interval, which dated from when scans
were the primary detection mechanism).
- On macOS, libraries scanned since the last FSEvents journal
discontinuity are tracked in a pref, since the journal -- and
therefore the validity of previous scans -- survives restarts.
Also:
- Check FSEvents event flags and fall back to full scans when events
were dropped or coalesced (MustScanSubDirs/UserDropped/KernelDropped/
EventIdsWrapped), and skip HistoryDone sentinel events
- Disable the watcher for the session and fall back to legacy scanning
on backend errors, including when the inotify watch limit is reached,
instead of continuing with silently incomplete coverage
- Prune scan records for deleted libraries, since SQLite can reuse a
deleted library's libraryID
https://forums.zotero.org/discussion/132120/
On macOS 27 and later, the installer gets access to the Word startup
folder via a folder-selection dialog rather than an OS permission
prompt, so support an adjusted banner message and add strings for the
folder dialog.
createDirectoryForItem() wipes and recreates the item's storage
directory before moving in a downloaded file. If removeDir() failed
(e.g., a locked file on Windows), the error bubbled up to zfs.js and
became the generic sync error. Route it through checkFileAccessError()
instead, so the user gets the actionable locked-file message and a
Show Parent Directory button.
https://forums.zotero.org/discussion/132097/
In Add/Edit Citation mode, display a preview of the citation
in the bottom section. The section can be hidden/displayed
via the toggle in the right corner.
Remove io.preview from editor instance, so that citation
dialog knows not to show the preview even if the preference
is set.
A minor refactor to have resizeWindow() resolve
when the animation is fully over, and clear minHeight
on window in list mode before resizing, restoring
it when resizing animation is done, same as in library mode.
It allows us to fully expand the window in list mode before
showing the preview.
Fixes: zotero#5910
The item tree's DOM id carries a view-specific suffix (e.g.
"item-tree-main-default", "item-tree-main-recentlyRead"), but three
call sites compared against a single hardcoded "item-tree-main":
- Collection highlighting on Ctrl/Option (zoteroPane.js) -- match on
the "item-tree-main" prefix to cover all views. This restores
highlighting in Recently Read, where it silently failed.
- Focusing the items list after Add Item by Identifier (lookup.js) --
use the current view's tree id instead of a literal that resolved to
null and threw.
- Shift-Tab from the item tree to the toolbar (zoteroPane.js) -- key the
actionsMap on the current view's tree id.
Add a test confirming focus lands on the items list after a lookup.
https://forums.zotero.org/discussion/130968/collection-of-selected-papers-is-not-highlighted-in-recently-read-panel
The earlier fix (ef7896ab5f) migrated 9.0.x's "<id>-default" prefs
forward to the refactor's new "<id>" key, but that left 9.0.x and 10
beta keying differently in treePrefs.json, so switching between versions
always reset 9.0.x to defaults.
Put the `-default` suffix back so Z9 and Z10 read and write the same
key, and migrate any unsuffixed `<id>` data written by an earlier beta
back to `<id>-default` on first load. The migration runs once per
process and can be removed in a future release once beta users have all
updated.
https://forums.zotero.org/discussion/131653/zotero-10-beta-5-reinitializes-the-columns-in-main-pane
example:
"Atmospheric Chemistry and Physics" is currently incorrectly abbreviated as "Atmospheric Chem. Phys." should be abbreviated "Atmos. Chem. Phys."
Zotero.Tags.init() keyed the id cache by the raw tag name from the
database, but getID() and Items._loadTags() both normalize names via
cleanData() (trim + NFC). A tag stored in a non-normalized form (e.g.,
non-NFC or with surrounding whitespace) could therefore never be matched
by getID(), which returned false. When such a tag was removed from an
item, that false was bound as the tagID parameter in the DELETE FROM
itemTags query, throwing "Invalid boolean parameter 1 'false'" and
aborting the save. Since this fired while applying changes during sync,
it persistently blocked sync.
To fix, normalize names in init() to match getID()/_loadTags().
https://forums.zotero.org/discussion/131901/error-report-seems-to-be-preventing-sync
vacuum() and the APFS-cloning offline backup path both close and
reopen the SQLite connection, which drops all ATTACHed databases --
including the in-memory "indexing" alias used for the fulltextWords
scratch table set up in Fulltext.init(). Once the connection is
reopened, indexing queries fail with "no such table:
indexing.fulltextWords", and indexItems()'s ignoreErrors path
routes the error to logError(), so indexing silently stops working
for the rest of the session.
Add an onConnect() hook on Zotero.DBConnection for per-connection
state that doesn't persist across reopens, and use it from
Fulltext.init() to re-attach the indexing DB on each reconnect.
Regression sources:
- 67288047f3 ("Use APFS cloning for file copies on macOS") flips
online idle backups to offline-with-clone on APFS, so every idle
backup interval (24h default) closes and reopens the connection.
Affects Mac users on APFS.
- b27c4cb023 ("Enable SQLite WAL mode and add periodic VACUUM
INTO") adds the vacuum path, which closes and reopens on the
first idle period in a session that passes the freelist/time
gates. Affects all users with sufficient DB churn, roughly once
per 14 days.
https://forums.zotero.org/discussion/131576/debug-id-d848621212-indexing-of-pdfs-fail-zotero-9-0-3https://forums.zotero.org/discussion/131718/possible-bug-regression-report-search-unusable-on-macos-zotero-9-0-3-with-large-library
For a dependent CSL style with no default-locale of its own, citeproc-js
parses the parent's XML and silently uses the parent's default-locale
over any user-selected locale, making the locale dropdown a no-op. Add
a Zotero.Style.effectiveLocale getter that falls back to the parent's
locale, and use it in updateLocaleList and the locale-selector custom
element to disable the dropdown in that case.
https://forums.zotero.org/discussion/comment/512891/#Comment_512891
The general HTTP layer's automatic 429/Retry-After retry only retries
the one failed request, but the sync layer wants to pause its entire
batch of concurrent requests via concurrentCaller.pause(). Add a
noRetryOnThrottle option to Zotero.HTTP.request() that throws on 429 or
503 with Retry-After so the caller can apply its own throttling, and
have syncAPIClient set it. Extend sync's catch block to also honor
Retry-After on 503 (previously only invoked via _check429).
When a user's account is deleted and then undeleted on the server, the
dataserver creates a fresh shardLibraries row at a low version, while the
client still has a much higher local library version. Subsequent syncs
then throw "_libraryVersion cannot decrease" and the user is stuck.
Tag the cannot-decrease errors from the library version setters with a
named error, catch it in Engine.start(), reset libraryVersion and
storageVersion to -1, and restart. The retry hits the existing
libraryVersion == -1 branch and runs _fullSync(), which re-uploads the
local library against the recreated server library.
Previously only 503 + Retry-After was retried automatically; 429 was
handled only inside the sync API client. Extend _retryOnServerError()
to also retry on 429, honoring Retry-After on both 429 and 503 and
falling back to the existing exponential backoff otherwise.
In list mode, sort libraries by the count of their items
cited in the current document, falling back to alphabetic sorting
when counts are equal, with "My Library" always getting
priority over other groups.
Fixes: #5924
The test opened an advanced search window but never closed it, which
likely caused the intermittent failures in the ZoteroPane focus() test
"should shift-tab across the zotero pane".
Ignore persisted compatibility flag from previous app version in non-stable releases to fix disabled for wrongly using old flag
Ignore max version compatibility for plugin update check on non-stable releases
Relevant: https://forums.zotero.org/discussion/131096/
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Before the refactor, main library column prefs were keyed under
"<id>-default", because the visibilityGroup getter returns 'default'
(truthy) for the main library. The refactor changed the suffix logic to
only append non-default groups, leaving existing prefs orphaned at
"<id>-default" while the new code reads/writes "<id>".
Fall back to the legacy key on load when the new key is missing or an
empty object (which can be written out by an unmodified post-refactor
build that flushed prefs on a visibility-group switch, like switching to
feeds), and drop the legacy key on the next write.
https://forums.zotero.org/discussion/131202/lost-columns-displayed-choice-in-the-items-tree-after-update-to-10-beta-4
Covers the regression fixed by a7d001fc1d. The prior test for
_setHighlightedRowsCallback() called the callback directly, bypassing
the focus check and the keydown handler.
After the item tree refactor, the items tree id became "item-tree-main"
instead of "item-tree-main-default". Update the remaining call sites,
restoring collection highlighting on Ctrl/Option, focusing the item tree
after Add Item by Identifier, and Shift-Tab focus movement from the item
tree to the toolbar (which only still worked due to the native tab
order).
HTTP.download() was rewritten in 0fe31b0f04 to use fetch() and build the
Basic auth header itself via btoa(username + ':' + password), but the
username and password come from nsIURI.username/password, which are
percent-encoded. As a result, a username like "user@example.com" was
sent as "user%40example.com", causing a 401 on every WebDAV download for
any user with @, :, space, etc. in their username. Other request types
were unaffected because they go through xmlhttp.open(method, url, true,
username, password), which decodes internally.
Decode username and password in _parseURI() so the values returned can
be used directly for Basic auth (and as a side effect, fix the other
display/use sites that were getting the percent-encoded form).
https://forums.zotero.org/discussion/131174/zotero-10-betas-1-2-3-cant-download-from-webdav-http-401
Use one registry to avoid competing of language detect between plugins,
as resources declared as optional is identical to resource missing and thus fails the check and returns null for the string (https://searchfox.org/mozilla-esr140/source/intl/l10n/rust/l10nregistry-rs/src/registry/asynchronous.rs#140)
Register the plugin FTL for all languages Zotero supports with proper fallback logic so that even resources are declared as required, the check doesn't fail when plugin doesn't provide the resource.
Fix Zotero.File.getResourceAsync to use NetUtil channel to handle jar: url with `@`.
In a newly created profile, we check whether another profile is using
the default data directory, and if so, we create a new data directory
named after the profile (e.g., "Zotero Work"). But on Windows, paths in
prefs.js are stored with escaped backslaches (C:\\Users\\foo\\Zotero),
so searching for the JS string with literal single backslashes
(C:\Users\foo\Zotero) always failed, we would conclude that the default
dir was unused, and we would reuse the existing database.
To fix, escape backslashes in dataDir before the substring check.
Mac/Linux paths have no backslashes so this is a no-op there.
The new-install branch in DataDirectory.init() read prefs.js from the
default Firefox profile to detect a pre-2017 dataDir setting, and the
read was unwrapped, making prefs.js access errors fatal at startup [1].
We could add a try/catch, but after nine years, it's probably safe to
just remove the migration. If anyone is returning from >9 years in the
wilderness and they still want old unsynced Zotero data, they can
manually move their data to the default location.
[1] https://forums.zotero.org/discussion/131176/installation-error-accessing-mozillas-pref-js-see-msg-pls
Columns now have properties: `enabledIn`, `disabledIn` and `defaultIn`,
corresponding to column picker availability and default visibility. The
properties now filter based on attached collection view type instead of
visibilityGroup.
Visibility groups are for views where we want distinct column sets to
persist, like the feeds view.
Collection type properties are used to specify which columns are
available for a given type, regardless of whether it's in a different
visibility group or not.
Split ItemTree megaclass into:
- ItemTree - concerned with drawing the virtualized table container and
column interaction
- ItemTreeRowProvider - provides rows and issues notifications for
render updates
- ItemTreeRow and subclasses - contains row-specific data and rendering
logic
- CollectionViewItemTree and its accompanying classes - a version of
ItemTree that renders items attached to a given Collection or
CollectionView (CollectionTreeRow).
Various improvements in logic and rendering, separation of concerns.
When a custom data directory is on a network share or in a cloud storage
folder, SQLite can fail to open with NS_ERROR_STORAGE_IOERR and the user
previously had to manually edit prefs.js to recover. Show a "Use Default
Location" button in the startup error dialog that resets the data
directory pref and restarts.
Addresses #4860
Repo moved to zotero/document-worker on GitHub
After pulling, run:
git submodule sync
git submodule update --init document-worker
If an old `pdf-worker/` directory is left behind, it can be removed manually.
Add a spy on Zotero.HTTP.download and update call-count assertions to
split between request() and download() calls. Previously download()
delegated to request(), so a single requestStub tracked everything.
- Replace XHR-based download with fetch() + response.body streaming,
writing chunks to disk via IOUtils instead of buffering the entire
response in memory
- Separate out the retry and URL-parsing logic so it can be reused
between request() and download()
- Split the ZFS download code into a request() with `followRedirects:
false` to get the metadata headers and a separate download() to
download the file
Fixes#5476, Downloading of large files is broken
- Restructure the pane layout: account info (username, email,
login/logout) at the top, followed by a separate "Sync" section with
an intro description and "Learn more" link, data sync options, and
file syncing
- Migrate hardcoded "Zotero storage" strings to FTL and fix
capitalization
- Rename the "Zotero" storage dropdown to "Zotero Storage".
Don't throw when updateDisplayTitle() retry fails after loading
annotation data for a corrupt item. Also add an integrity check to
delete annotation items without itemAnnotations rows.
Fixes https://forums.zotero.org/discussion/130872
These tested racing an offline backup against an in-progress online
backup, which wouldn't happen in practice (schema backups run at startup
before the idle observer, and someone is very unlikely to perform a DB
integrity check immediately after returning from an idle that triggered
a backup). The tests relied on fragile timing and a shared tmp file,
causing failures on Linux after the WAL checkpoint change. The tests
were already skipped on macOS after switching to APFS clones.
- Switch journal mode from DELETE to WAL for better write performance.
With EXCLUSIVE locking mode, SQLite uses heap memory for the WAL
index, avoiding an -shm file. Set synchronous=NORMAL (matching what
Mozilla uses for Places). Checkpoint WAL on database close so the
.sqlite file has all data (for copies or backups).
- Add periodic database compaction on idle (after DB backup) using
VACUUM INTO and do an atomic file swap back to zotero.sqlite if no
writes occurred during the operation. Check if vacuuming is needed
based on time interval (default 14 days) and freelist ratio (default
10% threshold).
- Disable auto_vacuum, which causes fragmentation and is unnecessary
with periodic VACUUM
- Remove the VACUUM call from the integrity check, which was always just
an awkward hack to let people trigger a VACUUM without having an
explicit button
Closes#652
Fetch pinned-incrementals-{platform} from the deploy server and merge
with the normal last-N incrementals list. This ensures incremental MARs
are always built from key versions (e.g., last release of each major
version) regardless of how many newer versions have been released.
Split build_and_deploy into build_for_deploy (builds, uploads to S3,
rsyncs manifests) and a deploy script in client-downloads (updates
updates JSON, updates incrementals, runs deploy command).
Release scripts support -b flag for build-only (deploy later manually).
Beta/dev/test scripts build and deploy automatically via SSH.
Move incrementals files from S3 to deploy server. Update DEPLOY_PATH to
root of client-downloads so manifests and deploy script paths are
derived from it.
The `_getColumns()` column cache key didn't account for library type, so
switching between personal and group libraries reused cached columns
from the previous library type. This caused `groupLibrariesOnly` columns
(Added By, Modified By) to be missing in group libraries or incorrectly
shown in personal libraries.
Additionally, `VirtualizedTable.componentDidUpdate()` only recreated its
internal `Columns` object when the id prop changed, not when the columns
prop changed, so even with correct column rebuilding in `ItemTree`, the
table wouldn't pick up the new column list.
https://forums.zotero.org/discussion/130521/added-by-column-missing
Add Zotero.File.copyFile(), which uses clonefile() on APFS with a
fallback to IOUtils.copy(), and use it for all significant file copies.
APFS is detected and cached for each parent folder via
Zotero.File.isAPFS(), which uses statfs().
On APFS, all database backups now use the offline (close/clone/reopen)
path instead of the SQLite online backup API. Cloning is nearly
instant, and backup files share disk blocks via copy-on-write, saving
potentially gigabytes of space.
Closes#5330
Move file-renaming-* and rename-files-preview-* strings from zotero.ftl
to a dedicated fileRenaming.ftl. Rename rename-files-preview-* keys to
file-renaming-preview-window-* for consistency.
- prevent the focus handler from resetting the cached
items in search handler
- wait for search to finish after changing dialog type
in "should not display empty note child rows" test
Pass filterChildItems to itemTree from citationDialog
to filter out unwanted child rows before adding them
in toggleOpenState.
In annotations mode, do not add child notes and
attachments without annotations
In add-note mode, only keep child notes that have content
- updated annotations dialog type icons for dark and light mode
- use checkmark instead of right arrow for "accept" button
- fix incorrect sizing of "accept" and "cancel" icons
- Allow different min-width for list vs library layouts.
In library mode, we need at least 1000px to have a functional
itemTree, especially when sidepane is present. But for list
mode, 1000px is very wide, so min-width should be 800px.
Increased min-height of itemTree from 200px to 400px
- When the dialog closes, store the width/height of
the current dialog's layout. Then, when the dialog opens
next time in that layout, set window's width/height based on last saved
params. That way, one can open a citation dialog in list
mode at 800px, then open annotations dialog (always library mode)
at 1000px, and then have the citation dialog in list mode open
again at its intended 800px.
- If one switches from library to list mode, add a smooth
window resizing animation to shrink diagonally from current width
to 800px. If one switches from list to library mode,
window smoothly resizes to 1000px min-width. This is
not applied to linux where win resizing behavior is a bit jumpy.
If the server returns a write report with 403 for admin-only settings,
mark successful settings as synced and reset rejected settings to
their remote values by re-downloading from the server.
Needed for #5862
- fix breakage when initially setting annotation dialog
mode to library
- on click of a non-annotation item in suggested items,
select the top-level library if the item could not be selected
in the initial state of the itemTree
- increase min-width of the dialog to 1000px to ensure
there's always space for the sidebar.
Fetch emails from /keys response and login session result, persist them
in the local database, and display them in the sync settings for both
logged-in and logged-out-but-linked states.
Also unify the separate logged-out-linked and logged-in containers into
a single layout that toggles elements based on login state.
Requires dataserver and stream-server changes
When the server returns an attachment without lastRead (because it was
cleared), fromJSON was not clearing the local value since it only
processes fields present in the JSON. Add explicit check after field
processing to clear lastRead when absent.
Use itemID offset to advance through batches instead of comparing
result counts, which would bail out when the number of remaining items
equaled the previous batch size.
Backfill groupItems rows that have lastModifiedByUserID but null
createdByUserID (not totally sure how), and track completion with a
per-library flag in the settings table so the backfill only runs once
(since createdByUserID could potentially be permanently unavailable).
Fire a modify notifier event after _updateGroupItemUsers() so the
item tree refreshes to show the newly backfilled Added By / Modified By
values. Also change error handling from return to break so partial
progress still triggers the refresh.
When a lastRead synced setting for a group item was downloaded and
applied via AttachmentReadObserver.notify(), the group attachment item
was saved without skipSyncedUpdate, causing it to be marked as synced=0.
On the next sync, this could trigger an upload attempt that failed with
403 for read-only group libraries.
When a user logs out without removing local data, the sync pane now
shows a third UI state with the linked username, a "Log In" button
(constrained to the same account), and a "Switch Accounts…" button
that shows a hard confirmation dialog requiring the user to type
"remove local data" before proceeding. On confirmation, local data
is reset and the app restarts, reopening the account preferences.
Deleted settings were written to syncDeleteLog but never uploaded. Add
dedicated setting deletion upload in _startUpload() using individual
DELETE requests per setting, since the API doesn't support batch setting
deletion.
The same skipEditCheck fix from 5dc9705294 but for the synced setting
handler, which updates lastRead when a lastRead setting arrives via
sync for a group attachment.
Replaces the login form with a "Log In" button that creates a dataserver
login session and opens the website for logging in. The client polls for
session completion every few seconds and also watches for a notification
from the streaming server.
- Non-force delete clears `lastRead` on all child attachments (with
prompt), force delete moves to trash (matching collection behavior)
- Re-run search on item modify in Recently Read view so items disappear
when lastRead is cleared (also handles sync changes)
- Add columns as options in item tree, visible only in group libraries
- Fall back to createdByUserID for Modified By when lastModifiedByUserID
is not set
- Update lastModifiedByUserID on local saves when dateModified changes
- Fix backfill in _updateGroupItemUsers() to process all batches
- Update formatColumnName() to support Fluent string keys
Closes#233
When stop() or stopOnError rejected queued task promises, the rejection
handlers (via Promise.allSettled) weren't attached yet, causing Mozilla
to report each one as "uncaught exception: Object". Add no-op catch
handlers before rejecting so the runtime knows they'll be handled
downstream. Also fix stopOnError path to reject with CanceledException
instead of undefined, and remove legacy Bluebird handledRejection flag.
Track when attachments are last opened or read, storing a `lastRead` Unix timestamp on the attachment. For user library items, `lastRead` syncs as an attachment property in item JSON. For group library items, it syncs via a per-user synced setting (like `lastPageIndex`).
- Add `lastRead` column to `itemAttachments`
- Add `AttachmentReadObserver` to update `lastRead` on file open and page change (throttled to 5 min for page changes)
- Add "Recently Read" virtual collection (items read in last 14 days, sorted by `lastRead` descending)
- Add `lastRead` search condition with date operators
- Add `lastRead` item tree column with new `dependsOnChildren` property for parent item aggregation
- Add `getItemLastRead()` to return max `lastRead` across child attachments
Also:
- Generalize collection tree SCSS to support universal (context-fill) icons alongside themed icons
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
After editing the parent item, switching back to the attachment item before the parent item's save completes creates a race condition between the save and the attachment render.
Moved item pane tests related to file renaming to a separate describe() block.
The firstCreator/sortCreator SQL now falls back through primary →
editor → director → contributor. This ensures Video Recording items
with only a Director creator show that creator in the items list.
Also refactors the repetitive SQL-building in _getFirstCreatorSQL()
and _getSortCreatorSQL() into helper functions.
Closes#5720
The %PDF- sniffer entry had no offset constraint, so it matched anywhere
in the file sample. This could cause ZIP files containing PDFs to be
misidentified as application/pdf when opened, since %PDF- could appear
within the first 200 bytes of the archive. Pin the match to offset 0,
where it must appear in a valid PDF.
Fixes https://forums.zotero.org/discussion/130161/zip-files-and-indexing
Plus guidance-panel changes:
- Fix description not updating when multiple panels exist
in the document
- Fix nonfunctional noautohide attribute
- Show "Got It" button for noautohide with no navigation
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
resetData() clears user data and sync state using application APIs
instead of doing a full Zotero shutdown/reinit cycle, making it much
faster for tests that just need clean state between runs.
Also make Tags.init(), Creators.init(), Users.init(), and
SyncedSettings.loadAll() properly clear their caches before reloading,
so they are safe to call more than once.
Instead of testing against the build/ directory directly, zip it up first
and then unzip it back into build/ before running tests. This ensures
tests run against the same artifact that gets uploaded to S3 and used by
the deployment scripts, catching issues that only manifest after a zip
round-trip (e.g., resolved symlinks, missing files).
Also makes Ruby setup conditional on deployment branches
Similar to how quickSearch is cleared already.
To avoid rerunning itemTree refresh twice (once for
clearing quickSearch and once for clearing tag filter),
clear tags and search text directly on collectionTreeRow.
And then call itemTree.refreshAndMaintainSelection to
refresh everything. Without it, tag selector sometimes
would find itself out of sync with current search
results and display tags as if quickSearch was not
cleared.
Fixes#5601
Use platform-native file change APIs to track which stored-file
attachments have been modified, avoiding expensive full scans of all
attachment files during sync.
Backends:
- macOS: FSEvents (persistent event journal, survives restarts)
- Windows: ReadDirectoryChangesW (live recursive directory watch
via overlapped I/O polling)
- Linux: inotify (live per-directory watches)
On unsupported platforms or on error, falls back to existing scan logic.
Don't mark attachments for download when server has no file (mtime is
null), and clear TO_DOWNLOAD state on 404 responses in both ZFS and
WebDAV so items aren't re-checked on every subsequent sync.
Move _ObjectType, _objectTypePlural, _ObjectTypePlural, and
_ObjectsClass from per-instance properties to prototype properties,
computed once per subclass on first instantiation. This avoids a huge
number of duplicate string allocations and reduce memory usage a bit
(e.g., ~35 MB in a 19K-item test library).
setOutputFormat() just sets this.opt.mode and this.fun.decorate, so
there shouldn't be a need to cache separate engines per format. The
format is excluded from the cache key, and setOutputFormat() is called
on cache hit instead.
Also simplify _preloadFormat() in quickCopy.js to only create one
engine instead of two (one per format).
Saves ~7 MB of resident memory (~140 MB before the citeproc-js
sort-macro caching optimization).
Previously, macros referenced by sort keys were expanded inline into
the sort token array every time they were encountered. Non-sort macros
were already cached in state.macros[name] and referenced via closures,
but sort-mode macros were duplicated across sort keys.
With a complex style like chicago-shortened-notes-bibliography (155
macros, 6 macro-based sort keys), this produced 179,605 tokens and
355,652 closures in bibliography_sort alone -- 99.9% of all tokens.
This commit caches sort-mode macro expansions in state.sort_macros[name]
using the same closure-reference approach as non-sort macros.
In addition to the memory savings, avoiding the redundant macro
expansion makes engine initialization dramatically faster with complex
styles.
Results with chicago-shortened-notes-bibliography:
- Engine initialization: 464 ms -> 57 ms (8x faster)
- Per-engine heap (Node.js): 123 MB -> 6.7 MB (94.6% reduction)
- Resident per engine (Firefox): ~140 MB -> ~7 MB
- Total tokens: 187,893 -> 11,669 (93.8% reduction)
- Total closures: 370,982 -> 21,823 (94.1% reduction)
- All 1,502 citeproc-js tests pass (at f88a47e6)
- Same 1501/1505 pass on master (same 4 pre-existing failures)
The previous approach of capturing and replaying the Authorization header
from an initial request worked for Basic auth but broke with Digest auth,
where the header includes the HTTP method and URI in the hash. This caused
each replayed request to be rejected with a 401, which Firefox wouldn't
retry because the request already included Authorization.
Instead, split _channelAuthorization into _basicAuthHeader and
_digestParams. For Basic auth, the header is still replayed as-is. For
Digest, the challenge parameters (realm, nonce, qop, etc.) are cached
and a fresh Authorization header is computed per request.
_getAuthorizationHeaders() now takes method and URI parameters so it can
compute the correct Digest response hash for each request.
By adding a missing getter override. If a setter is overridden, the
corresponding getter also needs to be overridden, or it will implicitly
return undefined.
Fixes#5793
Failed when running CollectionTree and ZoteroPane tests together
- Make `deleteSelectedItems()` async and await `deleteSelection()` so
the full Notifier commit chain (including items list refresh)
completes before the function resolves. Update tests to call
`deleteSelectedItems()` directly instead of calling fire-and-forget
keyboard events, and add a stubbed test for the Delete key binding.
- Fix "Delete Item…" menu test to explicitly re-select after `saveTx()`
to guarantee selection state
`checkServer()` was capturing the Authorization header from OPTIONS and
explicitly setting it on PROPFIND via `setRequestHeader()`. If the auth
type didn't match what the server required for PROPFIND (e.g., Basic vs.
Digest, though maybe other things too), the explicit header could
prevent Firefox from negotiating the correct auth scheme on the 401
challenge.
To fix, move `onAuthorizationHeader()` from OPTIONS to PROPFIND and
don't pass captured auth to PROPFIND, letting Firefox handle the auth
transparently.
https://forums.zotero.org/discussion/129665/webdav-error-for-zotero-8-0-3
Some WebDAV servers allow unauthenticated OPTIONS requests, so the
Authorization header capturing added in 089701eca8 wouldn't work.
PROPFIND with Depth: 0 reliably requires authentication while only
returning properties of the directory itself.
https://forums.zotero.org/discussion/comment/506993/#Comment_506993
- on all platforms, Space will open the menulist popup
- on Windows, ArrowUp/ArrowDown will change selection without triggering command event.
This is needed to avoid alerts that are meant to fire after a confirmed
selection when the user is just trying to navigate the menu (e.g. item type menu)
- on Windows, fix odd appearance of checked menulist options in advanced search
condition dropdowns when "More" option is selected
- on Linux, fix menulist focus ring not appearing in Advanced Search
or Preferences (https://forums.zotero.org/discussion/129498/advanced-search-drop-down-menus-are-not-highlighted-during-keyboard-navigation)
Fixes: #5772
Firefox no longer seems to send a previously used Authorization header
with subsequent requests. This results in extra requests, since every
WebDAV request triggers a 401, and also results in errors, because a PUT
is sent without Authorization, causing some WebDAV servers to
immediately send a 401 and close the connection, which the HTTP layer
interprets as a connection failure (status 0). (It's also not good to
try to send a large file just to get a 401.)
There might be some way to share context between requests, but instead,
just get the used Authorization header and include that explicitly in
future requests.
To test this properly, we have to switch to using httpd.js for all
WebDAV requests, since the mocked XHR doesn't trigger the 401 retry.
https://forums.zotero.org/discussion/129194/webdav-uploads-fail-on-zotero-8-put-sent-without-authorization-server-closes-connection
Any modification to an item belonging to collections in the trash would
cause those collections to be removed from the item when the item was
uploaded.
Fixes#5766
* citation dlg: local. CSL locator in guidance panel
Use localized short CSL page locator in guidance panel
text example ("p10")
Fixes: #5745
Also, fix dialog mode switch buttons not properly sizing
in locales where list/library string is longer.
* Remove `overflow: visible` in RTF Scan to prevent richlistbox from expanding the width and causing clipping
* Introduce small margins as an alternative to prevent focus rings from being clipped in the `wizard`
* Fix "Display as" alignment on Windows
- after a new bubble is added to the citation, it is recorded
as a just-added bubble. The next locator typed without a search query
will go to that item instead of going to the item before where the locator was typed.
Same logic applies when multiple bubbles are added at once.
- the record of just-added bubbles is cleared on focusout
or keypress of an arrow key. That way, it's discarded if the user
is almost certainly not intending to immediately type a locator.
- added a special case to recognize a numeric value as a page locator
if it is typed when just-added bubble is recorded. That special locator
will be added to the just-added bubble as one is typing without
pressing Enter after debounce. Enter will immediately add the locator
without waiting for debounce.
- if a just-added bubble is recorded, cmd-z will clear
whatever numeric locator may have been typed and place
it back into the input, in case one meant to type an
actual search query
- added a special case to recognize ":<number>" as a page locator
in the same circumstances that "page <number>" is currently recognized
- do not use year extraction (SearchHandler._cleanYear)
when parsing input. It strips the first number from
a range of numbers and conflicts with the new locator
logic.
- ensure a bubble with a very long locator does not overflow
Instead of adding listeners to the window icon unless
no-titlebar-icon is set on the window, add listeners
to the icon whenever it exists. That way, one doesn't have
to remember to add the no-titlebar-icon attribute to
every window that doesn't have an icon.
Fixes: #5733
- restore guidance panel with instructions appearing on the first run
- clear firstRunGuidanceShown.quickFormat pref
- set slightly higher arrow padding on win/linux for the guidance
panel. If the arrow is too close to the edge, it appears
partly disconnected from the panel.
Fixes: #5738
* makeFormattedBibliographyOrCitationList(): Don't modify CSL.Engine state
Use previewCitationCluster(), which leaves the processor state alone,
instead of calling appendCitationCluster(), which has side effects and
isn't used anywhere else in Zotero.
* Make CSL.Engine caching opt-in
* Remove unnecessary CSL.Engine reinitialization
* Code style
* Add/update tests
After citation-style-language/styles#7928 renamed Vancouver styles to
NLM terminology, Zotero installations end up with both vancouver.csl
and nlm-citation-sequence.csl. To fix, on init, delete any installed style
whose ID appears in the renamed-styles mapping if the target style
also exists.
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Load tags and annotations before itemData, since `_loadItemData()`
calls `updateDisplayTitle()`, which needs creators, tags, and
annotations to be loaded for some item types.
This should fix slow startup times in Zotero 8 in libraries with many
(e.g., tens of thousands) of annotations.
The slowdown here was introduced by #3416 loading annotations in the
`catch`, but it was already incorrect for tags to be loaded that way at
startup. Creators were loaded before itemData, but tags weren't.
The `catch` might not be necessary at all now that the loading order for
`loadAllData()` is fixed, but we'll need to test further to confirm
that.
Fixes#5724
We don't need to try to load annotations for regular items or creators
or tags for annotations.
Every annotation item's annotation is still loaded independently, but we
can try to fix that separately. For now, just don't try to load data
that will never exist.
Addresses #5724
Disable wrap_url_and_doi for citeproc instances used in word processor
integrations to prevent brackets and other special characters in DOIs
from being double-encoded.
Fixes#5557
fix: #5683fix: #5684fix: #5688
Rename tab hook focus -> focusFirst
Replace tab-focus custom event with tab hook refocus
Move context pane _handleTabReady to Zotero_Tabs
Focus note editor after opening tab
Use setTimeout to ensure the focus does not stay in tab element
This avoids redundant field migration on other devices if it has already
run on one. Conflict resolution should automatically ignore identical
changes, so it's not the end of the world if it runs in multiple places,
but we might as well avoid it if we can.
Creators shouldn't be extracted if creators of the same type already
exist on the item, to follow citeproc-js behavior, but the code to do
that was broken.
File renaming has a Notifier handler for item modifications that needs
best-attachment titles, but attachments and their item data might not be
loaded even when parent items are. (This came up when migrating fields
from Extra after a global schema update before the main window had
loaded.)
- Replace polling with a `ready` promise for initialization
- Use fixed height for the rich list and re-order initialization to avoid layout shifting
- Ensure the "accept" button is disabled until initialization completes
Allow one to switch between "Add/Edit Citation" and "Add Note" modes
in the citation dialog via segmented control in the bottom
left corner.
One cannot switch modes when editing an existing citation or
when inserting a citation in the note editor.
Use the same segmented control UI to switch between
list and library modes for clarity. Replace icons with
"List" and "Library" strings.
Collect some of the state variables into DIALOG_STATE object
for better organization.
Fixes: #5582
Also, remove earlier logic of not resizing window on refocus
on Linux if focus was lost <100ms ago. That workaround
no longer works, and now that we retain the window sizing
after search reruns, it is less of an issue.
Addresses: #5590
When fetching open tabs for the citation dialog, fall back
to the last saved session state if the main window is closed.
Session state was already being updated when zotero pane
is closed.
Fixes: #5661
When a Zotero plugin is disabled, user-modified preferences can be inadvertently reset to their default values. This occurs under specific conditions where a parent preference has not been user-modified, but a child preference within that branch *has* been modified.
The root cause lies in the `Zotero.Plugins.clearDefaultPrefs` function. For preferences that do not have a user-set value (`!branch.prefHasUserValue(pref)`), it incorrectly calls `Services.prefs.getDefaultBranch('').deleteBranch(pref)`. While intended to clear unmodified defaults, `deleteBranch(pref)` operates on the *entire preference branch* starting with `pref`, inadvertently removing any user-modified sub-preferences as well.
This commit changes the problematic line from `branch.deleteBranch(pref)` to `branch.clearUserPref(pref)`. `clearUserPref(pref)` correctly removes only the user-set value for the *specific* preference `pref`, leaving default values and any user-modified sub-preferences intact. This ensures that only truly unmodified default preferences are cleared, preserving user data for related sub-preferences.
This fix prevents unintended loss of user settings when plugins are disabled.
After 015769a removed pointer-events: none from table cells,
clicking on the collectionTree would always trigger
a focusout event, ZoteroPane.handleBlur would call
collectionTree.setHighlightedRows, which would always
redraw the collectionTree. This complete tree redraw on
every click made it impossible to register double-clicks.
With this change, setHighlightedRows won't have
any effect if called with the same rows to highlight
as before (including no rows).
Without constant tree redraws, double-clicks fire as expected.
Fixes: #5655
Fix new collection dialog appearing broken when
opened via "New Collection" option of context menu
on "My Library" or a group in collection tree.
On macOS, the popup of collections would never leave
and on windows, subcollection would never appear.
This is a followup to zotero#5409
that fixes this issue for all collections. The reason
why it didn't work for groups is that the command event
would fire not on a menuitem but on the <command> node itself,
which is not what the workaround expects.
Now, if we get such an event, we'll try to use the
original 'command' event dispatched on the <menuitem>
from event.sourceEvent to locate the <menupopup>
and as a blueprint for redispatching the event.
Fix createParentDialog hanging when opened during
quicksearch or in saved search. After a parent item
is added, itemTree is being refreshed. If _refreshPromise
is resolved after a timeout, it will not complete until the
modal dialog is closed. Instead, resolve _refreshPromise
after Zotero.Promise.delay, which uses the XPCOM global
timer unaffected by modals.
Fixes: #3026
Another potential fix to the test failure. Earlier fix
from 30784dd241 seems
to not have worked.
A new explanation is that the test before it does not properly
wait for the trash to refresh before trying to select the library, in
which case collectionTree select event will be suppressed
and library selection will not happen.
Fixes: #5584
This test would sometimes fail, most likely due to
the library sometimes not getting re-selected in the previous test
'should update custom header for items in the trash'.
A likely explanation is that the selection event in
collectionTree would still be suppressed when selectLibrary
is called, so make sure to wait for item deletion to
go through before trying to re-select the library.
Fixes: #5584
If the tag selector is not initialized in setScope,
initialize it before proceeding. This fixes
the tag selector being stuck in loading state
after expanding.
Fixes: #5573
- Remember diffs from test runs, show in sidebar when test is selected
- Add button to immediately update test, without a prompt
- Indicate when test has a custom defer delay set (although no tests currently do)
- Use a persistent test store as source of truth, instead of data attributes on the listbox rows
- A bit of code cleanup
Closes#5420, closes#5419
QuickCopy.init() calls Style#getCiteProc(), and a side effect of
citeproc-js locale initialization could cause a recursive call back to
QuickCopy.init(), which would again call Style#getCiteProc() (after it
had checked its cache but before it had added the new engine to it),
leading to duplicate CSL.Engine initialization. Now we correctly check
whether we've already been initialized, eliminating the race condition.
Separately, remove async keyword from init(). It never actually was
async - even before Bluebird removal, it didn't await anything or return
a promise. And we can't make it await its pre-caching work because
schemaUpdatePromise won't resolve until init() resolves/returns.
* Fixed incorrect new name generation for attachments without a file extension and not present on disk
* Fixed invalid preview in certain edge cases
* Added tests covering file renaming in general and the above issues
When an embedded attachment note is converted to
a standalone note, relate the standalone note to
the attachment and place it in the same collections
as the attachment.
When drag-dropping items into a collection in another library,
perform the addition to collection in the same transaction
as creating a new item in the target library.
When the librariesCollectionsBox refreshes on the `modify` event
when a newly created group item is linked to the selected item,
it re-loads the data of the linked item via item.loadAllData().
This could happen after the item is added to the collection
but before this change is saved. In that case, item._changed.collections
would be cleared, and when the item is saved, there would be
no changes to collections to save.
Fixes: #5539
Also, cleanup leftover unused logic of restoring linked item
from trash on drop that was removed in 2dd16b44d6
When a tab is closed via clicks on "X" button, record
and lock all tabs' widths. That way, the "X" button
of the tab that follows the one that was just deleted
will land under the mouse cursor, allowing one to
delete many tabs. In addition, preserve the scroll position
of the tab bar. If it's impossible because too many tabs
were deleted, add a spacer after the last tab.
Fixes: #5523
Post-fx140, menulist labels in chrome windows are not
announced by VoiceOver. Setting the menulist's role to 'combobox'
addresses this and gets VoiceOver to announce the labels again.
This is interesting because 'combobox' is supposedly
menulist's implicit role, so it's not clear why this helps.
It does slightly change the way VoiceOver announces
menulists: it announces "combobox" instead of "popup button",
but semantically combobox is still appropriate per
https://www.w3.org/WAI/ARIA/apg/patterns/combobox/examples/combobox-select-onlyFixes: #5424
- Remove the delete button below the style table
- Make selected rows stand out less. They will only have an outline,
like focused rows. This should remove the impression that this is the
table where one selects the style for citations.
- Add minus icon buttons to each row to delete the style without having
to select the row. The icon of the selected row is focusable, so it
can be tabbed onto.
- improve labels and appearance of buttons to add a new style
- move "Reset Styles…" to "Cite" section and rename to "Restore Default
Styles…"
Revert change from 8e2790e2d2.
Post fx140, dragstart fires fine on rows that are not
yet selected. This allows us to remove the workaround that
limited pointer events to the rows of virtualized table (vs its cells).
With it, we no longer need most of the manual handling
of the tooltip in virtualized table - just adding title attribute is enough.
We also don't need to add extra mouse events handling on
clickable button columns as in citationDialog - we can use
proper :hover and :activate effects, as well as attach
click handler directly to the button.
On non-macOS platforms, Delete (not Backspace) on a table row
will remove all of this item's bubbles from the citation.
This is to be consistent with similar delete operation
handling in other tables.
On macOS, this is still done via Backspace as before.
Per https://github.com/zotero/zotero/pull/5518#discussion_r2310293170
We were skipping updating of `clientDateModified` in order to prevent an
attachment item upload, but that hasn't been used since Zotero 4. This
might mean that, in some rare situations (e.g., a filename that needed to
be shortened due to filesystem path length?), people in read-only or
non-files-editable groups may have ended up triggering access errors and
needing to reset local data after a file download.
Now, we properly skip marking the attachment item as unsynced.
We've always done this, but it's no longer necessary:
- If a file exists locally when a remote filename change comes in, we
now rename the local file.
- If a file doesn't exist locally and it's a single file, we save the
file with the new filename on the next file download.
- If a file doesn't exist locally and it's an old multi-file snapshot,
we now try to find the primary HTML file and rename it.
If a file was renamed remotely and a new copy wasn't uploaded for some
reason, the ZIP wouldn't contain the new filename. We already renamed a
single file within the ZIP to match the new filename, but now we also
rename a single HTML file in old multi-file snapshot ZIPs. If there are
multiple HTML files for some reason (old-style ZIP with iframes?), we
let the user fix it.
And then we can stop reuploading files after renames.
Previously, the local file wasn't renamed, so it would become unlinked.
Since we currently force reuploading/reregistering of files when they're
renamed, opening the attachment would then redownload the modified
remote file, but there's no need for the file to become unlinked in the
first place.
restoreProcessorState() breaks locale-specific punctuation (and is
deprecated).
rebuildProcessorState() is enough if we're passing an empty citation
list; if we're reinitializing with a new non-empty list, call
updateItems() first.
Also:
- Add tests for this issue and for potential regressions in disambiguation handling
When resizing the window in list mode to match the height
of the content, account for the difference between
outerHeight and innerHeight of the window on Linux.
outerHeight > innerHeight (perhaps Linux includes chrome,
borders, etc. in its outerHeight), so we have to add their difference
to the desired height for resizing. Otherwise, the visible
dialog ends up receiving only some of the desired height and
the last row is cut off.
Also, ensure that in tests, we wait for searching triggered by
switching dialog mode to finish before running the next search.
Fixes: zotero#5502
By using getSelectedItems() instead of getSelectedObjects(). The code
below all assumes that `items` contains items, and we handle general
(item or collection) options above, so this seems more correct.
And fix duplicate var.
Fix ghost relations to erased items reappearing after
sync. The dataserver ensures that related relations are
bidirectional. If a related item is erased, syncing
would first upload the update to the remaining related item
(where the relation is removed), followed by the deletions.
While handling update upload, the dataserver would return
the old, should-be-removed, "dc:relation" entry for that item
because it does not know that the other item is erased.
To handle this, swap the order of operations during sync:
upload deletion first and then upload updates after.
Then, when the dataserver handles the update of the non-erased item,
there is no related item to enforce bidirectional relations to.
Fixes: #5481
Fixes tabbing out of Scaffold's abstract box causing the test item to be
added to the user's library.
And use same format as itemPaneHeader.js in abstractBox.js.
Hopefully a better fix for #4981, which wasn't working properly on at
least some Linux systems because the variables were getting restored
before the subprocess launched. This delays a (debounced) second, to
give the subprocess time to start, and then automatically restores the
variables.
An explicit restart via app code also immediately restores the
variables, since restarts on Linux inherent the environment and don't
use our startup script. (An upgrade restart could still happen when the
variables were cleared, but you'd have to be extremely unlucky --
launching URLs or files while also performing a manual restart the same
second.)
I've run into this randomly. It only occurs if the first operation on a
Google Doc since Zotero restart is the edit bibliography dialog which
you cancel. Due to how the HTTP integration client is implemented, the
missing await causes it to try to send another response to the Connector
(which is no longer waiting on the /response endpoint), failing and
causing a forever pending promise, which makes subsequent attempts to
interact with the Google Docs plugin no-op.
We get occassional reports from users about Google Docs getting stuck
that is fixed by Zotero restart, so hopefully this will reduce those.
Resize the window to fit content on load after the
header is rendered. If the window is resized on
DOMContentLoaded, nothing is rendered yet and the
window will look cutoff on Linux.
Addresses #5453
- close hyphenationTest window.
- properly close zoteroPane window in preferences_syncTest.
after() hook inside of the test would not close the window until the
very end of the entire test run. Without this, during
subsequent tests that open their own zoteroPane window, there
are two instances of zoteroPane open at the same time.
- close open tabs in readerTest cleanup, since subsequent
tests expect library tab to be selected.
Followup to https://github.com/zotero/zotero/pull/5444#issuecomment-3141879892
- value() is used in noWait export translation
- isResolved() technically means fulfilled or rejected -- unclear if we
ever depended on that, but it might as well be correct
Previously, we simply cleared the queue, leaving any pending promises
unresolved, and we checked for isResolved()/isPending() in
storageEngine.js (and maybe other places). storageEngine.js was changed
to simply await those promises, but that results in a hang if the queue
is stopped, since the promises were never resolved. Instead, when
stopping the queue, reject queued promises with a CanceledException and
ignore those promises in storageEngine.js.
As the comment says, this hasn't been useful (and isn't used anywhere),
and I honestly don't understand how the test ever passed - process()
would've resolved all the feed item promises already.
Fixes#5412
A test using `assert.eventually` was failing after the Bluebird removal
but worked with just `await`, and since there hasn't really been much
point to Chai as Promised since the introduction of `async`/`await` ages
ago, just remove the library instead of figuring out why.
OK, the timeout removed in 05e8da wasn't actually useless, but this test
was.
41706a changed erase() so it no longer forwards skipNotifier when
deleting child items. That means that we *would* get a notify() call for
the deleted child, so the test should have been failing.
However! Because of a bug in Bluebird's filtered catch() implementation,
Promise.TimeoutError wasn't recognized as an Error subclass, and it was
instead used as a predicate function, which would always pass.
So all errors were caught as "timeouts" and suppressed, making this test
always pass. Now it no longer always passes (but does pass with the
current behavior).
This is really messy and not ideal, but these tests were written for
Bluebird Promise state inspection, and it's difficult to make them look
nice without that being available.
Restore Zotero.Promise.method() wrapper on getCodeForTranslator()
because the translation framework needs it to return a synchronously
resolved promise, unfortunately.
Necessary for non-Mozilla C:\ProgramData folder now that we can no
longer do a string replace on the official DLL
The custom component ZIPs are currently stored in S3 and referenced by
SHA-256 hashes, which get committed. This is basically the same as Git
LFS, but since we now support per-arch fetch_xulrunner and building,
this avoids the need to download large xul.dll files for other
architectures. (Git LFS files also can't ever be removed, so
particularly while we're testing this, we don't necessarily want to
permanently store every copy.)
`fetch_xulrunner`, `dir_build`, and `build.sh` now take an optional `-a`
argument on Windows and Linux to specify the architecture to
fetch/build. If `-a` is omitted for `dir_build` (as in `build_and_run`),
it defaults to the current architecture.
xulrunner hashes are now computed for each architecture. This changes
some filenames in xulrunner/, so you may want to clean out some old
folders/files.
This removes the custom handling for CI, since it will now default to
the architecture of the test runner.
nsIURI#pathQueryRef no longer includes the authority component. A URI
that would've had pathQueryRef '//authority/path' before will now have
pathQueryRef '/path'.
Reader seems to be functional after this change.
All module code runs in strict mode, and that breaks Bluebird.
And add a partial shim for the simplest and most useful methods. Not
sure if we actually want that, or if we should just replace it entirely
with native methods.
Opening a dialog window via a command from a menupopup leads to
unexpected glitches in the dialog's popups (e.g., collections menu
in newCollection dialog on macOS won't go away when you click away,
and on Windows, child collections don't appear when a parent is hovered).
This is due to some kind of race condition that arises when the dialog
appears when the popup is still closing.
This workaround captures command events from menupopups, closes the popup,
stops propagation of those events, and then dispatches a copy of
that event after a delay. This gives the popup enough time to close.
Fixes: #5408Fixes: #4891
This was a regression from the switch to `Zotero.HTTP.download()`. 302
wasn't a success code, so `HTTP.download()` would throw, and since the
status wasn't set correctly on the `XMLHttpRequest` within
`HTTP.UnexpectedStatusException`, it would think it was an interrupted
S3 connection and trigger another download after a delay.
* Add missing icon for "My Library" in the "Copy To" menu in cases where there are no other libraries in that menu
* Add icon to the "New Collection..." option in the item context menu
If the id of the collection happens to be the same
as an id of an existing item, those objects may end
up being used interchangeably in itemTree.
The trashed collection could appear in "My Library"
instead of the item, or the not-deleted item could appear
in the trash instead of the collection.
This is addressed by using .treeViewID instead of the
.id of the objects in ItemTree.refresh and ZoteroPane.itemSelected.
* Fixed an issue where the window size did not match the content
* Replaced the semi-transparent progress bar with a spinner
* Fixed a problem that allowed triggering a search while one was already in progress
* Reduced code complexity
This allows the locator dropdown to be as wide as it wants
(depending on the longest option in the current locale),
with inputs occupying the remaining space. This is
a more flexible layout that avoids broken UI in
cases if <select> dropdown tries to become too wide
while <input>s occupy a fixed space to remain aligned.
Fixes: #5350
Fixes missing tag swatches borders, a regression due to a change of the
default values for the `focus-states` mixin in #4961 (61b62d0).
Also introduces opaque tag colors so that tag swatches are not
semi-transparent.
Together with all collections and libraries.
To account for tags that may contain "," characters,
expect connector to send tags as an array. If tags
are sent as a string, it will be split by "," for
compatibility with older connector versions.
Addresses: #1706
* support adding a note from connector
- pass canUserAddNote flag in ping response to tell
the connector that it should display a note field
- when the connector sends a note, create a child note item
if it does not yet exist and set its content. If the
child item for user-added notes is already created,
update it
- if the connector sends an empty note, user-added child
note item is deleted
- record child notes added by the user in _userAddedNotes
cache to distinguish between user added notes and
notes added by the connector automatically
- if the target library changes, child note items are
moved to the new library as well. _userAddedNotes
will be updated to reflect new itemIDs
- if there are multiple toplevel items being added,
the note is added to all of them
Also fix breakage when switching between collections
from different libraries. Fixes: #5331
The predicate name used as an object key was being converted to an id,
so if relations weren't reloaded properly, they would be broken after a
save.
Discovered while working on #5297
Extend scrollToRow in windowed-list.js to allow to
scroll the specified row to the top of the window
even if it is located below the current scroll window.
With this, when scroll position is restored in itemTree,
the same row remains at the top.
Fixes: #5233
* Show "Remove All Tags..." when right-clicking textbox
* Fix pasting multiple tags adding concatenated tag
* Remove unnecessary check
* Fix tag count changing when unsaved tag is removed
On Windows, after right-clicking on a title field.
this.querySelector('popupset') matched the popupset in the
collapsible-section head, not our own.
Fixes#5294
* citation dialog: interactive before io.getFields
- added io.allCitedDataLoadedPromise, which is resolved
when both io.fieldIndexPromise and io.citationsByItemIDPromise
are resolved. Resolved io.allCitedDataLoadedPromise essentially
means that calls to io.sort() and io.getItems()
will be fast because all necessary data is already loaded.
- citation dialog uses io.allCitedDataLoadedPromise to
not await for functions relying on io.sort() and io.getItems()
before the data is loaded, as it could take an arbitrary
amoung of time. Speicifcally, SearchHandler._getCitedItems() and
CitationDataManager.sort. As soon as allCitedDataLoadedPromise
is resolved, cited items will be sorted.
This means that when retrieving fields takes a long time, one can
still add new items, their bubbles will just not immediately
be sorted.
- this replaces earlier SearchHandler.loadCitedItemsPromise, which
was a special case of this handling.
- added a new test ensuring that bubbles can be added even
when io.allCitedDataLoadedPromise is not resolved yet
- cleanup for buildCitation function to remove handling
of io.citation.sortedItems, which is always empty on
load before io.sort() runs
- added a few Zotero.debug statements for future debugging
- Uses a dummy promise if no allCitedDataLoadedPromise
(e.g. to accomodate the note editor)
- replace objects with zoteroItem, citationItem, etc.
as the main representation of an entry in the citation
(CitationDataManager.items) with instances of a new BubbleItem class.
BubbleItem has two static methods to create
a new instance: BubbleItem.fromCitationItem() used
to create entries from io.citation.citationItem on load,
and BubbleItem.fromItem used to create entries from
Zotero.Item when a new item is being added into the citation.
BubbleItem.getCitationItem will return a citationItem
in a format that citeproc consumes. This streamlines
the flow of data, makes it easier to manage and avoids
unnecessary conversions between citationItem and Zotero.Item.
- This refactoring fixes the issue of added suffix/prefix
sticking to an item and persisting even if the bubble
is removed and re-added. Fixes: #5282
- add csl data to virtual item in io.customGetItem
to remain consistent with Zotero.Integration.Citation.loadItemData.
That way, csl data is always found on Zotero.Item
and we don't have to check citationItems.
Also:
- apply deduplication to cited items, so that a cited
item from the library that appears in selected or
opened section does not have a duplicate entry in
the cited section.
- added initial test suite for citation dialog
Use `uninstall: true` in `update_url` JSON `updates` that is valid
for the installed plugin to uninstall the plugin the next time it
checks for updates
This actually checks for a "primary" column, so if an extension hides the "title" column but provides its own "primary" column, the "title" column remains hidden.
Unhandled Enter keypress from inside of a panel will now close it.
Enter on "Omit Author" checkbox or on the inactive locator
label dropdown will close the item details popup.
Enter on "Keep sources sorted" checkbox of settings
popup will close it for consistency.
Fixes: #5277
- right-click on annotation rows in trash will display
"Delete Permanently" option, instead of an empty menu
- in trash and other non-editable collection rows, disable
"Add Note from Annotations" button in itemPane header
- remove ability to add tags to annotations and other
items by dragging them into tag selector from trash
- remove ability to add items to collections by
dragging them into collection tree from trash
- fix attachment row being removed from trash if
its child annotation row is erased
Also:
- fix contextPane in reader tab being editable for item in trash
- fix title being editable in attachmentBox for item in trash
Fixes: #5261
Do not update itemTree selection on each annotation row
removal when an attachment row is collapsed.
It causes a significant delay when attachment with
many annotations is collapsed if selection is on any
row below the attachment. It is also redundant because
the selection is updated at the end of _closeContainer anyway.
Fixes: #5255
- collapse all rows before removing them. It makes
sure that annotation rows will be cleaned up properly
when an attachment row is moved into another parent.
- refactor conditionals to handle changes of parent
regardless of whether the item is a container
(attachment with annotations) or not (a note).
- added tests for itemTree's handling of changing the
parent of attachments and notes, as well as ensuring
there are no leftover annotation rows in Unfiled Items.
Fixes: zotero#5246
- Finalize renaming of pdf-reader to reader
- Remove client- prefixes from URL paths
- Update pdf-worker to use the new document-worker path in preparation for repository rename #5052
- Ensure note-editor properly uses the ZIP build instead of silently falling back to a local rebuild
- Log errors to console
- disable context menu on collapsible-sections of annotations in `itemPane`
- fix `arrowRight`/`arrowLeft` being able to expand/collapse annotation sections when they are focused
- set proper aria-label and aria-description on annotation cards
- fix misaligned annotation icons
Fixes: #5231Fixes: #5234Fixes: #5240
- Annotations are displayed in itemTree under their file attachments
on a third level. The annotation spans the entire row.
- The title is constructed on the go. When possible, it
includes annotation quote and comment as pseudo-columns
of the row. The comment occupies about twice as much space as
the quote. Otherwise, (if there is no quote) only the annotation
comment
is included as the "title" part of the row
- Non-CJK segments of the quote part of the annotation row are
italicized. CJK segments are left as is. If CJK segments are present,
there is more padding between quote and comment parts.
- Search matches the actual attachment instead of its parent file.
- Can create child notes from annotations of the same item or
standalone notes from annotations across different items from
the context menu or the header button.
- When an annotation (or multiple annotations) are selected, the
annotationItemPane component is displayed where annotations are
grouped by their top-level item. Annotations are displayed fully,
without having their content cut off.
- Special treatment for annotations to always prompt
to erase the item regardless of what collectionTree row
is selected (e.g., if a collection is selected, we
still want one to be able to delete the annotation).
This only applies if all selected items are annotations.
If multiple items are selected, some annotations and
some not, do nothing. This is until the trash is
ready. In the future, we may send annotations to trash
- strip all HTML tags from annotation for now, until the logic to
properly render annotation markup is copied over from the reader
(applies to both annotation-row component and the annotation
item rendered in the itemTree)
- Added a generalized "expandToItem" function to itemTree to
expand all ancestors of a given item, similar to "expandToCollection"
from collectionTree
- add annotation conditions to advanced search
- show [Image not available] if no annotation file for ink or image
annotations
- only keep annotation-specific context menu options when some
annotations are selected in itemTree
- enable Quick Copy of annotations from itemTree via drag-drop,
shortcut key, or Edit → Copy Annotation
- Minor refactoring of Zotero.Annotation.toJSON() to pull out async code
that handles ink and image annotations, so that
Zotero.Annotation.toJSONsync() for highlight, underline, and note
annotations does not have to be awaited. Since ink and image
annotation don't seem to work for drag-drop Quick Copy, they are just
skipped for now.
Remove redundant and wrong call to changeCollectionTreeRow() in
advancedSearch which triggers test errors sometimes, and disable prompt
button delay when running tests everywhere, not just CI
- when something is dragged over a collapsed collection
or group, it will be expanded to reveal children
so one could drop the dragged item/collection/ into them
- when drag is over, the expanded collections will
collapse, except for the ancestors of the newly selected
collection
- trigger collapse of expanded collections on dragend
on the entire document (in case dragend happens not over
the collection tree), or from onDrop handler, from where
we will wait for selection to change before collapsing
rows
Fixes: #4952
This is caused by the persisted splitter state value to be blank from either older versions of client or plugins.
Resetting the empty values fixes this bug.
Z7 removed an `OS.File.open()` in `_checkForUpdatedFiles()` that would
throw on missing files and cause them to be marked for download in a
`catch`.
This likely caused the fix for #1753 not to work in Z7.
Do not switch to library tab while changing target
collection in connector.
Deprecate inLibraryRoot as an independent boolean
param in ZoteroPane.selectItems to instead have
a composite options param { inLibraryRoot, noTabSwitch }
Fixes: #4262
* citation dlg: cleaner search value
Additional cleanup of the search value to not include
some punctuation ("(),.;:"), localized "and",
ampersand, and localized "et al". That way, if one
copies a citation like "(Smith et al., 2020)" and pastes it into the
citation dialog, it is more likely that appropriate search
results will appear.
Fixes: #1864
* handle citations with locators
Clean the input before fetching the locator so that
if one copies and pastes (Smith et al., 2025, p. 15),
the locator is still correctly extracted.
This fixes the breakage in tabs menu on initial load
if one of the tab is for an item from not-loaded group.
Now, every component relying on tab items (e.g. tabs menu)
can just assume that items are all loaded.
Fixes: #5192
On initial load, the grippy does not immediately
appear because it needs to be loaded. This means that
when itemBox is rendered for the first time, the grippy
has a width of 0, which later becomes 18px. If the
itemPane is narrow, it may push the columns out
and lead to text of value components wrapping
without the rows getting stretched.
This sets explicit width to the grippy to avoid such behavior.
Fixes: zotero#5166
- removed quickFormat, insertNoteDialog, and addCitationDialog
.xhtml and .js files, since all of them are now replaced
with new citationDialog
- removed quickFormat and insertNote strings from
en-US localization files
- replaced all mentions of quickFormat in comments and
variables of integration.js and editorInstance.js with
citationDialog
- removed big-sur-specific styling for quickFormat
Closes#5046
Do not lowercase the entire title before capitalizing
the first letter of each word to preserve abbreviations
or spelling of unusual words (e.g. 3D or iPhone).
Fixes: #5165
Also:
- Introduce a mixin for a toolbar button that appears on hover or focus
- Unifies the border radius of such buttons, which was previously different for the "Libraries and Collections" toolbar button.
- last-closed -> last-used for citationDialogMode pref value
- pref label citationDialogLastClosedMode -> citationDialogLastUsedMode
- update Zotero.Prefs version. While migrating, if
integration.useClassicAddCitationDialog is true, set
integration.citationDialogLastUsedMode to 'library'
so the citation dialog opens in library mode on the first run.
If citationDialogLastClosedMode pref exists, it is
migrated to citationDialogLastUsedMode and cleared
- redone logic of extracting the locator to allow for better
detection of locators and to not rely on long regular
expressions that are hard to troubleshoot. Added tests for it.
These are the new rules for extracting locators:
- locator value can be either a set of numbers potentially
with some punctuation in between OR text surrounded with
single or double quotes
- locator labels can be typed in any of 3 formats:
full (chapter), short (chap.), or short with no
punctuation (chap)
- locator labels go before the locator values and there
may or may not be a space in between. For example,
chap.10 is the same as chap. 10.
Fixes: #5092Fixes: #5093
- get opened items asynchronously and load their data
in case if there are items opened across multiple
libraries, and some of them are unloaded.
This fixes#5161, which makes the dialog unresponsive
upon initial load.
- enfore io.filterLibraryIDs in library mode, the same
way it already is being applies in list mode. This
is currently used to only show items from the current
library when adding citations in a note.
- account for the change above in collectionTree to
by focusing the first library from io.filterLibraryIDs,
if those are present, and only default to userLibraryID
if io.filterLibraryIDs is empty.
Keep the placeholder node to retain the positioning
of the text consistent but remove the border
so the item card is invisible.
Also: remove spinner
Fixes: #5151
- pass ref to itemsView.changeCollectionTreeRow
to avoid errors thrown when typing numbers (related to
tags shortcuts)
- return string equivalents of true/false in getExtraField.
virtualized-table expects a string in its typing handler.
Previously returned boolean was causing an error.
Remove refocusing of the input on Enter from item details
popup - just let focus go back to the bubble.
Input refocusing was initially added with the idea that once
you make some edits and confirm them with Enter,
you are ready to go back to typing in the search input.
But then, Enter on Done or Cancel is treated as equivalent
to click, so the focus goes back to the bubble. When
the dialog is cancelled via Escape, focus lands
on the bubble as well, in case you need to keep going to
locate the another item to customize.
This different focus behavior on Enter is indeed a bit
inconsistent and likely no longer needed, now that
you can always refocus input with Cmd/Ctrl+F.
https://forums.zotero.org/discussion/122823/some-advice-for-citation-dialog-in-beta
Before making the dialog interactable.
Initialize loading cited items via io.getItems when the dialog appears.
Once they are loaded, itemsList is refreshed to include
cited matching items and all subsequent searches will include
them too. During the refresh immediately after cited items load,
try to preserve focused/selected state of items.
Added IOManager.preInit that is ran as soon as the dialog
is loaded. Move accept/cancel initialization there.
IOManager.init runs at the very end of onLoad because
it relies on most data and layouts being loaded,
It means if there is some delay or error during loading,
the dialog cannot be closed. To avoid it, setup
cancel/accept buttons separately in the beginning.
Also, minor refactoring of SearchHandler to handle
refreshing of cited items separately from selected and
open items.
Fixes: #5121
- add dropmarker to bubbles after the text to indicate
that bubbles are clickable
- remove X remove button that was there before
- no more mask over the bubble on hover, since dropmarker
is just always visible
- 4px distance between the dropmarker and bubble text
Fixes: #2712
In library mode, apply highlighting to bubbles that
correspond to selected items in itemTree only when
the items table is focused. When the focus leaves,
highlighting is removed from bubbles.
Fixes: #5138
Add "Restart in Troubleshooting Mode" checkbox, checked by default, and
remove "Restart Later" option, since it's not possible to restart later
in Troubleshooting Mode. I doubt anyone used that, and if they really
need it, they can use -ZoteroDebug at the command line or toggle the
`debug.store` pref manually.
Closes#4580
- restore earlier behavior of arrowDown on a bubble
opening itemDetails popup
- remove arrowUp closing the popup
- remove arrow navigation between rows of bubbles on arrowUp/down
- remove no longer needed edge case of arrowDown
handling on the last row of bubbles
Fixes: #5118
This resolves an issue where, after the selected theme was deleted, it was impossible to select a newly added custom theme. This happened because the preference key for the selected theme uses consecutive IDs and wasn't cleared when the theme was deleted. As a result, the newly created theme would receive exactly the same ID. When the newly created theme was selected, the preference observer did not trigger (because the ID did not change), leading to this bug.
The fix here is to clear the theme preference when it is deleted. The added benefit is that when this happens in dark mode, the dark theme is auto-picked instead of falling back to "Original".
We switched file downloads to use XHR to fix downloads failing via
authenticated proxies in fx115+, but that made them subject to our
default 30-second timeout in `Zotero.HTTP.request()`. That timeout uses
the XHR `timeout` property, which applies to the whole request, even if
data is actively being downloaded. As a result, file downloads would
fail for people downloading large files or on very slow connections.
This implements manual connect and inactivity timeouts when using
`Zotero.HTTP.download()`. Currently, these both use `options.timeout`,
or the default 30 seconds, but we could probably take separate options
and have lower defaults now that they no longer apply to the whole
request.
Fix issue where clicking accept button would not work
when the dialog itself is unfocused. Add a small delay in the
focus handler of the dialog to allow accept button clicks
to be handled before rerunning the search.
Fixes: #5110
While filtering selected/open/cited items, include
items that match EVERY word from the search string,
as opposed to items that match ANY of the searched words.
Fixes: #5099
In library mode when there are no selected/open/cited items,
Enter keypress from input will add the first row of itemTree
into the citation only if it is the only row.
Fixes: #5098
This is done on purpose. We distinguish between tooltips that are
command/feature titles and those that are action descriptions. So we
have "New Note" but "List all tabs". Apple does the same -- e.g., "Edit
Tags" and "Perform tasks with the selected items" next to each other in
Finder.
This reverts commit 011c5bd6c0.
IOManager relies on everything being loaded, so it
should be initialized last. Otherwise, any IO handler
trying to access currentLayout would throw an error.
Also, swap the order of operations in bubble-input to
set input width only after the first input is added.
Otherwise, first input will have an arbitrary width
during initial load.
https://forums.zotero.org/discussion/122554/beta-new-insertion-bar-problem-with-citations-but-not-notes
Implemented redesigned citation dialog with library
and list modes one can switch between. This dialog is
a direct replacement of quickFormat and classic citation
dialogs.
In Library Mode:
- items table has a new + button column to add items from the
citation and rows of selected items are highlighted
- open, cited and selected items appear in a section
between the items table and bubbleInput. When there are no
matches, a message is shown.
- only top-level items are shown when citing
items and only notes/notes' parents - when adding a note
- selected items are gathered in a collapsible deck to save space.
Click on the deck will expand it. All selected items can be
added via "Add all" button.
- when an item is added, bubble-input may increase in height and
push itemTree lower. To try to preserve relative positioning
of the mouse, itemTree will scroll to be over the row that was just clicked
In List mode:
- arrow up/down from the input will change the selected item
with the focus remaining in the input
- selected items are a collapsible list section
Other behaviors and fixes:
- one can add any locator (not just pages) by typing its full or short name
in an input and pressing Enter (e.g. line 10, or l. 10, or chap. "test chapter")
- Added a new preference to select if the citation dialog
should always open in list mode, library mode or in the last
mode that was used
- bubbles whose items are selected in library or list mode
are highlighted
- arrow up/down from a bubble will focus the bubble
above/below it for easier navigation across bubble-input
- multi-select is supported on items in list mode or item cards in
library mode via Shift-arrow or Cmd+click to select multiple items
- Cmd/Ctrl + Enter will always accept the dialog no matter what
is focused
- when there are no bubbles, accept button is disabled
- after an item is added, bubble-input is always refocused
- added suppressed property to itemTreeMenubar to be able to
hide if in list mode, where it is not relevant. Fixed menubar
getting stuck or re-appearing on Alt keypress on Linux by setting
height: 0 vs hiding it via hidden.
- added initialFolder and onActivate prop to collectionTree to set which collection
should be reopened when dialog opens and to be able to set a custom
onActivate handler.
- added getExtraField prop to itemTree to get data on if
an item is in a citation or not
Implementation details:
- citationDialog.js is the main file. It relies on a number of
helper files in citationDialog/* directory to keep the main
file less cluttered. popupHandler.js contains the logic of
opening/closing the item details popup to add locator/prefix/suffix/etc.
keyboardHandler.js is responsible for overall keyboard navigation
throughout the dialog. searchHandler.js contains the logic
for running the search based on user's query. Finally, Helpers.js
has general helper functions that don't handle any actual logic.
- SearchHandler is set to run search in two ways: for cited/selected/open
items (which are cached and do not involve any actual SQL query) and
general search for items across all libraries. When layout.search
runs, firstly cited/selected/open items are updated after which the
SQL search runs.
- bubbleInput.js is a customElement responsible for bubbles interface.
bubbleInput.refresh takes a list of items and handles adding/removing/reordering
of bubbles as needed. It always has two inputs on each side of
a bubble, as opposed to having inputs inserted dynamically. It
allows keyboardHandler.js to handle navigation with arrows within
bubbleInput.
- when user interacts with bubbleInput, it emits custom events that are
handled by IOManager singleton in citationDialog.js, which
may update the items information and pass them back to bubbleInput.refresh
to have the list of bubbles updated
- CitationDataManager singleton is responsible for storing
items added into the citation in CitationDataManager.items
in an object with both Zotero.Item and the actual citation item.
It is easier to pass both items to other components
and helpers, as opposed to sharing functions to convert
items back and forth.
No idea why this was ever here, but I don't think it's actually used
(and if it is, it shouldn't be). The only options I see are handled
directly within `executeTransaction()` itself.
When an item is erased (removed from the trash or cleaned up from a
feed), we set a flag to purge values in `itemDataValues` on the next
startup, with this query:
DELETE FROM itemDataValues WHERE valueID NOT IN (SELECT valueID FROM itemData);
For some people, that query was incredibly slow and would result in
Zotero intermittently hanging on "Loading items…" for a long time at
startup. It's possible this is mostly limited to people who subscribe to
high-volume feeds and have a lot of item churn. One affected person had
>900K values in `itemDataValues` despite having only 20K items.
It turns out the slow query is due to the foreign-key constraint on
`itemData(valueID)` that references`itemDataValues(valueID)`. SQLite is
checking every row being deleted from `itemDataValues` against
`itemData`, even though the query is specifically removing rows that
don't exist in `itemData`! For the 900K-value DB, disabling foreign-key
checks causes the `DELETE` query to take 25 seconds instead of...some
much longer time that I didn't wait for.
We already had an `executeTransaction()` flag, `disableForeignKeys`, to
temporarily disable foreign-key checks, but it didn't do so in a way
that was safe for post-initialization usage -- a write query outside of
a transaction could've run between the transaction commit and
foreign-key checks being re-enabled. This commit changes it to properly
block all other queries unless they include an `ignoreDBLock` option,
meaning that queries within the function passed to the transaction need
to include that option. (And since that's not realistic for the couple
other uses of `disableForeignKeys` -- one for a test and one in code
that almost certainly hasn't been run by anyone in 15 years -- those now
just run `PRAGMA foreign_keys=OFF|ON` explicitly, leaving this as the
only current use.)
Adds a new function, Zotero.HTTP.download(), that uses
Zotero.HTTP.request(). This fixes downloads via authenticated proxies in
Zotero 7 and gives us other request() functionality (e.g., 5xx retrying)
for free.
The downside is that this is probably less efficient, potentially
loading large downloads in memory. We should create a replacement for
request() based on fetch() that supports getting the body as a
ReadableStream.
Fixes#5062
A browser window was no longer actually needed for charset detection
(on macOS, at least, and hopefully elsewhere) because we switched to a
HiddenFrame-based hidden browser. Remaining uses now call
`loadZoteroWindow()`.
After navigating to the collection from "Libraries and Collections"
section of an item in the trash. Deleted item does not appear in actual
collections and trying to select it will just re-select the trash.
Fixes: #5060
Define missing cancel method on io object passed from the
editor instance to quickFormat dialog. Without this
Escape keypresses throws an error because io.cancel is
undefined.
Followup to #4859
Asks the user if they want to discard the changes in their existing dialog if
they have made changes.
If there is no citation dialog, but a command is running (like updating the document)
an alert is displayed.
Closes#4855
I don't know if something changed in my Ubuntu installation or in the
Firefox runtime (probably the former), but that code crashes Zotero on
wayland.
Window activation (e.g. citation window) does not seem to be affected.
* Display the progress bar on macOS integration commands more
Avoid displaying before Add/Edit Citations, Add Note and Doc Prefs
since that brings Zotero window to the foreground along with the
relevant dialog, but display the progress bar after confirming those
dialogs, which will bring Zotero to the front, but is still better than
no progress indication.
Display the progress bar for Refresh, Doc Prefs, Unlink and Add/Edit
Bibliography
* Fix progress bar indeterminate state not being visible and not working.
- Update _progressMeter.scss for fx115+
- Some adjustments/resizing to progress bar display in quick format
`firstInSession` wasn't being properly set, so previously failed data
objects weren't being retried until a manual sync. Local files also
weren't being scanned for changes for three hours, but that's probably
better in the age of DB-stored annotations, so leave that off for now
and address properly in #5025.
After further consideration, doing this in-app makes more sense. This
way, we'd never be able to update the desktop file automatically (e.g.,
to add a new MimeType), and removing the template zotero.desktop file
would break all existing installs using our symlink instructions.
In-app, we can update the existing desktop file on restart after an
upgrade, and we can remove the existing symlink.
This reverts commit 0ac57f8262.
Possible we'll get complaints about no longer being able to open
multiple copies of at least runJS and csledit, and we can reconsider if
so, but reopening the existing window is certainly the intended
behavior.
Reverts a change I made to the PR. Only `since=0` would be valid in the
web API, but local items don't have a version by default, so it has to
be `-1` to match those. This probably isn't a great solution, since tools
targeting the web API would use `0`, but this will fix the test for now.
- Fix background color on post-upgrade banner
- Add drop shadow between stacked banners (e.g., post-upgrade banner and
Mac Word banner, which will generally appear together on an upgrade
and have the same background color)
- Align right edge of buttons in post-upgrade and Mac Word
- Unify other spacing and font sizes
- Move 16px padding into the tabbox, which lets us eliminate the
negative margin on #right-pane and the 5px padding to make room for
the focus ring
- Remove width: 100% from the textarea - not necessary and causes
horizontal overflow on macOS
Tested on macOS and Windows.
Treat page as immediately ready after readyState == 'complete', or give
it 100ms after 'interactive'. This is rough, but 'interactive' isn't
always fully ready and 'complete' sometimes never happens.
And enable migration of Extra rows to valid fields in non-strict mode,
but since that's only used by translators now, that probably won't do
anything
Fixes#3422
Fix error if local note or attachment is added to a parent item while
the remote version was added to a collection. Now, the new local parent
item will be added to any collections the remote child item was added
to.
Fixes#2934
- Remove duplicate/missorted locales
- Treat empty files in langpacks as missing and use en-US
- Skip empty lines and comments when using en-US file for empty
non-en-US files, in line with Mozilla langpack files
searchButton setter in search-textbox element adds aria-autocomplete
to the input. It makes screen readers announce that there is an autocomplete
popup, which is misleading (even though it is appropriate for
firefox that actually does have autocomplete-like dropdown).
This overrides the searchButton setter to remove aria-autocomplete
attribute.
Addresses: zotero#4932
Do not close details panel on arrowUp when locator
type menulist is focused to allow for default handling
(e.g. select the next/previous option on windows).
Fixes: #4953
A new function, `Zotero.DB.backUpDatabase()`, by default makes offline
backups, by closing the database, doing a regular file copy, and
reopening the database. It takes an options object with an `online`
flag to make online, incremental backups that can take multiple minutes
to complete, though for now we're still running them on idle.
The old function, `Zotero.DB.backupDatabase(suffix, force)`, is
deprecated and proxies to the new function, making offline backups.
Fixes#4935
When tagsBox item is changed by clicking on an itemTree row,
the blur even never fires on the currently focused tag. So
whatever changes were made will be discarded. To avoid it,
blur any opened tag rows (which triggers a saveTx)
when an item is being set, same way it is done in itemBox.
Also, a small tweak to properly fetch the focused tag
via editable-text:focus-within selector, since editable-text:focus is
always empty because the focus is on the input inside of
editable-text.
Fixes: #4942
For a URL like
`https://ezproxy.school.edu/login?url=http://resolver.ebscohost.com/openurl`,
without a trailing `?`, we were just adding a `%` to the end, making the
URL invalid after the redirection (`/openurl&url_ver=Z39.88-2004`
instead of `/openurl?url_ver=Z39.88-2004`).
(And apparently no one who sent in these URLs actually tested them
in-app?)
Followup to: #4924
Addresses: #4922
Pre-fx128, --toolbar-field-border-color was ThreeDShadow,
while right now it is --input-border-color, which looks
more prominent.
Setting border color explicitly for ThreeDShadow now would make
the border look a bit blurry, while transparent border looks closest
to pre-fx128 look.
Caching Basic Auth credentials with an OPTIONS on /zotero no longer
seems to cause Firefox to pass an Authorization header for
/zotero/AAAAAAAA.prop, so update test to send 401 for the .prop request.
(Presumably /zotero/BBBBBBBB.prop would at least still get the Authorization
header without a 401, but I didn't test that.)
To avoid redownloading from Mozilla whenever fetch_xulrunner runs. We
might want to add cleanup of old versions at some point.
We should add this for Windows and Linux too.
And:
- Use a different port for tests so server tests don't break when Zotero
is already running
- Remove no-op httpServer.enabled pref updates in tests
- It's already enabled, no test disables it, and any test that did
would clean up after itself
- Updating that pref has no effect without a separate call to
Zotero.Server.init()
- Remove unused arg to Zotero.Server.init()
- Cramped tabs due to insufficient padding (new in fx128)
- Off-center text due to margins (our styles)
- Off-center text due to padding-top (platform styles)
Something funky is going on with Content-Disposition: attachment on
displayable content in fx128. It's correctly blocking the download, but
we're no longer getting a STATE_STOP notification when that happens.
The HTML test didn't really test anything different from the
octet-stream one, so we can just remove it.
Fixes#4881
Mac Firefox builds now specify the update channel via a custom macOS
framework:
https://bugzilla.mozilla.org/show_bug.cgi?id=1799332
We set the update channel in the build script, so bundle a custom
ChannelPrefs framework set to "source" and then patch the binary as
necessary for non-source channels. What could go wrong?
- Make the annotations importer more resilient to missing data
- Fix notebooks not being imported in some cases
- Fix an instance of a hidden browser leaked by the Mendeley Importer
We fix `://` or `//` automatically after #3483, but a leading ':'
character would still show an internal `NS_ERROR_MALFORMED_URI` error.
Instead, just say "[url] is not a valid WebDAV URL".
Zotero does not use GitHub Issues for bug reports or feature requests.
Please post all such requests to the Zotero Forums at https://forums.zotero.org, where Zotero developers and many others can help. For confirmed bugs or agreed-upon changes, Zotero developers will create new issues in the relevant repositories.
Development questions involving code, APIs, or other technical topics can be posted to the zotero-dev mailing list at http://groups.google.com/group/zotero-dev.
See https://www.zotero.org/support/zotero_support for more information on how Zotero support works.
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Zotero is a Firefox-based desktop research management application. It runs as a XUL/XHTML application on a Mozilla platform (not Electron, not a web app). The UI uses a mix of XUL/XHTML, custom elements (Web Components), and React components.
## Build & Development Commands
```bash
npm install # Install dependencies
npm run build # Full build (Babel transpile JS/JSX, compile SCSS, browserify, symlink)
npm run clean-build # Clean build output and rebuild
```
### Running Tests
Tests run inside a built Zotero application instance via `test/runtests.sh`:
```bash
test/runtests.sh # Run all tests
test/runtests.sh item # Run a single test file (omit "Test" and .js extension)
test/runtests.sh item collections # Run multiple test files
test/runtests.sh -g "pattern" # Run only tests matching a grep pattern
test/runtests.sh -f # Stop after first failure
test/runtests.sh -s item # Start at a specific test file
test/runtests.sh -e item # End at a specific test file
test/runtests.sh -c # Open JS console, don't quit on completion
test/runtests.sh -b # Skip bundled translator/style installation (for faster startup when not necessary)
Tests use Mocha + Chai (assert style) + Sinon. Test files live in `test/tests/` and follow the naming convention `<module>Test.js`. Test helpers are in `test/content/support.js`.
The test runner automatically triggers `npm run build` if the watch process isn't running, so it's not necessary to manually run a build before running tests.
Tips:
- Pass flags before test names (e.g., `test/runtests.sh -f item`).
- `-f` should almost always be used, since test failures can cause later spurious test failures.
- To view debug logging for a specific test when running multiple tests, add `Zotero.Debug.init(1)` at the beginning of the test and run with `-d 5`.
### Linting
```bash
npx eslint <file> # Lint a specific file
```
ESLint v9 flat config is in `eslint.config.mjs`. Uses `@zotero/eslint-config`, Babel parser, and React plugin. Currently, the code base is not fully linted, so only worry about issues with new or modified code lines.
## Architecture
### Core Layers
- **XPCOM modules** (`chrome/content/zotero/xpcom/`) -- Core business logic. Loaded sequentially by `chrome/content/zotero/zotero.mjs`. All modules attach to the global `Zotero` namespace (e.g., `Zotero.Items`, `Zotero.Sync.Runner`).
- **Data model** (`chrome/content/zotero/xpcom/data/`) -- ORM-like classes for database entities. `Zotero.DataObject` is the base class; `Zotero.Item`, `Zotero.Collection`, `Zotero.Search`, `Zotero.Library`, etc. extend it. Plural classes (`Zotero.Items`, `Zotero.Collections`) manage object caches and queries. Data objects use an async `saveTx()` pattern for persistence.
- **Database** (`chrome/content/zotero/xpcom/db.js`) -- SQLite via Mozilla's mozStorage API. Accessed through `Zotero.DB.queryAsync()`, `Zotero.DB.executeTransaction()`.
- **Sync system** (`chrome/content/zotero/xpcom/sync/`) -- `syncRunner.js` orchestrates sync. `syncEngine.js` handles object-level sync logic. `syncAPIClient.js` communicates with the Zotero API server. `syncLocal.js` manages local sync state. Similar `storage*` files for file syncing, along with zfs.js (Zotero Storage) and webdav.js (WebDAV).
- **HTTP server** (`chrome/content/zotero/xpcom/server/`) -- Local HTTP server for browser connector integration and local API.
- **Custom elements** (`chrome/content/zotero/elements/`) -- XUL custom elements inheriting from `XULElementBase` (defined in `elements/base.js`), which provides lifecycle helpers (`init()`, `destroy()`, `content` getter for templates). Registered via `customElements.define()`.
- **React components** (`chrome/content/zotero/components/`) -- Used for complex interactive UI (tag selector, virtualized table, item tree, collection tree). Major tree views are `itemTree.jsx` and `collectionTree.jsx`.
- **SCSS styles** (`scss/`) -- Compiled to CSS. Platform-specific overrides in `scss/mac/`, `scss/win/`, `scss/linux/`.
- **Localization** -- Fluent (`.ftl` files) in `chrome/locale/en-US/zotero/`. Accessed via `data-l10n-id` attributes or `Zotero.getString()` for legacy `.properties` strings. All new strings must be added to `.ftl` files -- do not add new strings to `.properties` or `.dtd` files.
### Submodules
Several features are developed in separate repos and included as Git submodules:
- `reader/` -- PDF/EPUB/snapshot reader with annotations
- `note-editor/` -- Rich text note editor
- `document-worker/` -- PDF processing (extraction, manipulation)
- `translators/` -- 760+ web translators for importing metadata from websites
Custom Node.js build system (not Webpack). `js-build/config.js` defines what gets built:
- **JS/JSX files** in `chrome/`, `components/`, `defaults/`, `test/` are transpiled via Babel (React JSX, CommonJS modules)
- **SCSS files** in `scss/` and `chrome/skin/` are compiled with Dart Sass
- **Browserify** bundles select npm packages (react-select, sinon, url) for use in the XUL environment
- **Symlinks** connect source files and submodules into the `build/` directory
- The built app reads from `build/`, not directly from source
## Code Style
- Tabs for indentation, not spaces
- Use `let` instead of `const` except for true scalar constants (e.g., `const MAX_SECONDS = 5 * 60;`)
- No cuddled braces (opening brace on same line, but `else`/`catch`/etc. on their own line)
- Use two hyphens `--` in comments, not an em dash
- Indent blank lines to match surrounding indentation level
- Objects attach to the `Zotero` global namespace rather than using ES module exports
- Async code uses `async`/`await` throughout
- Mozilla/XPCOM APIs are available globally: `Cc`, `Ci`, `Cu`, `Cr`, `Services`, `ChromeUtils`, `IOUtils`, `PathUtils`
## Translators
See `translators/CLAUDE.md` for guidelines. Key rules: **never** generate translators from scratch, **never** generate UUIDs, **never** generate test cases -- all of these must be done through Zotero's Scaffold tool.
<menuitemlabel="&scaffold.testing.copyToClipboard;"tooltiptext="Copy the URL or data for the current test to the clipboard"oncommand="Scaffold.copyToClipboard()"/>
<menuitemid="testing_editImport" label="&scaffold.testing.edit;"tooltiptext="Edit the input data for the current test"oncommand="Scaffold.editImportFromTest()"/>
<menuitemid="testing-editImport" label="&scaffold.testing.edit;"tooltiptext="Edit the input data for the current test"oncommand="Scaffold.editImportFromTest()"/>
<menuitemlabel="&scaffold.testing.openUrl.internally;"tooltiptext="Open the URL for the current test in the Scaffold browser"oncommand="Scaffold.openURL(false)"/>
<menuitemlabel="&scaffold.testing.openUrl.externally;"tooltiptext="Open the URL for the current test in your default browser"oncommand="Scaffold.openURL(true)"/>
<buttonobserves="validate-tests"label="&scaffold.testing.delete;"tooltiptext="Delete the selected tests"oncommand="Scaffold.deleteSelectedTests()"/>
<buttonobserves="validate-tests"label="&scaffold.testing.run;"tooltiptext="Run the selected tests"oncommand="Scaffold.runSelectedTests()"/>
<buttonobserves="validate-tests"label="&scaffold.testing.update;"tooltiptext="Run the selected tests and update the test definitions with the latest data"oncommand="Scaffold.updateSelectedTests()"/>
<buttonobserves="validate-tests"label="&scaffold.testing.create.web;"tooltiptext="Create a new test from the current page"oncommand="Scaffold.saveTestFromCurrent('web')"/>
<buttonid="create-test-button"observes="validate-tests"label="&scaffold.testing.create.web;"tooltiptext="Create a new test from the current page"oncommand="Scaffold.saveTestFromCurrent('web')"/>
