The embeddings are a local, rebuildable, model-specific index, so they
don't belong in the main database or its backups. Follow the full-text
content index pattern: a lazily attached embeddings.sqlite versioned via
PRAGMA user_version, tied to the main database by localUserKey, with
corruption recovery and idle-maintenance vacuuming via the DBConnection
hooks. Since a cross-database foreign key isn't possible, item deletions
now clear embeddings via the notifier, and the indexed-model identity
moves from a pref into the database's meta table.
"bestMatch" names what the condition does -- rank results by how well
they match the text -- rather than the current scoring mechanism, so
the name can account for later changes to how it works. Rename the
condition-facing identifiers with it; the embeddings engine keeps its
similarity vocabulary.
- added environment to run embedding models locally
(transformers.js, ONNX Runtime WASM binary, etc.). The actual
inference execution happens in a separate worker environment (worker.js)
- added local itemEmbeddings table to store embeddings locally
- in advanced preferences, one can select two options for
semantic search model: english and multilingual. English model
(bge-small-en-v1.5) is better for english-only corpus
but multilingual (multilingual-e5-small) is necessary to handle
abstracts with any other language than english. We can add
more language-specific models as needed.
- when the model is selected, Zotero.Embeddings.download
will download the model (quantized ~100mb) and store it locally.
- Zotero.Embeddings.Indexing will start a process to
index all regular items with title+abstract. It happens in batches
and takes some time. The progress will appear in the
advanced preferences pane. Embeddings are inserted
into itemEmbeddings SQL table. For now, the table is local
only, no syncing is involved.
- when embedding model pref is set to "Disabled", the model
is deleted and embeddings table is cleared.
- when an embedding model is selected, quick search dropdown
has a new "Similarity" mode, which will run semantic search
on the current scope of items.
- semantic search does not clearly define what counts
as "relevant" and what is "not relevant". In addition,
it will change depending on the library and query. So
we cannot semantically filter out items the way
it is done via SQL. Semantic search returns the ranking
but items cannot be sorted because it is done by the itemTree
based on column selection.
So in "similarity" quicksearch mode, there is also a dropdown
to select how many top relevant items to keep (top 5 - top 100).
It allows the user to keep the most relevant items depending
on the context, without conflicting with itemTree sorting.
- semantic search happens in-memory. On a large 5K library
it's fast, but we could consider sqlite-vec extension if
needed.
The menu labeled Annotation Type listed "Highlight annotation" and "Image
Annotation", from the strings the reader announces annotations with. Use
the short names, which existed for two of the six.
mozinference (which is currently CPU-only) is better for many tasks, but
plugins may want WebGPU for some features (e.g., chatbots), so just
follow Firefox, which currently enables it by default for Windows and
Apple Silicon macOS. Adds 5.8 MB compressed to the Windows installer.
Binding is meaningful for a condition that matches at every level -- a
tag bound to an attachment means the tag is on the attachment -- but a
group carrying one lost the binding as soon as the search was
serialized, so "items with an attachment tagged foo" couldn't be built.
The row label builder skipped values that weren't strings, so a numeric
cell -- a note count, or a pre-1000 year (which is parsed with
parseInt()) -- announced only its column
name.
Also switch to formatColumnName() while we're here, so row labels
resolve the column name the same way the header does.
The guard tested the newly clicked column's settings entry while the
delete targeted the previously sorted column's, so the old direction
survived when the new column had no entry, and clicking threw when the
old column had none.
Skip crashhelper (all platforms -- the crash reporter is disabled) and
desktop-launcher and nmhproxy.exe (Windows).
Also skip dxcompiler.dll (Windows, 22 MB), which is loaded only by
WebGPU, now also disabled via pref. Nothing renders with WebGPU, and
local ML -- including any future plugin API -- should go through
Firefox's inference libraries (mozinference/onnxruntime), which are
compiled CPU-only.
XULButtonElement only opens a type=menu button's menu when the button
is the event's original target, which worked before because xul.css
gave .toolbarbutton-menu-dropmarker 'pointer-events: none'. That rule
was removed along with the dropmarker element, so clicks on our
recreated dropmarker did nothing.
The menu icon element is now an <html:img> that renders the 'image'
attribute or 'content: var(--menuitem-icon)', and 'list-style-image' is
ignored, so iconic menuitems showed broken-image placeholders on
Windows and Linux. list-style-image is kept for native macOS menus,
which still read it.
Firefox 153 removed the chrome://browser/skin/window-controls/ SVGs and
now draws the caption buttons with Segoe Fluent Icons glyphs (falling
back to Segoe MDL2 Assets on Windows 10), which broke the Windows build.
Port the new browser.css button styling and stop keeping the removed
directory in build.sh.
An exact-match condition compared with SQLite's case-sensitive '=' and
skipped the normalized shadow columns, so 'publication is "review of
finance"' missed "Review of Finance" while every other kind of search
matched it.
Better fix than d5aacafe3f -- we don't need source-repo info in our
builds at all, so just disable MOZ_INCLUDE_SOURCE_INFO rather than
synthesizing it.
Firefox's official-build source-repo.h generator only auto-detects
Mercurial, so building from our git checkout left the file empty and
packaging failed with "no preprocessor directives found". Derive
MOZ_SOURCE_REPO and MOZ_SOURCE_CHANGESET from git in the mozconfig so
configure records them for any invocation, including a manual
./mach package.
app/win/build added the cross-compilation target to whatever toolchain
was default before switching to RUST_VERSION, so after a Rust version
bump the new toolchain lacked the aarch64/i686 target and the arm and
win32 builds failed at configure. Set the default first.
Bug 1974578 added oncommand to GlobalEventHandlers for the Invoker
Commands API, so the assignment in _initImportedNodesPostInsert() -- an
inert expando write until now -- started setting the WebIDL event
handler, and [LegacyTreatNonObjectAsNull] turns a string into null. That
wiped the handlers compiled when the pane fragment was imported, leaving
every inline oncommand handler in the preferences panes dead.
importNode() compiles the attribute handlers itself, so the loop can
just be removed.
Bug 1980856 removed both wantdropmarker support and the rule that gave
.toolbarbutton-menu-dropmarker its image, once Firefox's last consumer
stopped using it, so the New Item, Add Attachment, and New Note buttons
lost their chevrons, along with the ones in advanced search, the feed
settings dialog, and Scaffold.
Draw it with a background, since a <dropmarker> doesn't render
list-style-image, and use chevron-6 to match the quick search dropmarker
and the menu marker on our own buttons. The rule also moves out of the
toolbarbutton block, since a plain <button> can host one too.
Firefox now creates a clear button for <input type="search"> in chrome
documents regardless of layout.forms.input-type-search.enabled (bug
1655503), which doubled up with the one we draw ourselves.
There's no mac wizard.css -- the shared one from the global list is all
macOS needs, as on Linux -- so this only produced a missing-file warning
whenever a <wizard> was created.
Without -f, curl saves the error page and the download appears to succeed,
so a missing build fails later with a confusing "Unrecognized archive
format" from tar.
The sidebar customizations all broke: the category buttons are
moz-page-nav-buttons now rather than button[name="..."], and the Settings
and Support links moved out of <sidebar-footer> into a secondary nav. The
sidebar also has a heading now that duplicates the main pane's "Manage
Your Plugins", so hide the whole column instead of the pieces.
-moz-mac-help-button draws the question mark itself, and the Help label is
now painted over it at the bottom of the settings panes. Firefox hides the
label the same way for the Page Info help button (bug 2009625).
Babel strips the trailing newline from defaults/preferences/zotero.js, so
the platform-specific prefs appended after it shared a line with the last
Zotero pref.
SelectableProfile.sys.mjs now statically imports BackupService, which
statically imports chrome://browser/content/backup/backup-constants.mjs,
so startup failed once SelectableProfileService was instantiated as a
command-line handler.
nsMenuBarX now looks for a menuitem with the id menu_settings on Ventura
and later, falling back to menu_preferences only on older versions, so
declare both the way Firefox does.
Bug 2037682 removed nsIPK11TokenDB in favor of getting the internal key
token directly from @mozilla.org/security/internalkeytoken;1, and bug
2043434 replaced initPassword()/needsUserInit with changePassword().
Bug 2017957 made Cu.Sandbox freeze built-ins by default for
system-principal sandboxes, so the translation framework's assignments to
Promise silently failed and loading the framework threw on
Zotero.Promise.method.
Bug 2008041 made disabled, checked, hidden, collapsed, and selected
boolean attributes, so their value is empty and [disabled="true"] no
longer matches. Nothing sets any of them to "false", so matching on
presence alone is equivalent.
Bug 2008041's change to boolean attributes also covers hidden and
collapsed, whose UA selectors became [hidden] and [collapsed], so
setAttribute('hidden', false) now hides the element. Switch the setters
that can be passed a falsy value to toggleAttribute(), read them with
hasAttribute(), and match the [collapsed=true] selectors in our own
stylesheets to the new presence-only form.
Bug 2008041 made them html-style boolean attributes, so presence alone
means true and setAttribute('disabled', false) now disables the element.
Switch the sites that can be passed a falsy value to toggleAttribute(),
read them with hasAttribute(), and drop a disabled="false" from markup.
Sites that only ever pass true are left alone.
Bug 2009806 moved XUL checkbox click handling into C++ and dropped the
CheckboxStateChange event, so listen for command instead. The event now
only fires on user interaction, so the tests have to click the checkboxes
rather than assign to .checked.
Bug 2011326 moved remote type prediction into C++ and removed
E10SUtils.predictOriginAttributes() and getRemoteTypeForURI(). The
resulting TypeError meant nothing ever loaded in the hidden browser.
Translators are evaluated with the system principal, which worked until
now only because nsContentSecurityUtils::IsEvalAllowed() exempted any
profile with "JS hacks" present -- which for us meant
xpinstall.signatures.required being false. Bug 2038660 dropped that
exemption, so ask for eval explicitly.
findLogins() now throws, and removeLogin()/modifyLogin()/removeAllLogins()
are Async-suffixed and return promises. Without this, Zotero couldn't read
the stored API key and showed the corrupted-logins-database error on
startup.
There's no synchronous read left, so Zotero.Sync.Runner.enabled -- which is
consulted from places that can't await -- now reads a value that
hasCredentials() caches.
Bug 2043845 put the FORCE_ALLOWED_DTD content policy type behind a pref
that's off for everything but Thunderbird, so NS_NewChannel() refused the
load with NS_ERROR_CONTENT_BLOCKED and the XML parser silently skipped the
entity. Custom elements whose markup uses DTD entities then failed to
parse with "not well-formed XML".
Bug 2038660 applies "script-src chrome: resource: moz-src:" to every
chrome: document, which blocks inline <script>s and inline event
handlers, so the main window loaded but ran none of its scripts.
We should move our inline scripts and event handlers into separate files
so that we can drop the pref.
Mozilla removed the widget and its stylesheet in bug 1968916 without a
drop-in replacement, so bring both into Zotero. The two rules that lived
in Firefox's UA sheets (text-shadow and -moz-window-dragging) move to
_search.scss, and the aria-autocomplete removal that fetch_xulrunner
used to patch in is now part of the widget itself.
