The short version was derived from the first three characters of the
version string, so 10.0 produced a detailsURL of "10._changelog".
(cherry picked from commit 5757396197)
The build-and-deploy scripts run scripts and config from the local
checkout but build source files from the tip of the remote branch, so
a stale or wrong-branch checkout could silently build with the wrong
Gecko version or omni patches.
(cherry picked from commit d9d52de516)
While trying to [fix](https://github.com/NixOS/nixpkgs/pull/519431) another issue in the Zotero package for nixpkgs, I discovered a new bug. When setting a non-`source` update channel, the build fails on MacOS with x86 architectures.
The build fails with:
```
source not found twice in ChannelPrefs
source
stringWithCString:encoding:
```
Which definitely means it is caused by the `strings` command [here](https://github.com/zotero/zotero/blob/main/app/mac/set-channel-prefs-channel#L29).
When manually executing the strings command on the file, I can repeat getting only one `source` result on x86 mac, while getting two on arm64 or linux. After digging around, I tried with the flag `-arch all`, and both `source` values are found on x86 mac. Full command: `strings -n 3 -arch all $binary`.
Developer ID-signed Safari web extensions load only in Safari 18.4 and
later -- older Safari blocks them at the code-signing layer -- so macOS
11 and 12 users, whose Safari versions top out at 16.6 and 17.6, lost
the connector with the web extension conversion.
Embed the prebuilt legacy App Extension ($SAFARI_APP_EXTENSION, set by
the deploy scripts) at Contents/PlugIns/ZoteroSafariExtension.appex with
the historical bundle identifier, and move the web extension to
Contents/PlugIns/ZoteroSafariWebExtension.appex with a new
.SafariWebExtension identifier. The web extension's
SFSafariAppExtensionBundleIdentifiersToReplace key causes Safari
versions that can load it to hide the App Extension and migrate its
enabled state, while older Safari shows only the App Extension.
Also remove the removed-files entries for the App Extension's
resources, which would otherwise delete them on update.
https://forums.zotero.org/discussion/132925/
hdiutil makehybrid gives every file on the image non-empty Finder info
(it sets an icon location), which appears as a com.apple.FinderInfo
extended attribute. codesign --verify --strict rejects FinderInfo as
detritus, and it can cause Safari to fail to load the web extension.
(Reported by a user with Safari 17.6 on macOS 12. No other reports from
the beta, so maybe not universal.) The Safari App Extension apparently
wasn't affected, since it shipped with the same attributes for years.
Convert the hybrid image to a read-write image, mount it, strip the
attributes from its files, and compress from that. The volume header is
untouched, so the open-folder flag that makes Finder open the volume
window on mount is preserved.
Only fresh installs from the DMG were affected. The updater writes fresh
files without the attributes, so copies updated in place were clean.
(Notably, this is the reverse of the post-update extension breakage for
which the standard advice has always been to delete Zotero.app and
redownload. Following that advice is exactly what resulted in a broken
copy here.)
https://forums.zotero.org/discussion/132925/
The connector is now a Safari web extension, so the bundled appex is a
static wrapper stub whose placeholder resources are replaced at build
time with a connector build via $SAFARI_EXT_RESOURCES. The stub only
needs to be rebuilt when the native wrapper project changes, not for
connector releases.
JNA 5.x ships per-architecture macOS native libraries instead of a
single com/sun/jna/darwin/ directory, so the notarization re-signing
step failed with "filename not matched".
This step can be removed entirely once the plugin ships a jna.jar
without the macOS natives, which are never loaded.
Cache patches and compressed files by content hash so that they're reused
across FROM versions and releases. Cache entries are touched when used and
expire after 60 days.
The cache key includes a tag covering the xz/mbsdiff versions and the
compression options, so entries are invalidated when the toolchain changes.
Run the per-file mbsdiff/xz work in make_incremental_update.sh and
make_full_update.sh through a parallel job pool, largest files first, and
assemble the manifests serially afterward, producing byte-identical MARs.
Set UPDATE_PACKAGING_JOBS to override the number of parallel jobs.
The sorted file list is split into contiguous chunks of roughly equal total
file size, using size as a stand-in for run time, so new test files are
included automatically and slow test files are spread across shards rather
than landing in the same shard by chance. Contiguous chunks preserve the
alphabetical run order of a full run, so files keep the same preceding files
as in a full run except at chunk starts, and a shard can be reproduced locally
by passing its first and last files to -s and -e.
After a full build, dir_build saves a manifest of build/ files to
staging/.build-manifest. On subsequent runs, if all changed files are
ones that build.sh copies into omni.ja unmodified (chrome/, components/,
resource/, and test/ when tests are staged), zip just those files into
the staged omni.ja instead of rebuilding, taking rebuilds from ~15
seconds to ~0.3 seconds on an M1 Mac. Files are prescreened by size and
mtime so that only changed files need to be hashed. Zotero .ftl files
are also updated at their localization/<locale>/ paths, and test files
are also copied to the staged tests/ directory.
Any other change triggers an automatic full rebuild: files transformed
by build.sh (defaults/, chrome.manifest, version, translators/, styles/,
mozilla .ftl files, CSL locales), removed files, changes to build inputs
in app/ (detected via a size/mtime fingerprint, with xulrunner runtimes
covered by the hash-* files written by fetch_xulrunner), or requesting
tests or devtools that the staged build doesn't include.
Other changes:
- dir_build no longer takes -q and always skips omni.ja compression and
optimization, which only matter for distribution builds made via
build.sh. Use -f (dir_build or build_and_run) to force a full rebuild.
- build_and_run now always rebuilds. -r is deprecated, and -n skips the
rebuild and just launches the app.
- build_and_run no longer passes -purgecaches. Startup caches are
invalidated automatically when the BuildID changes, which now happens
whenever omni.ja is modified (including via add_omni_file), so
relaunching an unchanged build can use the startup cache.
- build_and_run and runtests.sh invoke js-build directly instead of via
'npm run', which saves ~270ms of npm overhead per build.
- The Word integration dylib is now ad-hoc-signed by dir_build, and only
on full rebuilds, since incremental updates don't invalidate the
existing signature. This also covers test builds, which were never
signed before.
- dir_build removes broken symlinks left in build/ when source files are
deleted, which previously broke rsync in prepare_build.
Previous command only worked with OpenSSL's labeled output where the hash is the second field; the update also handles LibreSSL (macOS), which prints just the bare hash.
Wraps the values stored in nsILoginManager with OSKeyStore, which derives
its master key from Keychain on macOS, DPAPI on Windows, and libsecret on
Linux. A copy of the profile alone is no longer enough to extract these
credentials.
Existing plaintext entries are mirrored once per session to a new
"(encrypted)" realm but preserved in the original realm so a user can
still downgrade to a release that doesn't know about encryption. Active
credential changes (sign in, sign out, password change) write to the
encrypted realm only and remove the legacy entry. A future version can
clear any remaining legacy entries on startup.
Patches MOZ_APP_BASENAME in the bundled runtime so the keychain master
key is labeled "Zotero Encrypted Storage" rather than "Firefox Encrypted
Storage", with a check_line guard so a future Mozilla change to the
OSKeyStore label format fails the build instead of silently rebranding
the entry. Also fixes check_line to take an explicit file argument.
On macOS 27 and later, the installer gets access to the Word startup
folder via a folder-selection dialog rather than an OS permission
prompt, so support an adjusted banner message and add strings for the
folder dialog.
Ignore persisted compatibility flag from previous app version in non-stable releases to fix disabled for wrongly using old flag
Ignore max version compatibility for plugin update check on non-stable releases
Relevant: https://forums.zotero.org/discussion/131096/
---------
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Fetch pinned-incrementals-{platform} from the deploy server and merge
with the normal last-N incrementals list. This ensures incremental MARs
are always built from key versions (e.g., last release of each major
version) regardless of how many newer versions have been released.
Split build_and_deploy into build_for_deploy (builds, uploads to S3,
rsyncs manifests) and a deploy script in client-downloads (updates
updates JSON, updates incrementals, runs deploy command).
Release scripts support -b flag for build-only (deploy later manually).
Beta/dev/test scripts build and deploy automatically via SSH.
Move incrementals files from S3 to deploy server. Update DEPLOY_PATH to
root of client-downloads so manifests and deploy script paths are
derived from it.
