Previously only 503 + Retry-After was retried automatically; 429 was
handled only inside the sync API client. Extend _retryOnServerError()
to also retry on 429, honoring Retry-After on both 429 and 503 and
falling back to the existing exponential backoff otherwise.
HTTP.download() was rewritten in 0fe31b0f04 to use fetch() and build the
Basic auth header itself via btoa(username + ':' + password), but the
username and password come from nsIURI.username/password, which are
percent-encoded. As a result, a username like "user@example.com" was
sent as "user%40example.com", causing a 401 on every WebDAV download for
any user with @, :, space, etc. in their username. Other request types
were unaffected because they go through xmlhttp.open(method, url, true,
username, password), which decodes internally.
Decode username and password in _parseURI() so the values returned can
be used directly for Basic auth (and as a side effect, fix the other
display/use sites that were getting the percent-encoded form).
https://forums.zotero.org/discussion/131174/zotero-10-betas-1-2-3-cant-download-from-webdav-http-401
- Replace XHR-based download with fetch() + response.body streaming,
writing chunks to disk via IOUtils instead of buffering the entire
response in memory
- Separate out the retry and URL-parsing logic so it can be reused
between request() and download()
- Split the ZFS download code into a request() with `followRedirects:
false` to get the metadata headers and a separate download() to
download the file
Fixes#5476, Downloading of large files is broken
Using the Headers class from the Fetch API.
Before, the added test would fail: `_requestInternal()`, not finding a header
named `Content-Type` (case sensitive), would set it to
`application/x-www-form-urlencoded`. XMLHttpRequest, upon being given both
`content-type`: `application/json`) and `Content-Type`:
`application/x-www-form-urlencoded`, would helpfully merge the two, producing
`content-type`: `application/json, application/x-www-form-urlencoded`. That's
obviously not the correct behavior.
- Move 5xx retries and connection checking out of the sync API client
and into HTTP.request() so that they apply to all requests. 429 handling
remains in the API client, since not all callers necessarily want to
handle that the same way. Callers can still handle 5xx themselves by
including the relevant 5xx status codes in `successCodes` or by passing
`errorDelayMax: 0`.
- Add `cancellerReceiver` option, which is a callback that receives a
function that will cancel the request, whether it's an active request
or an automatic delay before a 5xx retry.
This also updates Sinon to 7.3.2.
Currently only .status and .getResponseHeader() (for getting 'Location')
are available in the returned object, but we could make the body
available if necessary.
processDocuments() now uses an XHR 'document' request, wrapped to
provide a 'location' property, and uses promises for a simpler call
signature (though the old one will continue to work, for existing
translators). 'done' and 'exception' can now be handled via promises,
and in the translator sandbox an optional noCompleteOnError argument
instructs it not to automatically cancel the translation process with an
error (e.g., for supplementary materials).
Since we do need a hidden browser in some situations (e.g., for saving
snapshots), the old hidden-browser-based processDocuments() is still
available as Zotero.HTTP.loadDocuments().
This hopefully also fixes various problems with document property access
in translation-server.
