Commit graph

89 commits

Author SHA1 Message Date
Dan Stillman
7abcf23ef7 Move PMC lookup after open-access lookup
The PMC lookup in #5906 was added before the OA lookup, but the OA
lookup often returns a PMC URL of its own (including, at least now, for
the DOI given in the PR), with a direct file URL rather than a page to
scrape, so try it first and fall back to the PMCID-derived URL only when
it turns up nothing.
2026-07-30 20:56:28 -04:00
Tom Najdek
c39f16891f Extend the template engine and add template validation (#5989)
Some checks are pending
CI / Test (shard 1) (push) Waiting to run
CI / Test (shard 2) (push) Waiting to run
CI / Test (shard 3) (push) Waiting to run
CI / Test (shard 4) (push) Waiting to run
CI / Utilities Tests (push) Waiting to run
CI / Build, Upload (push) Waiting to run
- Fix multiple potential scenarios causing a template engine crash
 - Add support for specifying string literals in the template engine
 - Validate `if/else/elseif` order and matching clause closures
 - Validate to ensure every `{{` is properly closed with a matching `}}`
 - When a template is invalid, display a warning, do not offer batch-renaming tools, do not update synced setting
 - When a template is invalid, prompt the user to fix or reset the template when

Closes #5965
2026-07-29 13:30:12 -04:00
Dan Stillman
9eea7d9a0c Disable JavaScript when indexing HTML files
Page scripts otherwise run during indexing and can hang it. JavaScript
was disabled here in 2a7f31813e for the same reason, but that was lost
in the fx-compat rewrite (13adfd131c).

https://forums.zotero.org/discussion/132901/
2026-07-24 13:11:50 -04:00
Tom Najdek
6092b0335c
Fix auto-renaming not triggering when parent item type changes 2026-06-19 15:04:56 +02:00
pi-dal
f03004bb9e
fix(attachments): add PMCID fallback for PubMed resolver lookup (#5906)
PubMed saves can include a PMCID even when DOI-based full-text lookup
fails to produce a PDF. In that case the connector asks the desktop
attachment resolver flow for fallback sources, but getFileResolvers()
only considered DOI and URL data, so Zotero never tried the matching
PMC article URL.

Treat PMCID as a valid signal for file lookup eligibility and add a
PubMed Central OA resolver before DOI-based OA lookup. Cover the new
fallback path in attachment resolver tests and the connector
hasAttachmentResolvers endpoint.
2026-06-15 14:57:49 -04:00
Abe Jellinek
f0474fa250 Add support for clearing challenge in browser during translation 2026-04-23 15:34:28 -04:00
Dan Stillman
ce424c6b67 Fix attachment tests after HTTP.download() rewrite
Add a spy on Zotero.HTTP.download and update call-count assertions to
split between request() and download() calls. Previously download()
delegated to request(), so a single requestStub tracked everything.
2026-04-17 15:21:25 -04:00
Abe Jellinek
8b446b7fc5
Set auto attachment title when not renaming (#5748) 2026-01-27 11:44:50 -05:00
Tom Najdek
d6fc06cd7c
File renaming: Allow local time conversion for accessDate (#5707) 2026-01-20 11:00:52 -05:00
Tom Najdek
49325e27d9
Fix attachment matching filename isn't correctly retitled (#5637) 2025-11-17 10:40:38 -05:00
Tom Najdek
b75ebd6f41
Fix auto-rename not triggering when creators change (#5567) 2025-10-08 18:19:21 +02:00
Tom Najdek
5cb537274e
Fix a couple of issues with file renaming (#5562) (#5566)
* Fixed incorrect new name generation for attachments without a file extension and not present on disk
* Fixed invalid preview in certain edge cases
* Added tests covering file renaming in general and the above issues
2025-10-08 00:39:54 -04:00
Tom Najdek
e896fd0137
Add continuous-renaming functionality for attachment files (#3860)
Resolves #1685
2025-09-11 04:54:03 -04:00
Tom Najdek
cd82ab7c59
Enable support for Unicode features in file renaming by default (#5521) 2025-08-28 11:21:47 -04:00
Abe Jellinek
964b5a0e88 fx140: Fix remaining bare ChromeUtils.import() calls 2025-07-30 22:30:54 -04:00
Abe Jellinek
67d2e1cead fx140: Asyncify/ESMify tests
They seem to be succeeding when run individually, but failing when
run as a whole. Not sure why yet.
2025-07-30 22:30:53 -04:00
Abe Jellinek
488bcc484f Replace uses of Zotero.Promise.defer() as a constructor 2025-07-30 22:30:33 -04:00
Abe Jellinek
ece2b59700 fx140: More progress toward startup 2025-07-30 22:18:36 -04:00
Dan Stillman
d1ea8a397b Use BrowserDownload for 403 when enforcing type in downloadFile()
Fixed regression (probably in 1f401f0897) causing Find Full Text not to
use BrowserDownload for a 403

https://forums.zotero.org/discussion/125581/zotero-suddenly-fails-to-retrieve-full-text-although-i-have-access
2025-07-22 22:12:00 -04:00
Tom Najdek
e50787c94f
File renaming: Add pascal case. Fix unicode handling. (#4876) 2025-06-11 01:03:34 -04:00
Tom Najdek
4f03c5d360
File renaming: add support for counting creators (#5175)
Also extends the templating engine to support inequality comparisons.
2025-04-02 05:31:50 -04:00
Dan Stillman
a464fe0408 Update "Find Full Text" tests for Zotero.HTTP.download()
Downloads are now included in request() call counts
2025-03-05 06:34:25 -05:00
Abe Jellinek
d9550bb5df Server: Expose actual bound port (#4903)
And:
- Use a different port for tests so server tests don't break when Zotero
  is already running
- Remove no-op httpServer.enabled pref updates in tests
   - It's already enabled, no test disables it, and any test that did
     would clean up after itself
   - Updating that pref has no effect without a separate call to
     Zotero.Server.init()
- Remove unused arg to Zotero.Server.init()
2024-12-14 01:16:03 -05:00
Abe Jellinek
dd1601793c
Don't set default attachment title if not renaming file (#4459)
Except from Rename File from Parent Metadata.
2024-07-31 01:39:25 -04:00
Tom Najdek
00ae8bb9b2 Add more features to the file renaming functionality (#4424)
* New `attachmentTitle` field, returns the title of the current attachment (or
  the future title of the attachment being created)
* New function `match` to enable testing values with a regex.
* New function `start` to enable truncating from the beginning.
* Ignore new line characters in the template for easier editing.
* Avoid repeated characters when changing case (snake/dash)
* Increase the size of the template input field.

Closes #3252
2024-07-28 02:57:14 -04:00
Grace Dinh
29f4aece24 Add regex replace feature for file renaming (#3562) 2024-07-28 02:56:19 -04:00
Abe Jellinek
4187819cd1
File renaming: Match content type prefixes, add UI (#4431) 2024-07-27 03:01:43 -04:00
Abe Jellinek
7020d60351 Generalize Find Available PDF -> Find Full Text (#4397) 2024-07-27 02:11:22 -04:00
Tom Najdek
f227aeb6e0
File renaming: suppress duplicate suffixes #3317 (#4389) 2024-07-16 01:59:07 -04:00
Abe Jellinek
833ecca364
Set automatic titles in more or less all cases (#4369)
By moving the setAutoAttachmentTitle() calls to importFromFile() /
_addToDB().

Also:

- Chop off file extension when setting the parent's title based on the
  filename in Create Parent Item -> Manual Entry.
- Fix Manual Entry not renaming the attachment correctly by awaiting
  createEmptyParent().
2024-07-14 23:37:24 -04:00
Tom Najdek
1b751d675b
Trim spaces from values in getFileBaseNameFromItem (#3711) 2024-02-19 04:43:19 -05:00
Tom Najdek
abe8def0f1
Trim leading/trailing space in filename format string. Fix #3701 2024-02-16 10:17:03 +01:00
Adomas Ven
8b77c96e97
Displays a browser window to clear captcha when saving attachments. (#3526)
- Currently enabled only for ScienceDirect. Can be enabled via a whitelist
- Matches the HiddenBrowser loaded HTML page for a captcha element. If
  the captcha element class changes, this will break (but the
  alternative is potentially displaying a captcha clearing window when
  something else that is not a captcha guard is loaded).
- Captcha clear timeout for 60s.
- Doesn't automatically switch focus back to the browser which intiated
  the item save via the Connector.
- Stores the cookies used to clear the captcha for future saves from the
  same domain. Discards Connector supplied User Agent, since CF bot
  detector checks UA header against actual UA behavior like TLS handshake
  and if the UA acts different to what it's supposed to, the bot
  challenge is not cleared.

Other changes:
- Adjusted the cookie sandbox to allow multiple cookie sandboxes to be
  active (and simplified some legacy code that was meant to cover a bug
  in old FX codebase).
- HiddenBrowser API changed to be Object oriented, translator tester
  in the translate repo will need to be updated after a merge (have the
  change ready).
- Improved Connector Server attachment progress handling
2023-12-27 04:43:50 -05:00
Dan Stillman
fb96cd595d Add startHTTPServer() support function
Centralize httpd creation and add automatic retry to try to deal with
NS_ERROR_SOCKET_ADDRESS_IN_USE errors in CI.
2023-08-16 01:16:49 -04:00
Dan Stillman
69ba2310a2 Actually fix NS_ERROR_SOCKET_ADDRESS_IN_USE during tests 2023-08-09 08:05:30 -04:00
Dan Stillman
4ac3128b17 Use more ports for attachment tests
To try to avoid this stupid NS_ERROR_SOCKET_ADDRESS_IN_USE error
2023-08-07 17:36:19 -04:00
Dan Stillman
9b0ce9558c
attachmentRenameFormatStringattachmentRenameTemplate (#3249) 2023-08-04 05:58:15 -04:00
Dan Stillman
be1ab236c8 Better cycling through httpd.js ports to avoid CI failures 2023-07-26 07:25:22 -04:00
Abe Jellinek
676f820f87
Strip bidi control characters in filenames and elsewhere (#3208)
Passing unformatted = true to Item#getField() now returns a bidi control
character-less result, and we use that in Reader#updateTitle() and
getFileBaseNameFromItem() to prevent bidi control characters from showing up in
filenames and window titles (the former everywhere, the latter on Windows only).

We also strip bidi control characters in getValidFileName() to be extra safe.
2023-07-22 03:30:28 -04:00
Tom Najdek
0ba766f2e0
Customizable renaming rules #1413 (#2297) 2023-07-20 06:50:34 -04:00
Dan Stillman
14f7d3acad Cycle through httpd ports to prevent CI failures
We didn't seem to be doing this anymore, even though we had a comment
for it, and some tests were failing with NS_ERROR_SOCKET_ADDRESS_IN_USE.
2023-06-19 06:54:19 -04:00
Dan Stillman
0858960d33 Better logging for request count mismatches in Find Available PDF tests 2023-04-29 17:50:49 -04:00
Dan Stillman
2796e6c80a Fix attachment tests that depend on HTML indexing
HTML files are now indexed instead of read directly, and indexing was
previous skipped in tests and otherwise performed on a delay, so set a
flag in the affected tests that triggers inline indexing.
2023-04-15 00:24:35 -04:00
Abe Jellinek
0612a9e6f5 fx-compat: Run translation and SingleFile in [hidden] browser
And replace loadDocuments().
2023-04-14 11:44:44 -04:00
Dan Stillman
7ffc509ee6 Fix response content type in Find Available PDF test 2022-11-21 01:14:07 -05:00
Dan Stillman
c6df0b586c Use clearer name for Find Available PDF tests 2022-11-21 01:14:07 -05:00
Dan Stillman
b5862ba780 Handle relative PDF links when using custom PDF resolver 2022-11-21 01:14:07 -05:00
Dan Stillman
8e59e49d29 Avoid infinite/excessive loops in Find Available PDF
https://forums.zotero.org/discussion/100634/potential-infinite-loop-when-trying-to-find-available-pdf

Closes #2883
2022-10-30 04:44:31 -04:00
Dan Stillman
13adfd131c fx-compat: Update full-text indexing
Use the new PageData mechanism for character set detection, don't try to
index HTML files directly without properly detecting the charset, and
generally simplify the indexing code.

HTML files are now considered cached files that require indexing and
won't be indexed automatically in Zotero.FullText.findTextInItems(),
which breaks certain expectations, including in some tests. This will
need to be addressed.
2022-06-17 20:29:01 -04:00
Adomas Ven
4405b59044
Add a function to download PDFs via a browser (#2248)
Fixes zotero/translators#2739
2021-12-02 04:27:33 -05:00