Commit graph

108 commits

Author SHA1 Message Date
Abe Jellinek
81f80a62e3 Update RDF tests 2026-03-06 16:51:25 -05:00
Tom Najdek
818ce564bb
Improve RTF scan. Add test coverage. (#5672) 2026-02-02 15:03:57 -05:00
Dan Stillman
0af7c56b7e Update sample data for tests, again
There's some bug where this has to be generated multiple times to be
correct.
2026-01-13 13:14:00 -05:00
Dan Stillman
395aadff4e Update sample data for tests 2026-01-13 13:05:23 -05:00
Dan Stillman
689e59f161 Update global schema
Remove "Creator" from "Audio Recording"
2025-12-26 23:45:24 -05:00
Dan Stillman
3256273015 Update global schema
"Uploader" → "Creator" on Video Recording
2025-12-26 23:35:43 -05:00
Dan Stillman
f0ff173ad2 Update global schema to remove arXivID field 2025-12-26 15:36:13 -05:00
Dan Stillman
b94dac5586 Update sample data for new item fields 2025-12-25 00:20:34 -05:00
Abe Jellinek
08e4d6b06b Stop using restoreProcessorState() (#5516)
restoreProcessorState() breaks locale-specific punctuation (and is
deprecated).

rebuildProcessorState() is enough if we're passing an empty citation
list; if we're reinitializing with a new non-empty list, call
updateItems() first.

Also:

- Add tests for this issue and for potential regressions in disambiguation handling
2025-08-26 00:44:14 -04:00
Dan Stillman
d358915707 Update sample data for podcast/date mapping 2025-02-05 03:08:32 -05:00
Tom Najdek
d3ef6cfebd
Improve importing from Mendeley
- Make the annotations importer more resilient to missing data
- Fix notebooks not being imported in some cases
- Fix an instance of a hidden browser leaked by the Mendeley Importer
2024-12-11 12:44:13 +01:00
Abe Jellinek
bc2c7956db FeedProcessor: Handle dc:creator in Atom feeds
Fixes #4857
2024-11-20 10:05:39 -05:00
Tom Najdek
055fbb68fc
Import Mendeley notebooks (#4829) 2024-11-12 00:13:17 -05:00
Abe Jellinek
9f8c5c8581
Add support for importing KOReader & Calibre annotations (#4780) 2024-11-08 01:16:10 -05:00
Abe Jellinek
206496332d EPUB recognition: Fix DOI search results being thrown away
https://forums.zotero.org/discussion/116524/epub-articles-and-books-not-getting-metadata-like-pdfs
2024-08-06 11:07:29 -04:00
Dan Stillman
137a1ddcc0 Missing file for 639b4b5623 2024-06-15 00:45:28 -04:00
Abe Jellinek
3176516b79 Feeds: Fix error parsing items with rights information
- 59afef6 added a duplicate assignment to item.rights that was putting
  a TextConstruct(!!!!) in the item JSON field instead of a string
- info.rights was being set to a TextConstruct (preexisting issue!)
2024-05-23 14:20:44 -04:00
Abe Jellinek
59afef6c2e
Feeds: Refactor to move field extraction to FeedProcessor (#4158)
And support more PRISM fields.
2024-05-22 18:29:58 -04:00
Abe Jellinek
da1eb6fda9 Feeds: Prefer content to summary when available 2024-04-30 02:14:57 -04:00
Abe Jellinek
a5393ca0e5 Merge: Keep external annotations on master, don't erase on other
PDFWorker only re-imports external annotations when the file changes on
disk. Keep annotation items corresponding to external annotations so
that they don't disappear after the merge (and then come back when the
file is edited).

Tweaks behavior introduced in 2aa34a6.

https://forums.zotero.org/discussion/113943/zotero-7-beta-merging-pdf-files-leaves-ghost-external-annotations
2024-04-25 16:35:09 -04:00
Abe Jellinek
8b13f717b4 SAXXMLReader: Handle non-UTF-8 encodings (#3846) 2024-04-20 06:41:51 -04:00
Abe Jellinek
1f599283df
Fix indexing files with text content types that Firefox won't display (#3708) 2024-02-19 05:11:16 -05:00
Abe Jellinek
2ef560f7d8 Extract ISBNs and DOIs from EPUB content (#64)
And move EPUB functionality to class.
2023-08-07 16:07:55 -04:00
Abe Jellinek
cab0fa93e7 Extend Retrieve Metadata to support EPUBs (#57) 2023-08-07 16:07:55 -04:00
Abe Jellinek
ba1b1b0639 Add EPUB format to Zotero.MIME 2023-08-07 16:07:52 -04:00
Abe Jellinek
2639981dda
Block remote content when indexing HTML file (#3157) 2023-06-12 23:43:18 -04:00
Dan Stillman
3ba78e28bb Update connectorTypeSchemaData and test data for Dataset Number removal 2023-04-27 03:44:25 -04:00
Tom Najdek
a6042d3958
Mendeley importer: Fix issue with empty tags (#3018)
Also adds a test for this particular case and for importing tags in
general.
2023-04-06 17:10:13 +02:00
Tom Najdek
96022847d7
Mendeley importer: Fix issue with empty creators (#3016)
It does not appear to be possible to create a creator with no values in
Mendeley, however we got reports of these causing the imports to fail.
This tweak makes the importer more resilient by discarding empty/invalid
creators.
2023-04-06 17:10:13 +02:00
Tom Najdek
4b523555d6
Mendeley Import: Auth using direct login
* Importer will now ask user for a login and password via form and will
  perform sign-in directly using credentials rather than oauth
* Signing in this way enables importer to obtain desktop document ID
  which is now stored for each item
* It's possible to switch back to the old method (ouath) by setting
  `import.mendeleyUseOAuth` pref to `true`.
* New option to only import new items. This options only appears if
  database contains previously imported items.
* Importer will now update mendeleyDB:documentUUID on existing items to
  match value used in Mendeley Desktop if available
* Importer will no longer create collections when no new items are
  imported * Importer will only report number of new items imported on
  re-import * Importer will now preserve dateAdded on re-import

Co-authored-by: Dan Stillman <dstillman@zotero.org>
2023-04-06 17:10:12 +02:00
Dan Stillman
ab2e163234 Update test sample data for authority/legislativeBody base mapping 2023-04-03 00:41:35 -04:00
Dan Stillman
e27c1b5335 Add Dataset and Standard item types
zotero/zotero-bits#22
zotero/zotero-bits#52
2023-04-01 16:34:43 -04:00
Dan Stillman
9869eb4bc8 Restore cell.csl for style tests
Removed in a28908f0b4
2022-10-27 05:07:46 -04:00
Adomas Venčkauskas
a28908f0b4 Change integration test citation style to APA and some refactoring 2022-10-25 14:50:05 +03:00
Tom Najdek
223f44fdfd Map annotation colors colors on import #2819 (#2822) 2022-09-12 19:58:21 -04:00
Abe Jellinek
1f9e518581 Duplicates Merge: Preserve embedded annotations (#2728) 2022-08-11 03:52:40 -04:00
Tom Najdek
141258d564 Fix a bug in regex extracting fields to "extra"
Because regex is built using a template string, \s* is actually escaped
into s*, i.e. literal "s" appearing 0 or more times. In most cases this
would mean that output can have spacing slightly off. In extreme case,
when identifier starts with letter "s", this could this could lead to
identifier being stored incorrectly.

Also adjusted tests to be more strict and mock data to cover this case.
2022-08-11 02:46:37 -04:00
Dan Stillman
65318a442e Fix support tests after 28ed3e34b 2022-08-11 02:29:44 -04:00
Dan Stillman
cb2594f53f Feed import: Don't fail on OPML entry with no title or text
https://forums.zotero.org/discussion/96841/impossible-dimporter-ompl-rss
2022-08-11 02:28:03 -04:00
Dan Stillman
bbdcb92042 Add missing test data files 2022-06-21 01:39:07 -04:00
Dan Stillman
df3e7a600e Update Zotero.File.getContentsAsync() tests 2022-06-19 18:59:01 -04:00
Dan Stillman
6a2949be8a fx-compat: Add HiddenBrowser.jsm
Remove Zotero.Browser and add HiddenBrowser.jsm. Post-Fission, web/file
content loads in a separate process, so it's not possible (as best as I
can tell) to directly access the contents of a hidden browser -- it just
appears as about:blank in the parent process. We now use Mozilla's
JSWindowActor mechanism [1] to get page data, including character set
and body text for full-text indexing. We'll have to evaluate other uses
of hidden browsers to see how to handle them.

This also adds include.jsm for loading the Zotero object into a JSM.

[1] https://firefox-source-docs.mozilla.org/dom/ipc/jsactors.html
2022-06-17 20:28:58 -04:00
Abe Jellinek
9829ea7009
Update utilities, move tests, add to CI (#2584) 2022-04-30 04:55:11 -04:00
Tom Najdek
56321a7a6d
Fix regression: download() fails for non-http URLs (#2497)
This fixes compatiblity with some addons that use Zotero.File.download
to extract files from their XPI bundle.
2022-03-30 12:15:40 -04:00
Tom Najdek
776769f480
Citavi import: Tweak how page label is determined (#2494)
Instead of attempting to extract `PageRange` value we now let pdf worker
always determine page label.

Also improved citavi tests and fixtures.
2022-03-30 09:34:07 -04:00
Abe Jellinek
ef82becf00
Merge attachments and update notes (#2336)
We follow a different merge procedure for each attachment type:

- For PDF attachments, compare by MD5. If no match, get the top 50 words
  in the attachment's text and hash those, then check again for a match.
  Update references to item keys in notes and annotations.
- For web (snapshot / link) attachments, compare by title and URL.
  Prefer a title + URL match but accept a title-only match.
- For other attachment types, keep all attachments from all items being
  merged.

Also:

- Move most merge tests from Duplicates to Items#merge(). It just doesn't
  make sense to worry about the UI in these.
2022-03-09 17:26:26 -05:00
Tom Najdek
1ad2056674
Add support for importing Citavi annotatons (#2351) 2022-03-09 04:06:44 -05:00
Dan Stillman
1ce47bc404 Add Preprint item type to additional sample data for tests 2022-03-05 07:23:50 -05:00
Dan Stillman
32fc1cad9c Add Preprint item type to sample data for tests 2022-03-05 06:03:44 -05:00
Tom Najdek
092459dbfc
Mendeley Import: Tests for group annotations
Extended Mendeley Import test to include a scenario where other users
attached an annotation to an item in a group library that also exists
in user's library.
2021-11-15 11:39:57 +01:00