Add an importer for Citavi backup files

[lishangyu9]

Fixes #8322 .
This importer for Citavi support ‘ctv5bak’ and ‘ctv6bak’ files.

• Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Screenshots added in PR description (for UI changes)
• Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
• Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

What we have added

• The Citavi file format support

  Add the definition of Citavi file in the “StandardFileType.java”.

• The XSD file defined the Citavi XML file

  Add the file “scitavi.xsd” for generating the java class “CitaviExchangeData.java”.

• The importer for Citavi file “ctv5bak” and “ctv6bak”

  • Add the class “src/main/java/org/jabref/logic/importer/fileformat/CitaviXmlImporter.java”.
    Because the “ctv6bak” and “ctv5bak” are compressed files. We add the function “getReaderFromZip” to unzip the file before parsing.
  • The XML file inside the “ctv6bak” is UTF-8 BOM format. This format cannot be recognized by the default process of XML. Therefore We create a function “checkForUtf8BOMAndDiscardIfAny” to delete the byte at the beginning of the XML file.
  • The process of parsing XML data is similar to “EndnoteImporter”. However, the date structures of Endnote and Citavi are different. The primary different methods are “getAuthorName”, “getEditorName”, “getKeywords” and “getPublisher”. These data are located in different places with reference data. For example, we need to go through the author’s matching list and match the references ID. If the id is matched we got the author’s ID list. Finally, we can get the author’s name.

• The test of Citavi importer

  The test class of Citavi importer is “CitaviXmlImporterTestFiles.java”.
  There are 4 test cases:
  “CitaviXmlImporterTest1.ctv6bak”, “CitaviXmlImporterTest1.bib”
  “CitaviXmlImporterTest2.ctv6bak”, “CitaviXmlImporterTest2.bib”
  “CitaviXmlImporterTest3.ctv6bak”, “CitaviXmlImporterTest3.bib”
  “CitaviXmlImporterTest4.ctv6bak”, “CitaviXmlImporterTest4.bib”

How to use the Citavi Importer

1. Click the “File” → ”Import” → ”Import into new library”.
   
2. Select the file. Click “Open”.
   
3. The file is imported
   

[Siedlerchr]

Thank you very much for your work! Impressive 👍 I took a more in-depth look at the author handling code, that's really ugly but I couldn't find a general better approach. However, you do not have to iterate over the whole String, the ids are all UUIDs that have a fixed length of 36 characters. So you can improve your loops.

Second thing: I implemented an example of using JabRef's Author and AuthorLIst class for adding new authors. You just need to adjust your test bib files (Lastname, Firstname), can be done in JabRef's GUI: (Quality-> Cleanup -> Normalize in persons) and maybe the same for other persons fields.

[Siedlerchr]

@lishangyu9 I came up with a more efficient solution for the authors and editors.
In essential, the oneToN is similar to a relational database schema where the first part as you already identified is the referenced and the other ones are the person ids.

Instead of iterating through the whole author + person list for every reference, I create a mapping between referenceId and Persons. In addition, I introduced a second Map for storing Persons we already know (if the same author with the same uuid comes up multiple times, so we don't have to through all Person objects everytime.

Antoher thing we can use for our advantage: We know that all UUIDs have a length of 36 chars and we know that the first only the first UUID is the referenceID. So we can safely assume that the rest of the string are the Person ids. So we can use split to get an array of all person ids if we just split at the semicolon, after we removed the refId from it.

This can be adapted for other oneToN relationships as well

[lishangyu9]

Thank you @Siedlerchr . We will check and fix it today.

[lishangyu9]

Hi @Siedlerchr, we fixed other's OneToN relationships, "Keywords" and "Publishers". We followed the method you have made and created the "buildKeywordList" and "buildPublisherList" functions for mapping the data and reference Id. We appreciate your help.

[Siedlerchr]

Just some minor changes, but overall looks very good to me!

[Siedlerchr]

@claell You were using citavi previously, it would be nice if you could testt this PR.
(best is using github cli tools, gh pr checkout 8848 and then ./gradlew run should assemble & run jabref

[claell]

@Siedlerchr when I find the time, I'll try that. I don't have GitHub's cli tools installed. Is there another easy way or should I just use them? Also, I likely cannot do too in depth testing due to constrained time.

[calixtus]

gh cli is supposed to be the easy way... you can of course also follow the "command line instructions" on the bottom of this page

[calixtus]

[koppor]

In general, it is OK. Some nitpicks, but we can fix them for ourselves later.

The thing I see here, that there was strong test-driven development somehow:

buildPersonList, buildKeywordList, buildPublisherList should all have been package private and tested by CitaviXmlImporter.

However, I think, this project is over. Think about using JUnit and the play / debug button also for methods themselves the next java project.

[calixtus]

I just mentioned you are mixing the old io and the new nio file apis. Is there a chance to switch completely to the new nio API? Provides thread safety i.a....

[Siedlerchr]

I just mentioned you are mixing the old io and the new nio file apis. Is there a chance to switch completely to the new nio API? Provides thread safety i.a....

@calixtus Ehm they are meant to be interoperable, for example Files.newInputStream returns an InputStream. And Inputstream is just the base class in java.io. There is not all in nio.

[Siedlerchr]

Thanks again for your valuable contribution and your quick follow up! 👍
We just noticed by chance that the pages field contained unicode minus signs instead of the normal ascii ones. So I created a follow up PR which calls the pages formatter