Duplicate property key in Bundle.properties file V5.12.1

[stevenferey]

Hi Dataverse Community,

We detected a duplicate keys in the Bundle.properties file :

advanced.search.datasets.persistentId.tip=The persistent identifier for the Dataset.
advanced.search.datasets.persistentId.tip=The Dataset's unique persistent identifier, either a DOI or Handle

best regards
Steven Ferey.

[pdurbin]

@stevenferey thanks!

Yes, confirmed in the develop branch as of 26afd10

$ ack advanced.search.datasets.persistentId.tip src
src/main/java/propertyFiles/Bundle.properties
935:advanced.search.datasets.persistentId.tip=The persistent identifier for the Dataset.
937:advanced.search.datasets.persistentId.tip=The Dataset's unique persistent identifier, either a DOI or Handle

Interestingly, both lines are from the same commit (98acdca):

$ cd src/main/java/propertyFiles
$ git blame Bundle.properties | grep advanced.search.datasets.persistentId.tip
98acdca83a6 src/main/java/propertyFiles/Bundle.properties (Julian Gautier      2022-05-17 16:36:32 -0400  935) advanced.search.datasets.persistentId.tip=The persistent identifier for the Dataset.
98acdca83a6 src/main/java/propertyFiles/Bundle.properties (Julian Gautier      2022-05-17 16:36:32 -0400  937) advanced.search.datasets.persistentId.tip=The Dataset's unique persistent identifier, either a DOI or Handle

[qqmyers]

FWIW

dataverse/src/main/java/propertyFiles/Bundle.properties

Lines 2222 to 2225 in 26afd10

	ingest.csv.invalidHeader=Invalid header row. One of the cells is empty.
	ingest.csv.lineMismatch=Mismatch between line counts in first and final passes!, {0} found on first pass, but {1} found on second.
	ingest.csv.recordMismatch=Reading mismatch, line {0} of the Data file: {1} delimited values expected, {2} found.
	ingest.csv.nullStream=Stream can't be null.

are duplicates as well

[jggautier]

Looks like in the pull request for improving the tooltips in the Citation metadata block, I left both lines there? Sorry about that.

About the duplicate advanced.search.datasets.persistentId.tip lines, the second one on line 937 was the one we decided to use, but since later changes to Dataverse facilitate the use of PIDs other than DOIs and Handles, I think we should avoid mentioning specific PIDs, and something like the first one on line 935 would be better.

Should I create a PR so that only one advanced.search.datasets.persistentId.tip exists?

I don't understand how those ingest.csv... lines are mismatches, but I'm happy to help with that, too if you could write more about it @qqmyers. Or if I create a PR, could you fix that duplicate issue?

[qqmyers]

They are copies of

dataverse/src/main/java/propertyFiles/Bundle.properties

Lines 2116 to 2119 in 26afd10

	ingest.csv.invalidHeader=Invalid header row. One of the cells is empty.
	ingest.csv.lineMismatch=Mismatch between line counts in first and final passes!, {0} found on first pass, but {1} found on second.
	ingest.csv.recordMismatch=Reading mismatch, line {0} of the Data file: {1} delimited values expected, {2} found.
	ingest.csv.nullStream=Stream can't be null.

and I think you can just delete whichever set you like.

[jggautier]

Ah, thanks! I'll wait for this to be prioritized before creating a PR that removes from the Bundle.properties file:

• one of the "advanced.search.datasets.persistentId.tip" entries
• either of the duplicate sets of lines in @qqmyers's two comments

[poikilotherm]

Should we add an action that runs a duplicate detection check on the properties files when they are changed?

[poikilotherm]

I played with a little shell script (on Linux - no idea if this works on non-GNU MacOS, but it would in a Github Action):

for PF in $(find . -wholename '*/src/*.properties'); do
    FILTER=$(grep -a -v -E "^(#.*|\s*$)" "$PF" | cut -d"=" -f1 | sort | uniq -c | tr -s " " | grep -v "^ 1 ")
    if [ -n "$FILTER" ]; then
        echo "$PF"
        echo "$FILTER"
        echo ""
    fi
done

And it shows we have more problems with duplicates:

./src/main/java/propertyFiles/biomedical.properties
 2 controlledvocabulary.studyAssayMeasurementType.transcription_profiling

./src/main/java/propertyFiles/Bundle.properties
 2 advanced.search.datasets.persistentId.tip
 2 advanced.search.files.persistentId.tip
 2 authenticationProvider.name.builtin
 2 authenticationProvider.name.github
 2 authenticationProvider.name.google
 2 authenticationProvider.name.null
 2 authenticationProvider.name.orcid
 2 authenticationProvider.name.orcid-sandbox
 2 authenticationProvider.name.shib
 2 citationFrame.banner.countdownMessage.seconds
 2 ingest.csv.invalidHeader
 2 ingest.csv.lineMismatch
 2 ingest.csv.nullStream
 2 ingest.csv.recordMismatch

./src/main/java/propertyFiles/customGSD.properties
 2 controlledvocabulary.gsdCourseName.06337:_changing_natural_and_built_coastal_environments
 2 controlledvocabulary.gsdFacultyName.menges,_achim

It would be very easy to put this into an Action which runs on PRs that change properties files:

FAIL=0

for PF in $(find . -wholename '*/src/*.properties'); do
    FILTER=$(grep -a -v -E "^(#.*|\s*$)" "$PF" | cut -d"=" -f1 | sort | uniq -c | tr -s " " | grep -v "^ 1 ")
    if [ -n "$FILTER" ]; then
        echo "$PF"
        echo "$FILTER"
        echo ""
        FAIL=1
    fi
done

if [ "$FAIL" -eq 1 ]; then
    echo "Duplicate keys found, please check output."
    exit 1
fi

[poikilotherm]

Just because I was feeling like it, I went ahead and created #9176

Some fun leisure time hacking!

[stevenferey]

Hello,

Can we follow the result of the execution of the git workflow in order to propose a PR and remove key duplications?

Thank you so much,
Steven.

[pdurbin]

@stevenferey I may be misunderstanding but if you go to https://github.com/IQSS/dataverse/pull/9176/files you can see the reports of the duplicates (at the bottom). (Good work, @poikilotherm!) Here's a screenshot:

[jggautier]

Just noting a comment I updated last week about one of the duplicate keys:

About the duplicate advanced.search.datasets.persistentId.tip lines, the second one on line 937 was the one we decided to use, but since later changes to Dataverse facilitate the use of PIDs other than DOIs and Handles, I think we should avoid mentioning specific PIDs, and something like the first one on line 935 would be better.

It looks like the PR keeps the other key instead, the one with the value "The Dataset's unique persistent identifier, either a DOI or Handle".

I think this is okay for now. I've been hoping to continue improving and applying the metadata text guide, and the tooltip for persistentId field can be reviewed and improved then.

[stevenferey]

Thanks for your feedback,

I modified this key to insert the first proposal.

Steven.

[jggautier]

Ah okay. Thanks. So on the advanced search page, the tooltip for the persistent ID field label will be "The persistent identifier for the Dataset." This will be different than what's in the tooltip for the persistent ID field label on the dataset page, right?

Here's a screenshot of that tooltip on the dataset page (after clicking the Metadata tab):

[stevenferey]

Yes it is. I also propose to modify this label to align the two keys:

• dataset.metadata.persistentId.tip=The persistent identifier for the Dataset.

I note that this formulation is also present for the keys:

• dataset.metadata.alternativePersistentId.tip=A previously used persistent identifier for the Dataset, either a DOI or Handle

• advanced.search.files.persistentId.tip=The unique persistent identifier for a data file, which can be a Handle or DOI in Dataverse.

• file.metadata.persistentId.tip=The unique persistent identifier for a file, which can be a Handle or DOI in Dataverse.

do you want me to change these keys?
Steven.

[jggautier]

I'm not sure if this should be changed in the PR for this issue. It feels like scope creep to me, if that makes sense.

Do you know if your PR is for the next Dataverse release, 5.14?

I'm wondering if there would be time to create a new issue to make sure these tooltips are consistent and create a new PR for that issue.

[jggautier]

I was talking @scolapasta and it sounds like this PR was not planned to be in the next Dataverse release (v5.14).

We're thinking it would be better to leave the tooltip for the persistent ID field on the advanced search page as it is in 5.13, so it reads "The Dataset's unique persistent identifier, either a DOI or Handle" and leave the other tooltips that you found so far as they are. Then @scolapasta wrote that this PR could go into QA, although it may not be included in 5.14.

And then I'd open a separate GitHub issue about improving these tooltips and include the ones you listed. That way we can maintain the scope of this issue and PR and make sure we're not missing anything else when changing those other tooltips.

Does that sound okay? If so, could you change dataset.metadata.persistentId.tip back to "The Dataset's unique persistent identifier, either a DOI or Handle"?

[stevenferey]

Thank you for your reply,

Absolutely, so I modify the key to display the initial value.

Steven.

[jggautier]

Awesome! And thanks for spotting those other tooltips!