Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DANS/Local PermaLink PID Provider #8674

Merged
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
1fcfe3b
remove unused code, make notes about other refactors
qqmyers Mar 18, 2022
44cd500
initial permaLink provider
qqmyers Mar 18, 2022
cf2c096
Merge remote-tracking branch 'IQSS/develop' into
qqmyers May 24, 2022
06b5acf
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers May 26, 2022
34361f1
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Jun 27, 2022
8396e1d
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Aug 9, 2022
458bb88
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Aug 12, 2022
7f9c0b4
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Aug 18, 2022
129aada
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Sep 16, 2022
073c9dd
remove unused method
qqmyers Sep 16, 2022
9285b94
cleanup
qqmyers Sep 16, 2022
77a2e6a
override getTargetUrl
qqmyers Sep 16, 2022
af7662d
remove unused method
qqmyers Sep 16, 2022
a97ea66
update comments
qqmyers Sep 16, 2022
50316b0
add permalink provider to CommandContext
qqmyers Sep 16, 2022
aca81a2
move id generation onto ID service beans.
qqmyers Sep 19, 2022
72d4b01
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Sep 19, 2022
0d8a425
Big shuffle of Global ID checks/param passing - compiles
qqmyers Sep 20, 2022
09f914c
minor cleanup, add ToDos re: multi pid providers
qqmyers Sep 21, 2022
71477e9
remove alomost duplicate generateIdentifierAsRandomString methods
qqmyers Sep 21, 2022
3d37c5d
remove near duplicate method for generating ids from storedProc, cleanup
qqmyers Sep 21, 2022
c5d2467
adjust alreadyExists
qqmyers Sep 22, 2022
18a7b49
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Sep 22, 2022
e1bea9b
Move GlobalId from String to helper than scans providers and test
qqmyers Sep 23, 2022
b90e38b
using method for all dvobjects now
qqmyers Sep 28, 2022
c47d846
add citation support for files with PIDs
qqmyers Sep 28, 2022
0b7757a
fix named query, add findByGlobalId call
qqmyers Sep 29, 2022
e594b81
refactor to add parsing per PidProvider
qqmyers Sep 29, 2022
775bc7e
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Oct 5, 2022
b2eec0e
switch to provider map, add cached info to GlobalId
qqmyers Oct 5, 2022
e209065
refactor, add params to constructor
qqmyers Oct 6, 2022
624fc66
remove old methods
qqmyers Oct 6, 2022
356cded
add create from protocol/authority/identifier
qqmyers Oct 6, 2022
4e2a60c
Update all calls to match new design
qqmyers Oct 6, 2022
a039a25
update tests, add PiDUtil stub for tests
qqmyers Oct 7, 2022
b0301e9
remove constants from interface
qqmyers Oct 7, 2022
6c2f3cb
change to recognize DOI/Handle by default
qqmyers Oct 7, 2022
c33fadc
more .getGlobalId().asX NPEs
qqmyers Oct 7, 2022
10d854e
remove commented method, fix deprecate new Longs
qqmyers Oct 7, 2022
94481ff
todo was fixed
qqmyers Oct 7, 2022
676c1bb
remove deprecated getGlobalIdAsString method.
qqmyers Oct 7, 2022
e1b28f6
cache transient globalId
qqmyers Oct 7, 2022
aee9fbe
update xhtml files for removal of getGlobalIdAsString
qqmyers Oct 7, 2022
2589637
cleanup, bug fixes
qqmyers Oct 9, 2022
9d0812b
update Perma alreadyExists and urlPrefix
qqmyers Oct 9, 2022
5f6c254
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Oct 13, 2022
f3fb3fd
add UnmanagedDOI bean
qqmyers Oct 13, 2022
2a82402
merge error - remove Xrecord
qqmyers Oct 14, 2022
716c045
add unmanaged provider classes, adapt to those
qqmyers Oct 14, 2022
093c36e
restore method for unit tests
qqmyers Oct 14, 2022
3c6435e
handle file no PID case
qqmyers Oct 14, 2022
8ced491
add fine logging
qqmyers Oct 14, 2022
232f9ad
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Nov 7, 2022
d63472a
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Dec 12, 2022
9d68f3e
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Dec 14, 2022
02b9f06
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Jan 20, 2023
73268fc
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Jan 30, 2023
16e4579
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Feb 14, 2023
8cf2dd4
Apply suggestions from code review
qqmyers Mar 3, 2023
f1f2559
Merge remote-tracking branch 'IQSS/develop' into DANS/local_pid_provider
qqmyers Mar 3, 2023
de7c1a3
review inspired changes
qqmyers Mar 3, 2023
a91e6fe
merge issue
qqmyers Mar 3, 2023
3eb87a5
use default scope
qqmyers Mar 3, 2023
ad13e67
fix comment typos
qqmyers Mar 8, 2023
5a8d401
use :Protocol to decide inf PermaLinks are in use
qqmyers Mar 8, 2023
2599e84
use perma.baseurlstring instead
qqmyers Mar 8, 2023
475433e
release note and docs
qqmyers Mar 8, 2023
3392042
logging to fine
qqmyers Mar 8, 2023
33f382d
URL parsing bug fix and test method
qqmyers Mar 8, 2023
7d183b6
expand tests
qqmyers Mar 8, 2023
01fa9c2
fix link
qqmyers Mar 8, 2023
6919900
typo
qqmyers Mar 8, 2023
afcb56a
fix link path
qqmyers Mar 8, 2023
a78b49a
another typo
qqmyers Mar 8, 2023
c5a1258
unused imports
qqmyers Mar 10, 2023
d531489
add fine logging
qqmyers Mar 10, 2023
7950b15
fixed using abstract class with @EJB
qqmyers Mar 10, 2023
baf0999
unused import
qqmyers Mar 10, 2023
78d0421
unused imports
qqmyers Mar 10, 2023
aff44f9
use fine logging
qqmyers Mar 13, 2023
2cb3a54
Check for null file PID
qqmyers Mar 15, 2023
bec22e6
fix dependent file pid generation logic
qqmyers Mar 16, 2023
2650e96
fix pidprovider lookups
qqmyers Mar 16, 2023
41bfbe2
tweak comment
qqmyers Mar 16, 2023
091dcc1
set isConfigured true for handles
qqmyers Mar 16, 2023
38b9d3e
PermaLinks doesn't use :DoiProvider
qqmyers Mar 17, 2023
17e226b
update to reference perma.baseurlstring
qqmyers Mar 17, 2023
8fcdfbb
can't call setting service in constructor
qqmyers Mar 17, 2023
34a25cd
refactor to add isConfigured and use it to provide clearer error
qqmyers Mar 17, 2023
c369c27
add retrieving the dataset protocol to cheapandeasy
qqmyers Mar 17, 2023
3ef94eb
handle reg in publicize for file pids
qqmyers Mar 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
287 changes: 254 additions & 33 deletions src/main/java/edu/harvard/iq/dataverse/AbstractGlobalIdServiceBean.java

Large diffs are not rendered by default.

15 changes: 10 additions & 5 deletions src/main/java/edu/harvard/iq/dataverse/CitationServlet.java
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
*/
package edu.harvard.iq.dataverse;

import edu.harvard.iq.dataverse.pidproviders.PidUtil;
import edu.harvard.iq.dataverse.util.StringUtil;
import java.io.IOException;
import java.io.PrintWriter;
Expand All @@ -21,7 +22,7 @@
public class CitationServlet extends HttpServlet {

@EJB
DatasetServiceBean datasetService;
DvObjectServiceBean dvObjectService;

/**
* Processes requests for both HTTP <code>GET</code> and <code>POST</code>
Expand All @@ -37,10 +38,14 @@ protected void processRequest(HttpServletRequest request, HttpServletResponse re

String persistentId = request.getParameter("persistentId");
if (persistentId != null) {
Dataset ds = datasetService.findByGlobalId(persistentId);
if (ds != null) {
response.sendRedirect("dataset.xhtml?persistentId=" + persistentId);
return;
DvObject dob = dvObjectService.findByGlobalId(PidUtil.parseAsGlobalID(persistentId));
if (dob != null) {
if (dob instanceof Dataset) {
response.sendRedirect("dataset.xhtml?persistentId=" + persistentId);
} else if (dob instanceof DataFile) {
response.sendRedirect("file.xhtml?persistentId=" + persistentId);
}
return;
}
}
response.sendError(HttpServletResponse.SC_NOT_FOUND);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ public class DOIDataCiteRegisterService {
DataverseServiceBean dataverseService;

@EJB
DOIDataCiteServiceBean doiDataCiteServiceBean;
AbstractGlobalIdServiceBean doiDataCiteServiceBean;


//A singleton since it, and the httpClient in it can be reused.
Expand Down
47 changes: 14 additions & 33 deletions src/main/java/edu/harvard/iq/dataverse/DOIDataCiteServiceBean.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,23 @@
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.annotation.PostConstruct;
import javax.ejb.EJB;
import javax.ejb.Stateless;

import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;

import edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key;


/**
*
* @author luopc
*/
@Stateless
public class DOIDataCiteServiceBean extends AbstractGlobalIdServiceBean {
public class DOIDataCiteServiceBean extends DOIServiceBean {

private static final Logger logger = Logger.getLogger(DOIDataCiteServiceBean.class.getCanonicalName());

Expand All @@ -34,22 +38,20 @@ public class DOIDataCiteServiceBean extends AbstractGlobalIdServiceBean {
@EJB
DOIDataCiteRegisterService doiDataCiteRegisterService;

public DOIDataCiteServiceBean() {
@PostConstruct
private void init() {
String doiProvider = settingsService.getValueForKey(Key.DoiProvider, "");
if("DataCite".equals(doiProvider)) {
isConfigured=true;
}
}

@Override
public boolean registerWhenPublished() {
return false;
}

@Override
public boolean alreadyExists(DvObject dvObject) {
if(dvObject==null) {
logger.severe("Null DvObject sent to alreadyExists().");
return false;
}
return alreadyExists(dvObject.getGlobalId());
}


@Override
public boolean alreadyExists(GlobalId pid) {
Expand Down Expand Up @@ -103,29 +105,6 @@ public HashMap getIdentifierMetadata(DvObject dvObject) {
}


/**
* Looks up the metadata for a Global Identifier
* @param protocol the identifier system, e.g. "doi"
* @param authority the namespace that the authority manages in the identifier system
* @param identifier the local identifier part
* @return a Map of metadata. It is empty when the lookup failed, e.g. when
* the identifier does not exist.
*/
@Override
public HashMap<String, String> lookupMetadataFromIdentifier(String protocol, String authority, String identifier) {
logger.log(Level.FINE,"lookupMetadataFromIdentifier");
String identifierOut = getIdentifierForLookup(protocol, authority, identifier);
HashMap<String, String> metadata = new HashMap<>();
try {
metadata = doiDataCiteRegisterService.getMetadata(identifierOut);
} catch (Exception e) {
logger.log(Level.WARNING, "None existing so we can use this identifier");
logger.log(Level.WARNING, "identifier: {0}", identifierOut);
}
return metadata;
}


/**
* Modifies the DOI metadata for a Dataset
* @param dvObject the dvObject whose metadata needs to be modified
Expand Down Expand Up @@ -277,5 +256,7 @@ public List<String> getProviderInformation(){
return providerInfo;
}

//PID recognition


}
50 changes: 16 additions & 34 deletions src/main/java/edu/harvard/iq/dataverse/DOIEZIdServiceBean.java
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
package edu.harvard.iq.dataverse;

import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
import edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key;
import edu.ucsb.nceas.ezid.EZIDException;
import edu.ucsb.nceas.ezid.EZIDService;
import edu.ucsb.nceas.ezid.EZIDServiceRequest;
import java.util.*;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.ejb.EJB;
import javax.ejb.Stateless;

/**
*
* @author skraffmiller
*/
@Stateless
public class DOIEZIdServiceBean extends AbstractGlobalIdServiceBean {
public class DOIEZIdServiceBean extends DOIServiceBean {

EZIDService ezidService;
EZIDServiceRequest ezidServiceRequest;
Expand All @@ -23,16 +27,20 @@ public class DOIEZIdServiceBean extends AbstractGlobalIdServiceBean {
// get username and password from system properties
private String USERNAME = "";
private String PASSWORD = "";

public DOIEZIdServiceBean() {
logger.log(Level.FINE,"Constructor");
baseURLString = System.getProperty("doi.baseurlstring");
ezidService = new EZIDService(baseURLString);
USERNAME = System.getProperty("doi.username");
PASSWORD = System.getProperty("doi.password");
logger.log(Level.FINE, "Using baseURLString {0}", baseURLString);
logger.log(Level.FINE, "Using baseURLString {0}", baseURLString);
try {
ezidService.login(USERNAME, PASSWORD);
String doiProvider = settingsService.getValueForKey(Key.DoiProvider, "");
if ("EZID".equals(doiProvider)) {
//Guessing these are SYstem.getProperty rather than using settingsService because this is a constructir rather than a @PostConstruct method
baseURLString = System.getProperty("doi.baseurlstring");
ezidService = new EZIDService(baseURLString);
USERNAME = System.getProperty("doi.username");
PASSWORD = System.getProperty("doi.password");
ezidService.login(USERNAME, PASSWORD);
isConfigured = true;
}
} catch (EZIDException e) {
logger.log(Level.WARNING, "login failed ");
logger.log(Level.WARNING, "Exception String: {0}", e.toString());
Expand Down Expand Up @@ -102,32 +110,6 @@ public Map<String, String> getIdentifierMetadata(DvObject dvObject) {
return metadata;
}

/**
* Looks up the metadata for a Global Identifier
*
* @param protocol the identifier system, e.g. "doi"
* @param authority the namespace that the authority manages in the
* identifier system
* identifier part
* @param identifier the local identifier part
* @return a Map of metadata. It is empty when the lookup failed, e.g. when
* the identifier does not exist.
*/
@Override
public HashMap<String, String> lookupMetadataFromIdentifier(String protocol, String authority, String identifier) {
logger.log(Level.FINE,"lookupMetadataFromIdentifier");
String identifierOut = getIdentifierForLookup(protocol, authority, identifier);
HashMap<String, String> metadata = new HashMap<>();
try {
metadata = ezidService.getMetadata(identifierOut);
} catch (EZIDException e) {
logger.log(Level.FINE, "None existing so we can use this identifier");
logger.log(Level.FINE, "identifier: {0}", identifierOut);
return metadata;
}
return metadata;
}

/**
* Modifies the EZID metadata for a Dataset
*
Expand Down
51 changes: 51 additions & 0 deletions src/main/java/edu/harvard/iq/dataverse/DOIServiceBean.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package edu.harvard.iq.dataverse;

public abstract class DOIServiceBean extends AbstractGlobalIdServiceBean {

public static final String DOI_PROTOCOL = "doi";
public static final String DOI_RESOLVER_URL = "https://doi.org/";
public static final String HTTP_DOI_RESOLVER_URL = "http://doi.org/";
public static final String DXDOI_RESOLVER_URL = "https://dx.doi.org/";
public static final String HTTP_DXDOI_RESOLVER_URL = "http://dx.doi.org/";
public DOIServiceBean() {
super();
}

@Override
public GlobalId parsePersistentId(String pidString) {
if (pidString.startsWith(DOI_RESOLVER_URL)) {
pidString = pidString.replace(DOI_RESOLVER_URL,
(DOI_PROTOCOL + ":"));
} else if (pidString.startsWith(HTTP_DOI_RESOLVER_URL)) {
pidString = pidString.replace(HTTP_DOI_RESOLVER_URL,
(DOI_PROTOCOL + ":"));
} else if (pidString.startsWith(DXDOI_RESOLVER_URL)) {
pidString = pidString.replace(DXDOI_RESOLVER_URL,
(DOI_PROTOCOL + ":"));
}
return super.parsePersistentId(pidString);
}

public GlobalId parsePersistentId(String protocol, String identifierString) {
if (!DOI_PROTOCOL.equals(protocol)) {
return null;
}
GlobalId globalId = super.parsePersistentId(protocol, identifierString);
if (globalId!=null && !GlobalIdServiceBean.checkDOIAuthority(globalId.getAuthority())) {
return null;
}
return globalId;
}

public GlobalId parsePersistentId(String protocol, String authority, String identifier) {
if (!DOI_PROTOCOL.equals(protocol)) {
return null;
}
return super.parsePersistentId(protocol, authority, identifier);
}

public String getUrlPrefix() {
return DOI_RESOLVER_URL;
}

}
17 changes: 6 additions & 11 deletions src/main/java/edu/harvard/iq/dataverse/DataCitation.java
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ public String toString(boolean html, boolean anonymized) {

if (persistentId != null) {
// always show url format
citationList.add(formatURL(persistentId.toURL().toString(), persistentId.toURL().toString(), html));
citationList.add(formatURL(persistentId.asURL(), persistentId.asURL(), html));
}
citationList.add(formatString(publisher, html));
citationList.add(version);
Expand Down Expand Up @@ -298,7 +298,7 @@ public void writeAsBibtexCitation(OutputStream os) throws IOException {
out.write(persistentId.getIdentifier());
out.write("},\r\n");
out.write("url = {");
out.write(persistentId.toURL().toString());
out.write(persistentId.asURL());
out.write("}\r\n");
out.write("}\r\n");
out.flush();
Expand Down Expand Up @@ -387,7 +387,7 @@ public void writeAsRISCitation(OutputStream os) throws IOException {

out.write("SE - " + date + "\r\n");

out.write("UR - " + persistentId.toURL().toString() + "\r\n");
out.write("UR - " + persistentId.asURL() + "\r\n");
out.write("PB - " + publisher + "\r\n");

// a DataFile citation also includes filename und UNF, if applicable:
Expand Down Expand Up @@ -584,7 +584,7 @@ private void createEndNoteXML(XMLStreamWriter xmlw) throws XMLStreamException {
xmlw.writeStartElement("urls");
xmlw.writeStartElement("related-urls");
xmlw.writeStartElement("url");
xmlw.writeCharacters(getPersistentId().toURL().toString());
xmlw.writeCharacters(getPersistentId().asURL());
xmlw.writeEndElement(); // url
xmlw.writeEndElement(); // related-urls
xmlw.writeEndElement(); // urls
Expand Down Expand Up @@ -781,18 +781,13 @@ private GlobalId getPIDFrom(DatasetVersion dsv, DvObject dv) {
|| HarvestingClient.HARVEST_STYLE_ICPSR.equals(dsv.getDataset().getHarvestedFrom().getHarvestStyle())
|| HarvestingClient.HARVEST_STYLE_DATAVERSE
.equals(dsv.getDataset().getHarvestedFrom().getHarvestStyle())) {
// creating a global id like this:
// persistentId = new GlobalId(dv.getGlobalId());
// you end up doing new GlobalId((New GlobalId(dv)).toString())
// - doing an extra formatting-and-parsing-again
// This achieves the same thing:
if(!isDirect()) {
if (!StringUtils.isEmpty(dsv.getDataset().getIdentifier())) {
return new GlobalId(dsv.getDataset());
return dsv.getDataset().getGlobalId();
}
} else {
if (!StringUtils.isEmpty(dv.getIdentifier())) {
return new GlobalId(dv);
return dv.getGlobalId();
}
}
}
Expand Down
6 changes: 5 additions & 1 deletion src/main/java/edu/harvard/iq/dataverse/DataFile.java
Original file line number Diff line number Diff line change
Expand Up @@ -956,7 +956,7 @@ public JsonObject asGsonObject(boolean prettyPrint){
// https://github.com/IQSS/dataverse/issues/761, https://github.com/IQSS/dataverse/issues/2110, https://github.com/IQSS/dataverse/issues/3191
//
datasetMap.put("title", thisFileMetadata.getDatasetVersion().getTitle());
datasetMap.put("persistentId", getOwner().getGlobalIdString());
datasetMap.put("persistentId", getOwner().getGlobalId().asString());
datasetMap.put("url", getOwner().getPersistentURL());
datasetMap.put("version", thisFileMetadata.getDatasetVersion().getSemanticVersion());
datasetMap.put("id", getOwner().getId());
Expand Down Expand Up @@ -1034,6 +1034,10 @@ public String getCreateDateFormattedYYYYMMDD() {
return null;
}

@Override
public String getTargetUrl() {
return DataFile.TARGET_URL;
}

} // end of class

Expand Down
Loading