Add TwoRavens generated variable-level DDI to Dataverse API

[amberleahey]

For the Dataverse DDI Explorer add-on to solely rely on the DDI exported from Dataverse (without the need to load an additional file), the DDI requires the node to contain frequency and statistical information.
Including the frequency info simply requires the /codeBook/dataDscr/var/catgry/catStat nodes to be added. This frequency information was present in earlier editions of Dataverse (as can be seen from an older file migrated to our new dataverse instance https://dataverse.scholarsportal.info/api/datasets/export?exporter=ddi&persistentId=hdl%3A10864/10799), so it shouldn’t be too difficult to add it back.
The statistic information (including "vald","invd","max","min","mean","medn","mode" and "stdev" values) requires sumstat nodes with the following xml path /codeBook/dataDscr/var/sumStat.

If you could include this request, it would be greatly appreciated.

[djbrooke]

Thanks @amberleahey! Nice talking with you earlier today.

I'm going to tag @scolapasta in case there are additional technical details that are at top of mind after the call.

[djbrooke]

Hey @amberleahey - we talked about this in our backlog grooming meeting today. We discussed a few different approaches to exposing the statistical information. @landreev - you mentioned you were going to add some details and ask a few questions.

[amberleahey]

Hi folks, @lubitchv and I are revisting where this frequency and statistical information actually gets generated in DV and it would be good to understand a bit more about why and how we can get this stored in the DDI XML upon upload / ingest in DV rather than relying on seperate calls to R / Two Ravens generated files...again just getting up to speed on this, seems like it would be helpful for both Data Explorer and new Data Curation Tool to have this stored in DDI XML from the get go. Will investigate more and provide separate issue if we need to.

[pdurbin]

Related (I think):

• 8778 harvester ddi exporter #8786

Also, can we remove Two Ravens from the title? We haven't used it for years. I'm not sure how it's related.

---

Update:

v1 of Data Explore mentions the prep file: https://github.com/scholarsportal/Dataverse-Data-Explorer/blob/v2.1/assets/js/controllers/chart.js#L50

v2 doesn't mention prep but I'm not sure if it's used or not. I'm guessing @lubitchv knows.

[cmbz]

2024/09/09: @pdurbin do you recommend closing this issue since we no longer use Two Ravens? Should we open a different issue?

[pdurbin]

Yeah, I'll go ahead and close it. @amberleahey and @nana-boateng just gave a presentation of the new Data Explorer last week and they didn't mention needing this, so many years later. If you do need it, please let us know!