Problem: fatal error when SIP is a BagIt bag

[djjuhasz]

If the SIP delivered to Enduro is a BagIt bag the pre-processing workflow fails with a fatal error at the "CreateBagActivity":

CreateBagActivity: create bag: mkdir /home/preprocessing/shared/enduro3594443426/extract3168784964/ZippedBag/data: file exists

To Reproduce

Steps to reproduce the behavior:

1. Run Enduro with preprocessing-base enabled for preprocessing
2. Upload a zipped bag to the Enduro MinIO "sips" bucket
3. The above error occurs

Expected behavior

In real world implementations of preprocessing the SIP delivered by Enduro will be modified before being bagged and sent back to Enduro in which case the bag will need to be updated or its contents "unbagged" to prevent errors validating the bag payload against its manifest.

If the "CreateBagActivity" receives a BagIt bag as input, it should return the path of the bag, without altering the bag. Preprocessing will then deliver the unaltered bag to Enduro for further processing.

Additional context

See artefactual-sdps/enduro#805 for more information about the exchange of bags between preprocessing and Enduro.

[jraddaoui]

I guess making the CreateBagActivity noop when it receives a Bag won't hurt, but I was thinking to address this issue just with documentation. I see this template repository as an example never to be run as part of a real workflow.

[djjuhasz]

I tested https://github.com/LibraryOfCongress/bagit-python to see what it does when asked to bag a bag, and it just "double bags" the contents -- everything in the original bag (including manifests, metadata files, and the data directory) are put in a "data" directory and then it generates new manifests for everything. I don't know that I want to implement the same behaviour for the CreateBagActivity, but I think we should do better than the current error.

Another option is just to return a better error message like 'create bag: /path/to/dir is already a bag" or something similar.

[mcantelon]

I took the route of doing a rudimentary check to see if a directory already appears to be a Bag and, if so, return it's source path as the Bag path.

PR for CR: artefactual-sdps/temporal-activities#43

[mcantelon]

New PR to update the preprocessing-base repo to use the updated activity:

#20

[mcantelon]

PR merged.

[mcantelon]

I've merged a PR that updates the preprocessing-demo repo with this update as well (so when the demo test site is updated QA can be done).