Bug: Problems with ARC validation - metadata pipelined failed - invalid combination of header

[SabrinaZander]

We are trying to validate an ARC for Invenio Publication.
But the publication metadata pipeline failed with the following error message:

$ echo "Creating publication metadata..."
Creating publication metadata...
$ arc-to-invenio -p .
Internal Error:
Invalid combination of header Parameter [plasmid backbone]` and cells `pJC1-venus-term-BS ()`
$ cat metadata.json || exit 1
cat: metadata.json: No such file or directory
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: metadata.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/XXX(Path deleted) 
ERROR: No files to upload                          
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1

I created a new ARC with only the study containing the Parameter [plasmid backbone] and the cells pJC1-venus-term-BS() and the pipeline fails as well, so I suspect it is the study metadata causing the problem.

If I delete the Parameter [plasmid backbone] building block and reactivate the pipeline the error is shifted to the next building block.

Internal Error:
Invalid combination of header Parameter [Backbone Restriction] and cells BamHI/BcuI () 

I also created the study metadata sheet from scratch using Swate Alpha, that didn't help either.

Only if I delete the entire metadata sheet the pipeline works.

The study has been anonymized so you can view it:

isa.study.xlsx

Maybe someone has an idea what the problem could be.

[omaus]

Hey @SabrinaZander! 🙋🏻‍♂️
I am currently investigating this and saw some things in the study file that might be causing the problem. First, I added "user-specific" to every cell of the TSR and TAN columns for every building block since they were missing.
Here is the new file for this:

isa.study_altered.xlsx

Could you now try it again? and let's have a look if it works now. Just replace the old study file with the new one (make sure the name is "isa.study.xlsx" again). 🙂
If it does not, there's also the input and output columns missing. Might address this afterwards, too.

[SabrinaZander]

Hey @omaus ,
thanks for investigating this issue. I tested the altered file and the error remains the same

Internal Error:
Invalid combination of header `Parameter [plasmid backbone]` and cells `pJC1-venus-term-BS ()`

I also changed the first and last column to input and output and the error remains. But I also have some other isa.study sheets without input and output. This is normally not a problem.

[omaus]

I now tried every test separately with the study file. It passed every test.

My last idea would be to fill the empty row at the bottom of the table. Unlikely, but maybe it helps.

[SabrinaZander]

@omaus I spent hours investigating. But in the end I can narrow down the problem to the Component building block.

Steps to reproduce:

1. Create new ARC
2. Fill out investigation metadata
3. Add Invenio validation
4. Create study and fill out metadata
5. Test pipelines, should pass
6. Add new metadata sheet to study and add component building block
7. Test pipeline, should succeed
8. Add a row to your component building block
9. Test pipeline, pipeline fails with metadata error
10. Change component to something else, e.g. characteristic
11. Pipeline succeeds

I will now try replacing all component building blocks in our ARC and hope the pipeline succeeds.
I hope there are no other building blocks that also have the issue.
But please look at the script what could be the problem.

[omaus]

Thank you for the repro steps. 👍🏻

The error does not stem from the validation package hence I could not replicate it with the Study file itself since it is not incorrect when tested against the Invenio validation package.
In fact, the problem lies in ARCtrl when the linked method is called by arc-to-invenio tool.

I will open issues in the respective repo and close this one.

[omaus]

Fully closed now due to nfdi4plants/arc-to-invenio#20.