Update fastqc to produce multi-version versions.yml

[grst]

PR checklist

Closes #XXX

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the <SOFTWARE>.version.txt file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd
  • PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd
  • PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd

Exemplary PR to discuss changes of nf-core/tools#1247
to nf-core/modules.

Once this PR is reviewed, we'll need to update all modules (can be automated
mostly).

[drpatelh]

Looks awesome @grst @mahesh-panchal ! Couple of minor comments.

[drpatelh]

Will also need to update the linting code and other sections of the nf-core modules command that was built around using the old code. Ping @ErikDanielsson. It would be really nice if we can somehow sanity check the versions dumped to this YAML file, especially whilst reviewing PRs to make sure the correct commands are used. Not entirely sure how to do this though 🤔 @emiller88 would it be possible to upload the output files from pytest workflow as a comment on the PR? I know we have discussed this in the past and things get tricky whilst using all 3 of Singularity / Docker / Conda but thought I would ask anyway :)

[grst]

It would be really nice if we can somehow sanity check the versions dumped to this YAML file

yes, checking the yaml for validity would be great! Also, it should not contain END_VERSIONS (one space at the wrong position will lead to the delimiter ending up within the yaml file)

[grst]

Will also need to update the linting code and other sections of the nf-core modules command that was built around using the old code. Ping @ErikDanielsson. It would be really nice if we can somehow sanity check the versions dumped to this YAML file, especially whilst reviewing PRs to make sure the correct commands are used. Not entirely sure how to do this though thinking @emiller88 would it be possible to upload the output files from pytest workflow as a comment on the PR? I know we have discussed this in the past and things get tricky whilst using all 3 of Singularity / Docker / Conda but thought I would ask anyway :)

I had a quick look at the commenting feature, and it seems like it's difficult when the PR comes from a fork (which applies to basically all of them). Which of these issues do we need to address now? For the others we could create a separate issue and follow up later.

[mahesh-panchal]

I'm not understanding something here. What's the purpose of tests/test_versions_yml.py ?

[grst]

I'm not understanding something here. What's the purpose of tests/test_versions_yml.py ?

It parses the versions.yml created by the module and checks that it's valid YAML.

[mahesh-panchal]

I'm not asking the right question then.. This is a test to be run by what? I'm not familiar with all the mechanics here of testing.

Ah, wait this is meant to be run by the pytest workflow thing? And the versions.yml should be part of the output tested in test.yml as part of a standard modules test?

[grst]

Ah, wait this is meant to be run by the pytest workflow thing?

Exactely. This is what is called a custom test in the pytest-workflow documentation.

And the versions.yml should be part of the output tested in test.yml as part of a standard modules test?

So far, we don't test for version.txt at all (because it is not a published file, so there's nothing to test). Here I propose to publish the file iff. we know we are running on the CI.

Testing the checksum alone would not be helpful, as the checksum would be automatically generated by the tools command which doesn't help finding out if it actually is a valid versions.yml.

[mahesh-panchal]

Perhaps an alternative is to check an exclude list rather than explicitly refer to a file ( if file in params.modules.excludeFileList ). The exclude list could then be a default parameter in the DSL2 template, but left empty in the test workflows. The only complication is then for people who decide to use nf-core modules in their own workflow need to understand this is a thing too.

[grst]

Now, let's hope the CI works with the new #tools version

[drpatelh]

🤩