Support multiple file versions

[mahesh-panchal]

Add support for multiple software version emissions and multiple software versions emitted as a result.

PR checklist

• This comment contains a description of changes (with reason)
• CHANGELOG.md is updated
• If you've fixed a bug or added code that should be tested, add tests!
• Documentation in docs is updated

[codecov]

Codecov Report

Merging #1247 (06cdece) into dev (92c1651) will not change coverage.
The diff coverage is n/a.

❗ Current head 06cdece differs from pull request most recent head 82ce4b4. Consider uploading reports for the commit 82ce4b4 to get more accurate results

@@           Coverage Diff           @@
##              dev    #1247   +/-   ##
=======================================
  Coverage   67.45%   67.45%           
=======================================
  Files          50       50           
  Lines        5607     5607           
=======================================
  Hits         3782     3782           
  Misses       1825     1825           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92c1651...82ce4b4. Read the comment docs.

[mahesh-panchal]

So now I have some questions.

1. Is it ok that the tsv file that goes into pipeline info is now a yml file (and by consequence removes the need for the python script)?
2. How are the MultiQC and FastQC processes included in the template? Since there are changes here to the module template and the pipeline template, how should I be handling this? I see shasums so I figure a manual change is not correct?
3. Is the MultiQC input fine?

[grst]

Working on a reference implementation in rnaseq to check if anything breaks: nf-core/rnaseq#689

I already learned that a yaml dict looks like

foo:
  key: value

rather than

foo:
   -key: value

[mahesh-panchal]

Anyone got any pointers to this trim_galore failing test? I don't know what I need to change for this.

[mahesh-panchal]

Still to do:

• Docs?
• Linting ( lint HEREDOC in module ) (this is done by the new tests in the modules repo)
• MQC version / Software Versions version inclusion?
• Additional tests?

[grst]

Linting ( lint HEREDOC in module )

We check on the modules side if versions.yml is generated + valid YAML, so maybe this isn't even required. In that case the old check for *.version.txt would need to be removed, though.

[edmundmiller]

Sorry for adding myself as a reviewer, I thought this was supporting multiple versions of modules. This all looks really clean to me though, feel free to ping me if you want someone to test it out!

[mahesh-panchal]

I was actually waiting for input from you @emiller88 since you're the linting expert. There was a question whether we needed to add extra linting, and to check I haven't removed anything essential from linting too.

[mahesh-panchal]

Anyone got any pointers to this trim_galore failing test? I don't know what I need to change for this.

This is still an issue too.

[mahesh-panchal]

In that case the old check for *.version.txt would need to be removed, though.

I think we've already done this. I've gone through the code again to make sure there are no more references or checks for *version[s]?.txt

[grst]

The following tests fail (expectedly):

• The trimgalore test fails because the functions.nf from the modules repository does not include all expected functions (we add one to the template in this PR) and because the functions.nf is outdated.
• the "RunTestWorkflow" fails because the pipeline-template still contains old modules (which produce a version.txt instead of a versions.yml which can't be handled by the new module to collect versions.

[drpatelh]

Great work guys! Glad to be finally merging this :)