Fix tests after modules repository restructuring

[JoseEspinosa]

After #2141 was merged tests failed in the following modules as nicely reported by @grst in modules slack
Please assign yourself by editing this issue e.g. - [ ] module/foo (@JoseEspinosa) and mark it as resolved when your PR fixing the test is merged.

• amps Fixed the amps test #3244
• amrfinderplus/run @rpetit3
• amrfinderplus/update @rpetit3
• antismash/antismashlitel (not failing, cf Update antismash-lite tests: pin the versions to allow retro-active testing #2687 and Update ANTISMASH to 6.1.1 and enable_conda replacement #2688 )
• antismash/antismashlitedownloaddatabasesl (not failing, cf Update antismash-lite tests: pin the versions to allow retro-active testing #2687 and Update ANTISMASH to 6.1.1 and enable_conda replacement #2688 )
• arriba (@JoseEspinosa)
• bamcmp Fixed the bamcmpmodule tests #3245
• bases2fastq (@emiller88 fix(bases2fastq): Remove stray ls #2074)
• bcftools/reheader update bcftools version #2497
• bcftools/sort update bcftools version #2497
• biscuit/epiread Bug fix: Ensure string trimming is suffix at end of filepath #2475
• checkm/qa (not failing, cf Update CheckM: Fix occasional error to due to input file names and directory clash  #2464)
• cnvkit/batch update samtools to 1.16.1 #2492
• controlfreec/assesssignificance Updated the MD5 checksums of all controlfreec tests #3248
• controlfreec/freec Add type/pattern #2980
• controlfreec/freec2bed Updated the MD5 checksums of all controlfreec tests #3248
• controlfreec/freec2circos Updated the MD5 checksums of all controlfreec tests #3248
• controlfreec/makegraph Updated the MD5 checksums of all controlfreec tests #3248
• cooler/cload @pawelciurkaardigen (Fix subset of tests #2219)
• cooler/zoomify [COOLER] new functions - balance / makebins #1937
• custom/sratoolsncbisettings nf-test custom/sratoolsncbisettings #3964
• deeparg/downloaddata tracked in Deeparg moved to GitHub + Zenodo #4321
• deeparg/predict tracked in Deeparg moved to GitHub + Zenodo #4321
• deepvariant bump deepvariant version #2494
• delly/call add VCF output/input to delly/call #2909 and Added the MD5 checksums for the delly/call tests #3246
• endorspy Updating endorspy #3134
• entrezdirect/esearch entrezdirect/esearch test fix #3243
• ffq Fixed the ffq tests #3978
• flye Fixed the flye stub commands #3977
• freebayes bump freebayes version #2493 Cannot use MD5 checksums because the files contain the date of the day #3242
• gatk4/applybqsr @pawelciurkaardigen (Fix subset of tests #2219)
• gatk4/genotypegvcfs @pawelciurkaardigen
• gatk4/mutect2 @pawelciurkaardigen
• gtdbtk/classifywf #16665
• kallistobustools/ref (not failing)
• leehom @chriswyatt1 (need PR review)
• mafft (Seems to work fine- @chriswyatt1, confirmed by @muffato )
• maltextract @chriswyatt1 (need PR review)
• maxquant/lfq (not failing)
• metaphlan3/mergemetaphlantables Wrong module name in metaphlan3 test #3976
• motus/downloaddb Update mOTUs modules version #2631
• motus/merge Update mOTUs modules version #2631 (Conda failure dismissed Update mOTUs modules version #2631 (comment) )
• motus/profile Update mOTUs modules version #2631 (Conda failure dismissed Update mOTUs modules version #2631 (comment) )
• nanoplot nanoplot takes ages to install on conda, but is okay on mamba #2610
• ncbigenomedownload @chriswyatt1
• pairtools/dedup Pairtools version bump (+ improved tests) #3247
• pairtools/flip Pairtools version bump (+ improved tests) #3247
• pairtools/restrict Pairtools version bump (+ improved tests) #3247
• picard/sortvcf @pawelciurkaardigen
• pirate @rpetit3
• plasmidid This MD5 differs between Conda and Singularity #3746
• plink2/extract - tracked in plink2/extract test is broken #3965
• rtgtools/vcfeval (not failing, cf Update rtgtools/vcfeval #2734)
• shovill @rpetit3
• sistr @rpetit3
• stranger @pawelciurkaardigen (singularity and docker pass, for conda there is a problem creating conda environment - PackagesNotFound)
• strelka/somatic (not failing)
• tbprofiler/profile @rpetit3
• ultra/pipeline Update uLTRA to version 0.0.4.2 #2479
• unicycler - tracked in Broken Conda unicycler test #3982
• yara/mapper (Update yara index and yara mapper #2164)
• ariba/run @rpetit3
• busco @rpetit3
• dragonflye @rpetit3
• mobsuite/recon @rpetit3
• panaroo/run @rpetit3
• prokka @rpetit3
• roary @rpetit3
• snippy/core @rpetit3
• snippy/run @rpetit3

[rpetit3]

No Fixes needed (Tests (TMPDIR=~ nf-core modules test) passed with conda, docker, singularty)

• ariba/run
• busco
• dragonflye
• mobsuite/recon
• panaroo/run
• prokka
• roary
• sistr
• snippy/core
• snippy/run
• gatk4/genotypegvcfs
• gatk4/mutect2

[amasplund]

We have looked at bcftools/sort and it looks like the md5sum of the zipped output vcf-file is different if you use a conda or docker profile. If you unzip the files the md5sum is the same. The docker one is the same as what is stated in the tests/modules/nf-core/bcftools/sort/test.yml file. Seems like a problem that should probably have appeared before!? Is there a standard solution? Remove the md5sum check of the zipped file??

[pawelciurkaardigen]

@amasplund I had similar problem for gatk4/applybqsr and validating cram file's md5sum but for me it was depending if I run the test locally or in CI. My solution was, as you suggested, relaxing the assertion by removing md5sum which is validating only file's presence. I don't know however if there is a better way that preserves file's content verification.

[muffato]

I found quite a few that have been fixed in the past few weeks

[muffato]

(I've assigned this ticket to myself because I'll try to go through those issues one by one, but please don't hold me responsible for everything 😅 !)

[muffato]

Finally getting to the bottom of this !

1. I've forked two issues for broken tests: plink2/extract test is broken #3965 and Broken Conda unicycler test #3982
2. The deeparg/downloaddata test is failing because the underlying deeparg version, 1.0.2, needs data from a website that is gone. There is no command-line parameter to change the URL and there is no other version. I've sent an email to the author about it.
3. The deeparg/predict test fails because it first runs deeparg/downloaddata

My goal is to close this ticket once I've got an answer from the deeparg author.

[muffato]

Deeparg issue tracked in #4321