Add in path to all files in index directory to copy on Azure correctly.

[vsmalladi]

PR checklist

Closes #627

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the versions.yml file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware

[drpatelh]

Yo! Why doesn't the current behaviour of copying a directory work here? This would have been tested already on Azure?

[vsmalladi]

Never tested making the bwa index and saving them. I had just upload the index files already or just made them for the test.

[drpatelh]

Merging this in will need some thought because this will change the way we are dealing with indices in pipelines quite a bit and how they are defined by users. What error do you see when using the current behaviour on Azure?

[vsmalladi]

The error is that nextflow can't find the file to copy over.

[drpatelh]

But it should just be a directory no? This should then affect any directory defined as output on Azure?

[vsmalladi]

That would be my assumption, but i wonder if its just the way the directory contents are referred to. If you add directory/*
vs just directory. I think this will require some more investigation.

[adamrtalbot]

Does type: dir help?

[JakobBerg]

Looks good to me

[vsmalladi]

@adamrtalbot can you take a look?

[adamrtalbot]

I'm still a bit confused, what happens? I've been using BWA on Azure quite a bit and never seen this error, could you provide some steps to reproduce it?

[vsmalladi]

@adamrtalbot might not be an issue anymore. Maybe we can hold off until it becomes an issue again.

[SPPearce]

Closing this until someone flags it as an issue again.