Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure methyldackel process runs on each sample. #140

Merged

Conversation

robsyme
Copy link
Contributor

@robsyme robsyme commented Jan 28, 2020

Before this patch, the methyldackel process would receive inputs from the channels:

  • ch_bam_dedup_for_methyldackel (containing many bams)
  • ch_bam_index_for_methyldackel (containing many bais)
  • ch_fasta_for_methyldackel (containing only one fasta)
  • ch_fasta_index_for_methyldackel (containing only one fai)

The process would only run once as the ch_fasta_for_methyldackel and ch_fasta_index_for_methyldackel channels would be quickly exhausted.

This patch uses the combine operator to ensure that every bam+bai pair runs through the methyldackel process. I've noticed that this style of nextflow process coordination is not used very often in nf-core, so there is a chance the PR violates nf-core style guidelines. There were no indications of that from nf-core lint, but appreciate that the PR style might not be appropriate.

Many thanks to contributing to nf-core/methylseq!

Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs).

PR checklist

  • This comment contains a description of changes (with reason)
  • If you've fixed a bug or added code that should be tested, add tests!
  • If necessary, also make a PR on the nf-core/methylseq branch on the nf-core/test-datasets repo
  • Ensure the test suite passes (nextflow run . -profile test,docker).
  • Make sure your code lints (nf-core lint .).
  • Documentation in docs is updated
  • CHANGELOG.md is updated
  • README.md is updated

Learn more about contributing: https://github.com/nf-core/methylseq/tree/master/.github/CONTRIBUTING.md

Before this patch, the methyldackel process would receive inputs from the channels:
- ch_bam_dedup_for_methyldackel (containing many bams)
- ch_bam_index_for_methyldackel (containing many bais)
- ch_fasta_for_methyldackel (containing only one fasta)
- ch_fasta_index_for_methyldackel (containing only one fai)

The process would only run once as the ch_fasta_for_methyldackel and ch_fasta_index_for_methyldackel channels would be quickly exhausted.

This patch uses the combine operator to ensure that every bam+bai pair runs through the methyldackel process. I've noticed that this style of nextflow process coordination is not used very often in nf-core, so there is a chance the PR violates nf-core style guidelines. There were no indications of that from nf-core lint, but appreciate that the PR style might not be appropriate.
@robsyme
Copy link
Contributor Author

robsyme commented Feb 5, 2020

It looks like the Travis CI failures are due to network errors rather than breaking changes introduced by the commits in this PR. From the logs, there are plenty of errors from nextflow attempting to fetch resources:

CAPSULE: Transfer failed: capsule.org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact com.fasterxml.jackson.core:jackson-databind:jar:2.6.7.2 from/to central (https://repo1.maven.org/maven2/): Forbidden (403) (for stack trace, run with -Dcapsule.log=verbose)

There are also failures where Nextflow attempts to pull in input datasets from the web:

No such file: https://github.com/nf-core/test-datasets/raw/methylseq/testdata/SRR389222_sub3.fastq.gz

@ewels ewels requested a review from phue April 7, 2020 14:36
@ewels ewels mentioned this pull request Apr 7, 2020
@ewels
Copy link
Member

ewels commented Apr 7, 2020

Hi @robsyme,

This is brilliant stuff - good catch, and thanks for fixing!

Would you mind pulling in the upstream changes (we just merged a big PR) and then adding something to the changelog about this please?

Cheers,

Phil

@ewels
Copy link
Member

ewels commented Apr 7, 2020

ps. Apologies for the delay in getting back to you about this. This pipeline has been in need of a bit of love for a few months now. Hopefully going to get stuff updated and a new release out now though 👍

@ewels ewels changed the base branch from dev to methyldackel-multi-patch April 7, 2020 16:39
@ewels
Copy link
Member

ewels commented Apr 7, 2020

Given that we left this waiting for quite a while and now would like a fast change for a release, I've just switched this to a dedicated branch and will make my minor changelog edits there 👍

@ewels ewels merged commit 005088c into nf-core:methyldackel-multi-patch Apr 7, 2020
@ewels ewels mentioned this pull request Apr 7, 2020
@robsyme
Copy link
Contributor Author

robsyme commented Apr 7, 2020

Thanks Phil. I would have been happy to pull in the upstream changes, but I missed your first emails and it looks like you've got it covered. Anyway, happy to have helped in this very small way.

@ewels
Copy link
Member

ewels commented Apr 8, 2020

Ah no worries, I didn’t give you very long to respond! As it was such a small change I figured it would only take me a few second to do..

Thank you again for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants