Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds emit statement for FASTQs and metadata to SRA workflow #184

Merged
merged 3 commits into from
Aug 24, 2023
Merged

Adds emit statement for FASTQs and metadata to SRA workflow #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
81324d2
Adds emit statement for FASTQs and metadata to SRA workflow
adamrtalbot Aug 21, 2023
464397c
Add emit statement to synpase workflow
adamrtalbot Aug 24, 2023
97af5cd
Update CHANGELOG.md
adamrtalbot Aug 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Thank you to everyone else that has contributed by reporting bugs, enhancements
### Enhancements & fixes

- [#166](https://github.com/nf-core/fetchngs/issues/175) - Add NF-TEST pipeline end-to-end tests for existing CI tests
- [#184](https://github.com/nf-core/fetchngs/pull/184/files) Emits all files in workflow for testing and enabling workflow to be imported (not supported by Nextflow)

## [[1.10.0](https://github.com/nf-core/fetchngs/releases/tag/1.10.0)] - 2023-05-16

Expand Down
18 changes: 9 additions & 9 deletions modules/local/sra_to_samplesheet/tests/main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@
"sample_description": "RNA-Seq reads mapped onto L. Boulardi Toti-like virus genome",
"fastq_md5": "8d7d7b854d0207d1226477a30103fade;9fd57225d6c07a31843276d6df9b15c0;5a62e8f785687dce890cfb4fe3e607f9",
"fastq_ftp": "ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_1.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_2.fastq.gz",
"fastq_1": "ERX1188904_ERR1109373_1.fastq.gz:md5,9fd57225d6c07a31843276d6df9b15c0",
"fastq_2": "ERX1188904_ERR1109373_2.fastq.gz:md5,5a62e8f785687dce890cfb4fe3e607f9",
"fastq_1": "./results/fastq/ERX1188904_ERR1109373_1.fastq.gz",
"fastq_2": "./results/fastq/ERX1188904_ERR1109373_2.fastq.gz",
"md5_1": "9fd57225d6c07a31843276d6df9b15c0",
"md5_2": "5a62e8f785687dce890cfb4fe3e607f9",
"single_end": false
Expand All @@ -45,8 +45,8 @@
"sample_description": "RNA-Seq reads mapped onto L. Boulardi Toti-like virus genome",
"fastq_md5": "8d7d7b854d0207d1226477a30103fade;9fd57225d6c07a31843276d6df9b15c0;5a62e8f785687dce890cfb4fe3e607f9",
"fastq_ftp": "ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_1.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_2.fastq.gz",
"fastq_1": "ERX1188904_ERR1109373_1.fastq.gz:md5,9fd57225d6c07a31843276d6df9b15c0",
"fastq_2": "ERX1188904_ERR1109373_2.fastq.gz:md5,5a62e8f785687dce890cfb4fe3e607f9",
"fastq_1": "./results/fastq/ERX1188904_ERR1109373_1.fastq.gz",
"fastq_2": "./results/fastq/ERX1188904_ERR1109373_2.fastq.gz",
"md5_1": "9fd57225d6c07a31843276d6df9b15c0",
"md5_2": "5a62e8f785687dce890cfb4fe3e607f9",
"single_end": false
Expand All @@ -71,8 +71,8 @@
"sample_description": "RNA-Seq reads mapped onto L. Boulardi Toti-like virus genome",
"fastq_md5": "8d7d7b854d0207d1226477a30103fade;9fd57225d6c07a31843276d6df9b15c0;5a62e8f785687dce890cfb4fe3e607f9",
"fastq_ftp": "ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_1.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_2.fastq.gz",
"fastq_1": "ERX1188904_ERR1109373_1.fastq.gz:md5,9fd57225d6c07a31843276d6df9b15c0",
"fastq_2": "ERX1188904_ERR1109373_2.fastq.gz:md5,5a62e8f785687dce890cfb4fe3e607f9",
"fastq_1": "./results/fastq/ERX1188904_ERR1109373_1.fastq.gz",
"fastq_2": "./results/fastq/ERX1188904_ERR1109373_2.fastq.gz",
"md5_1": "9fd57225d6c07a31843276d6df9b15c0",
"md5_2": "5a62e8f785687dce890cfb4fe3e607f9",
"single_end": false
Expand All @@ -97,8 +97,8 @@
"sample_description": "RNA-Seq reads mapped onto L. Boulardi Toti-like virus genome",
"fastq_md5": "8d7d7b854d0207d1226477a30103fade;9fd57225d6c07a31843276d6df9b15c0;5a62e8f785687dce890cfb4fe3e607f9",
"fastq_ftp": "ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_1.fastq.gz;ftp.sra.ebi.ac.uk/vol1/fastq/ERR110/003/ERR1109373/ERR1109373_2.fastq.gz",
"fastq_1": "ERX1188904_ERR1109373_1.fastq.gz:md5,9fd57225d6c07a31843276d6df9b15c0",
"fastq_2": "ERX1188904_ERR1109373_2.fastq.gz:md5,5a62e8f785687dce890cfb4fe3e607f9",
"fastq_1": "./results/fastq/ERX1188904_ERR1109373_1.fastq.gz",
"fastq_2": "./results/fastq/ERX1188904_ERR1109373_2.fastq.gz",
"md5_1": "9fd57225d6c07a31843276d6df9b15c0",
"md5_2": "5a62e8f785687dce890cfb4fe3e607f9",
"single_end": false
Expand All @@ -108,6 +108,6 @@
]
}
],
"timestamp": "2023-08-17T10:55:52+0000"
"timestamp": "2023-08-23T09:38:15+0000"
}
}
20 changes: 16 additions & 4 deletions workflows/sra/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ workflow SRA {
.set { ch_sra_metadata }
ch_versions = ch_versions.mix(SRA_RUNINFO_TO_FTP.out.versions.first())

fastq_files = Channel.empty()

if (!params.skip_fastq_download) {

ch_sra_metadata
Expand Down Expand Up @@ -111,10 +113,13 @@ workflow SRA {
)
ch_versions = ch_versions.mix(FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS.out.versions.first())

SRA_FASTQ_FTP
.out
.fastq
.mix(FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS.out.reads)
// Isolate FASTQ channel which will be added to emit block
fastq_files = fastq_files.mix(
SRA_FASTQ_FTP.out.fastq,
FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS.out.reads
)

fastq_files
.map {
meta, fastq ->
def reads = fastq instanceof List ? fastq.flatten() : [ fastq ]
Expand Down Expand Up @@ -161,6 +166,13 @@ workflow SRA {
CUSTOM_DUMPSOFTWAREVERSIONS (
ch_versions.unique().collectFile(name: 'collated_versions.yml')
)

emit:
fastq = fastq_files
samplesheet = SRA_MERGE_SAMPLESHEET.out.samplesheet
mapping = SRA_MERGE_SAMPLESHEET.out.mappings
multiqc_config = MULTIQC_MAPPINGS_CONFIG.out.yml
versions = ch_versions.unique()
}

/*
Expand Down
5 changes: 5 additions & 0 deletions workflows/synapse/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,11 @@ workflow SYNAPSE {
CUSTOM_DUMPSOFTWAREVERSIONS (
ch_versions.unique().collectFile(name: 'collated_versions.yml')
)

emit:
fastq = ch_fastq
samplesheet = SYNAPSE_MERGE_SAMPLESHEET.out.samplesheet
versions = ch_versions.unique()
}

/*
Expand Down
16 changes: 16 additions & 0 deletions workflows/synapse/tests/main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"samplesheet": {
"content": [
"id_mappings.csv:md5,3e287081f1e9349921fa8c709a874565",
"multiqc_config.yml:md5,a97047e12b0ea3f46d285629e4053548",
"samplesheet.csv:md5,9b3c35e8af74e9af194baa2498aaf4d4"
],
"timestamp": "2023-08-24T14:05:07+0000"
},
"software_versions": {
"content": [
"software_versions.yml:md5,f37983a6b1ce2334fe62fa188b31ca51"
],
"timestamp": "2023-08-24T14:05:07+0000"
}
}