Merge pull request #154 from Joon-Klaps/docs-visual-legends

Update docs for more streamlined docs & figures

.github/workflows/build-docs.yml

@@ -40,16 +40,11 @@ jobs:
       - name: Install Nextflow
         uses: nf-core/setup-nextflow@v2
       - name: Build parameter docs
-        run: nf-core schema docs --format markdown --columns parameter,description,default --output docs/parameters.md --force
-      - name: Read parameter tip
-        id: read_tip
+        run: nf-core pipelines schema docs --format markdown --columns parameter,description,default --output docs/parameters.md --force
+      - name: Prepend template to parameters.md
         run: |
-          tip=$(cat docs/parameter_tip.md)
-          echo "tip_content=$tip" >> $GITHUB_ENV
-      - name: Append content to parameters.md
-        run: |
-          modified_content=$(awk '/^#/ { print; print "${{ env.tip_content }}"; exit } 1' docs/parameters.md)
-          echo "$modified_content" > docs/parameters.md
+          cat docs/template/parameter_tip.md docs/parameters.md > tmp.md
+          mv tmp.md docs/parameters.md
       - name: Build docs
         run: |
           if [[ "${{ github.ref }}" == "refs/heads/dev" ]]; then

.nf-core.yml

@@ -32,6 +32,7 @@ lint:
     - assets/nf-core-viralgenie_logo_light.png
     - conf/test.config
     - conf/test_full.config
+    - docs/README.md
   actions_awsfulltest: false
   actions_ci: false
 template:

CHANGELOG.md

@@ -21,6 +21,7 @@ Initial release of Joon-Klaps/viralgenie, created with the [nf-core](https://nf-
 - Add new module to inculde custom mpileup-vcf file for intra-host analyses ([#151](https://github.com/Joon-Klaps/viralgenie/pull/151))
 - Update docs ([#150](https://github.com/Joon-Klaps/viralgenie/pull/150))
 - Make custom-mpileup.py postion 1 index based and not 0 index to follow bcftools ([#153](https://github.com/Joon-Klaps/viralgenie/pull/153))
+- Update docs for more streamlined docs & figures ([#154](https://github.com/Joon-Klaps/viralgenie/pull/154))
 
 ### `Fixed`
 

README.md

@@ -59,7 +59,7 @@
 10. [Optional] Remove clusters with low read coverage. `bin/extract_clusters.py`
 11. Scaffolding of contigs to centroid ([`Minimap2`](https://github.com/lh3/minimap2), [`iVar-consensus`](https://andersen-lab.github.io/ivar/html/manualpage.html))
 12. [Optional] Annotate 0-depth regions with external reference `bin/lowcov_to_reference.py`.
-13. [Optional] Select best reference from `--mapping_constrains`:
+13. [Optional] Select best reference from `--mapping_constraints`:
     - [`Mash sketch`](https://github.com/marbl/Mash)
     - [`Mash screen`](https://github.com/marbl/Mash)
 14. Mapping filtered reads to supercontig and mapping constrains([`BowTie2`](http://bowtie-bio.sourceforge.net/bowtie2/),[`BWAmem2`](https://github.com/bwa-mem2/bwa-mem2) and [`BWA`](https://github.com/lh3/bwa))

assets/schemas/mapping_constrains.json → assets/schemas/mapping_constraints.json

@@ -1,8 +1,8 @@
 {
     "$schema": "https://json-schema.org/draft/2020-12/schema",
-    "$id": "https://raw.githubusercontent.com/Joon-Klaps/viralgenie/dev/assets/schemas/mapping_constrains.json",
-    "title": "Joon-Klaps/viralgenie pipeline - params.mapping_constrains schema",
-    "description": "Schema for the file provided with params.mapping_constrains",
+    "$id": "https://raw.githubusercontent.com/Joon-Klaps/viralgenie/dev/assets/schemas/mapping_constraints.json",
+    "title": "Joon-Klaps/viralgenie pipeline - params.mapping_constraints schema",
+    "description": "Schema for the file provided with params.mapping_constraints",
     "type": "array",
     "items": {
         "type": "object",

bin/custom_multiqc.py

@@ -101,7 +101,7 @@ def file_choices(choices, fname):
     )
 
     parser.add_argument(
-        "--mapping_constrains",
+        "--mapping_constraints",
         metavar="MAPPING CONSTRAINS",
         help="Mapping constrains file containing information on the sequences that need to be used for mapping against the samples, supported formats: '.csv', '.tsv', '.yaml', '.yml'",
         type=lambda s: file_choices(("csv", "tsv", "yaml", "yml"), s),
@@ -121,14 +121,6 @@ def file_choices(choices, fname):
         help="Checkv summary files for each sample",
         type=Path,
     )
-    parser.add_argument(
-        "--filter_level",
-        metavar="FILTER LEVEL",
-        choices=["normal", "strict", "none"],
-        default="normal",
-        type=str,
-        help="Specify how strict the filtering should be, default is normal.",
-    )
 
     parser.add_argument(
         "--clusters_files",

bin/utils/module_data_processing.py

@@ -275,7 +275,7 @@ def reformat_constrain_df(df, file_columns, args):
         return df, df
 
     # Add constrain metadata to the mapping constrain table
-    constrain_meta = filelist_to_df([args.mapping_constrains])
+    constrain_meta = filelist_to_df([args.mapping_constraints])
 
     # drop unwanted columns & reorder
     constrain_meta = drop_columns(constrain_meta, ["sequence", "samples"])
@@ -295,12 +295,12 @@ def reformat_constrain_df(df, file_columns, args):
 
     # add mapping summary to sample overview table in ... wide format with species & segment combination
     logger.info("Creating mapping constrain summary (wide) table")
-    mapping_constrains_summary = create_constrain_summary(df, file_columns).set_index("sample")
+    mapping_constraints_summary = create_constrain_summary(df, file_columns).set_index("sample")
 
     logger.info("Coalescing columns")
     coalesced_constrains = coalesce_constrain(df)
     coalesced_constrains = drop_columns(coalesced_constrains, ["id", "selection", "rank"])
-    return coalesced_constrains, mapping_constrains_summary
+    return coalesced_constrains, mapping_constraints_summary
 
 
 def generate_ignore_samples(dataframe: pd.DataFrame) -> pd.Series:

conf/modules.config

@@ -372,7 +372,7 @@ process {
             ext.args   =
                 [
                     params.spades_mode ? "--${params.spades_mode}" : '' //,
-                    // params.mapping_constrains ? "--trusted-contigs ${params.mapping_constrains}" : ''
+                    // params.mapping_constraints ? "--trusted-contigs ${params.mapping_constraints}" : ''
                 ].join(' ').trim()
             publishDir = [
                 [
@@ -1668,7 +1668,6 @@ process {
     withName: CUSTOM_MULTIQC {
         ext.args   = [
                 params.save_intermediate_polishing ? "--save_intermediate" : '',
-                params.contig_filter_level ? "--filter_level ${params.contig_filter_level}" : '',
             ].join(' ').trim()
         publishDir = [
             [

conf/tests/test.config

@@ -39,7 +39,7 @@ params {
     kaiju_db                    = "https://kaiju-idx.s3.eu-central-1.amazonaws.com/2023/kaiju_db_viruses_2023-05-26.tgz"
 
     reference_pool              = "https://github.com/Joon-Klaps/nextclade_data/raw/old_datasets/data/nextstrain/sars-cov-2/MN908947/sequences.fasta"
-    mapping_constrains          = "${projectDir}/assets/samplesheets/mapping_constrains.csv"
+    mapping_constraints          = "${projectDir}/assets/samplesheets/mapping_constraints.csv"
 
     save_intermediate_polishing = true
     intermediate_mapping_stats  = true

conf/tests/test_fail_mapped.config

@@ -34,7 +34,7 @@ params {
     skip_read_classification  = true
     kaiju_db                    = "https://kaiju-idx.s3.eu-central-1.amazonaws.com/2023/kaiju_db_viruses_2023-05-26.tgz"
     reference_pool              = "https://github.com/Joon-Klaps/nextclade_data/raw/old_datasets/data/nextstrain/sars-cov-2/MN908947/sequences.fasta"
-    mapping_constrains           = "${projectDir}/assets/samplesheets/mapping_constrains_fail.tsv"
+    mapping_constraints           = "${projectDir}/assets/samplesheets/mapping_constraints_fail.tsv"
 
     min_mapped_reads            = 100
     intermediate_mapping_stats  = true

conf/tests/test_full.config

@@ -40,7 +40,7 @@ params {
     kaiju_db                    = "https://kaiju-idx.s3.eu-central-1.amazonaws.com/2023/kaiju_db_viruses_2023-05-26.tgz"
     reference_pool              = "https://github.com/Joon-Klaps/nextclade_data/raw/old_datasets/data/nextstrain/sars-cov-2/MN908947/sequences.fasta"
 
-    mapping_constrains          = "${projectDir}/assets/samplesheets/mapping_constrains.csv"
+    mapping_constraints          = "${projectDir}/assets/samplesheets/mapping_constraints.csv"
     checkv_db                   = "https://github.com/nf-core/test-datasets/raw/phageannotator/modules/nfcore/checkv/endtoend/checkv_minimal_db.tar"
 
     save_intermediate_polishing = true

conf/tests/test_umi.config

@@ -42,7 +42,7 @@ params {
     kaiju_db                    = "https://kaiju-idx.s3.eu-central-1.amazonaws.com/2023/kaiju_db_viruses_2023-05-26.tgz"
 
     reference_pool              = "https://github.com/Joon-Klaps/nextclade_data/raw/old_datasets/data/nextstrain/sars-cov-2/MN908947/sequences.fasta"
-    mapping_constrains           = "${projectDir}/assets/samplesheets/mapping_constrains.csv"
+    mapping_constraints           = "${projectDir}/assets/samplesheets/mapping_constraints.csv"
 
     save_intermediate_polishing = true
     save_intermediate_reads     = true

docs/CITATIONS.md

@@ -1 +1,5 @@
+---
+hide:
+  - navigation
+---
 --8<-- "CITATIONS.md:"

docs/CONTRIBUTING.md

@@ -1,3 +1,7 @@
+---
+hide:
+  - navigation
+---
 # Contributing Guidelines
 
 Hi there!

docs/customisation/configuration.md

@@ -1,8 +1,8 @@
 # Custom configuration of modules
 
-Within viralgenie, all modules (tools, eg. `FASTP`, `FASTQC`) can be ran with specific arguments. The pipeline has a default configuration but this can be overwritten by supplying a custom configuration file. This file can be provided to viralgenie using the `-c` Nextflow option.
+Within viralgenie, all modules (tools, e.g., `FASTP`, `FASTQC`) can be run with specific arguments. The pipeline has a default configuration but this can be overwritten by supplying a custom configuration file. This file can be provided to viralgenie using the `-c` Nextflow option.
 
-To see which specific arguments or variables are used for a module or tool, have a look at the [`modules.config` file](https://github.com/Joon-Klaps/viralgenie/blob/dev/conf/modules.config). Here the arguments of a module is specified as followed:
+To see which specific arguments or variables are used for a module or tool, have a look at the [`modules.config` file](https://github.com/Joon-Klaps/viralgenie/blob/dev/conf/modules.config). Here the arguments of a module are specified as follows:
 
 ```groovy hl_lines="3-6 9-13"
 withName: IVAR_CONSENSUS {
@@ -26,25 +26,25 @@ withName: IVAR_CONSENSUS {
 In this example, the `IVAR_CONSENSUS` module is configured with the arguments `-q 20 -m 10` for the tool [`ivar consensus`](https://andersen-lab.github.io/ivar/html/manualpage.html#autotoc_md19) and `--ignore-overlaps --count-orphans --max-depth 0 --no-BAQ --min-BQ 0` for [`samtools mpileup`](https://www.htslib.org/doc/samtools-mpileup.html) as iVar uses the output of `samtools mpileup` directly.
 
 !!! Tip
-    The `ext.args` and `ext.args2` are used to specify the arguments for the tool. If unsure which tools uses which arguments (`ivar:ext.args`and `samtools:ext.args2`), have a look at the nextflow module file directly! For example, at [`modules/nf-core/ivar/consensus.nf`](https://github.com/Joon-Klaps/viralgenie/blob/dev/modules/nf-core/ivar/consensus/main.nf), "$args" and "$args2" are used to specify the arguments for the tools:
+    The `ext.args` and `ext.args2` are used to specify the arguments for the tool. If unsure which tools use which arguments (`ivar:ext.args` and `samtools:ext.args2`), have a look at the nextflow module file directly! For example, at [`modules/nf-core/ivar/consensus.nf`](https://github.com/Joon-Klaps/viralgenie/blob/dev/modules/nf-core/ivar/consensus/main.nf), "$args" and "$args2" are used to specify the arguments for the tools:
     ```groovy hl_lines="5 10"
 
     """
     samtools \\
         mpileup \\
         --reference $fasta \\
-        $args2 \\               # can be modified with ext.args2
+        $args2 \\               // can be modified with ext.args2
         $bam \\
         $mpileup \\
         | ivar \\
             consensus \\
-            $args \\            # can be modified with ext.args
+            $args \\            // can be modified with ext.args
             -p $prefix
     ...
     """
     ```
 
-In case we do want to modify the arguments of a module, we can do so by providing a custom configuration file, the easiest way to do this would to then just copy a segment from the modules.config and modify the arguments. This way, none of the other configuration will get lost or modified. For example, setting the minimum depth to call consensus to 5 and the minimum quality score of base to 30 for the `IVAR_CONSENSUS` module:
+In case we do want to modify the arguments of a module, we can do so by providing a custom configuration file. The easiest way to do this would be to copy a segment from the modules.config and modify the arguments. This way, none of the other configurations will get lost or modified. For example, setting the minimum depth to call consensus to 5 and the minimum quality score of base to 30 for the `IVAR_CONSENSUS` module:
 ```groovy title='custom.config' hl_lines="5-6 12"
 process {
     withName: IVAR_CONSENSUS {
@@ -78,4 +78,4 @@ nextflow run Joon-Klaps/viralgenie \
 ```
 
 !!! Tip
-    This guide not entirely clear? Also have a look at the [nf-core guide for customising tool arguments](https://nf-co.re/docs/usage/configuration#customising-tool-arguments).
+    This guide not entirely clear? Also have a look at the [nf-core guide for customizing tool arguments](https://nf-co.re/docs/usage/configuration#customising-tool-arguments).