Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge template version 3.1.1 #445

Merged
merged 3 commits into from
Jan 10, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -34,3 +34,7 @@ trim_trailing_whitespace = unset
# ignore python and markdown
[*.{py,md}]
indent_style = unset

# ignore ro-crate metadata files
[**/ro-crate-metadata.json]
insert_final_newline = unset
12 changes: 6 additions & 6 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# nf-core/chipseq: Contributing Guidelines
# `nf-core/chipseq`: Contributing Guidelines

Hi there!
Many thanks for taking an interest in improving nf-core/chipseq.
@@ -55,23 +55,23 @@ These tests are run both with the latest available version of `Nextflow` and als

:warning: Only in the unlikely and regretful event of a release happening with a bug.

- On your own fork, make a new branch `patch` based on `upstream/master`.
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- A PR should be made on `master` from patch to directly this particular bug.
- Open a pull-request from `patch` to `main`/`master` with the changes.

## Getting help

For further information/help, please consult the [nf-core/chipseq documentation](https://nf-co.re/chipseq/usage) and don't hesitate to get in touch on the nf-core Slack [#chipseq](https://nfcore.slack.com/channels/chipseq) channel ([join our Slack here](https://nf-co.re/join/slack)).

## Pipeline contribution conventions

To make the nf-core/chipseq code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
To make the `nf-core/chipseq` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.

### Adding a new step

If you wish to contribute a new step, please use the following coding standards:

1. Define the corresponding input channel into your new process from the expected previous process channel
1. Define the corresponding input channel into your new process from the expected previous process channel.
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new parameters to `nextflow.config` with a default (see below).
@@ -84,7 +84,7 @@ If you wish to contribute a new step, please use the following coding standards:

### Default values

Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`.

Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.

1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -9,7 +9,6 @@ body:
- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
- [nf-core/chipseq pipeline documentation](https://nf-co.re/chipseq/usage)
- type: textarea
id: description
attributes:
21 changes: 17 additions & 4 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: nf-core AWS full size tests
# This workflow is triggered on PRs opened against the master branch.
# This workflow is triggered on PRs opened against the main/master branch.
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
# It runs the -profile 'test_full' on AWS batch

on:
pull_request:
branches:
- main
- master
workflow_dispatch:
pull_request_review:
@@ -21,18 +22,30 @@ jobs:
matrix:
aligner: ["bwa", "bowtie2", "chromap", "star"]
steps:
- uses: octokit/[email protected]
- name: Get PR reviews
uses: octokit/[email protected]
if: github.event_name != 'workflow_dispatch'
id: check_approvals
continue-on-error: true
with:
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews?per_page=100
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- id: test_variables

- name: Check for approvals
if: ${{ failure() && github.event_name != 'workflow_dispatch' }}
run: |
echo "No review approvals found. At least 2 approvals are required to run this action automatically."
exit 1
- name: Check for enough approvals (>=2)
id: test_variables
if: github.event_name != 'workflow_dispatch'
run: |
JSON_RESPONSE='${{ steps.check_approvals.outputs.data }}'
CURRENT_APPROVALS_COUNT=$(echo $JSON_RESPONSE | jq -c '[.[] | select(.state | contains("APPROVED")) ] | length')
test $CURRENT_APPROVALS_COUNT -ge 2 || exit 1 # At least 2 approvals are required
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
with:
18 changes: 10 additions & 8 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
# This workflow is triggered on PRs to `main`/`master` branch on the repository
# It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev`
on:
pull_request_target:
branches: [master]
branches:
- main
- master

jobs:
test:
runs-on: ubuntu-latest
steps:
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
- name: Check PRs
if: github.repository == 'nf-core/chipseq'
run: |
@@ -22,7 +24,7 @@ jobs:
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
@@ -32,9 +34,9 @@ jobs:
Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) ${{github.event.pull_request.base.ref}} branch.
The ${{github.event.pull_request.base.ref}} branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to ${{github.event.pull_request.base.ref}} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -45,7 +45,7 @@ jobs:
profile: "singularity"
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- name: Set up Nextflow
uses: nf-core/setup-nextflow@v2
49 changes: 30 additions & 19 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@ name: Test successful pipeline download with 'nf-core pipelines download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - when a PR is opened or reopened to main/master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
@@ -17,25 +17,31 @@ on:
- edited
- synchronize
branches:
- main
- master
pull_request_target:
branches:
- main
- master

env:
NXF_ANSI_LOG: false

jobs:
download:
configure:
runs-on: ubuntu-latest
outputs:
REPO_LOWERCASE: ${{ steps.get_repo_properties.outputs.REPO_LOWERCASE }}
REPOTITLE_LOWERCASE: ${{ steps.get_repo_properties.outputs.REPOTITLE_LOWERCASE }}
REPO_BRANCH: ${{ steps.get_repo_properties.outputs.REPO_BRANCH }}
steps:
- name: Install Nextflow
uses: nf-core/setup-nextflow@v2

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: "3.12"
architecture: "x64"
@@ -51,65 +57,70 @@ jobs:
pip install git+https://github.com/nf-core/tools.git@dev
- name: Get the repository name and current branch set as environment variable
id: get_repo_properties
run: |
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> ${GITHUB_ENV}
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> ${GITHUB_ENV}
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> "$GITHUB_OUTPUT"
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> "$GITHUB_OUTPUT"
- name: Make a cache directory for the container images
run: |
mkdir -p ./singularity_container_images
download:
runs-on: ubuntu-latest
needs: configure
steps:
- name: Download the pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
run: |
nf-core pipelines download ${{ env.REPO_LOWERCASE }} \
--revision ${{ env.REPO_BRANCH }} \
--outdir ./${{ env.REPOTITLE_LOWERCASE }} \
nf-core pipelines download ${{ needs.configure.outputs.REPO_LOWERCASE }} \
--revision ${{ needs.configure.outputs.REPO_BRANCH }} \
--outdir ./${{ needs.configure.outputs.REPOTITLE_LOWERCASE }} \
--compress "none" \
--container-system 'singularity' \
--container-library "quay.io" -l "docker.io" -l "community.wave.seqera.io" \
--container-library "quay.io" -l "docker.io" -l "community.wave.seqera.io/library/" \
--container-cache-utilisation 'amend' \
--download-configuration 'yes'
- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}
run: tree ./${{ needs.configure.outputs.REPOTITLE_LOWERCASE }}

- name: Count the downloaded number of container images
id: count_initial
run: |
image_count=$(ls -1 ./singularity_container_images | wc -l | xargs)
echo "Initial container image count: $image_count"
echo "IMAGE_COUNT_INITIAL=$image_count" >> ${GITHUB_ENV}
echo "IMAGE_COUNT_INITIAL=$image_count" >> "$GITHUB_OUTPUT"
- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
run: nextflow run ./${{needs.configure.outputs.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ needs.configure.outputs.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
- name: Run the downloaded pipeline (stub run not supported)
id: run_pipeline
if: ${{ job.steps.stub_run_pipeline.status == failure() }}
if: ${{ steps.stub_run_pipeline.outcome == 'failure' }}
env:
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -profile test,singularity --outdir ./results
run: nextflow run ./${{ needs.configure.outputs.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ needs.configure.outputs.REPO_BRANCH }}) -profile test,singularity --outdir ./results

- name: Count the downloaded number of container images
id: count_afterwards
run: |
image_count=$(ls -1 ./singularity_container_images | wc -l | xargs)
echo "Post-pipeline run container image count: $image_count"
echo "IMAGE_COUNT_AFTER=$image_count" >> ${GITHUB_ENV}
echo "IMAGE_COUNT_AFTER=$image_count" >> "$GITHUB_OUTPUT"
- name: Compare container image counts
run: |
if [ "${{ env.IMAGE_COUNT_INITIAL }}" -ne "${{ env.IMAGE_COUNT_AFTER }}" ]; then
initial_count=${{ env.IMAGE_COUNT_INITIAL }}
final_count=${{ env.IMAGE_COUNT_AFTER }}
if [ "${{ steps.count_initial.outputs.IMAGE_COUNT_INITIAL }}" -ne "${{ steps.count_afterwards.outputs.IMAGE_COUNT_AFTER }}" ]; then
initial_count=${{ steps.count_initial.outputs.IMAGE_COUNT_INITIAL }}
final_count=${{ steps.count_afterwards.outputs.IMAGE_COUNT_AFTER }}
difference=$((final_count - initial_count))
echo "$difference additional container images were \n downloaded at runtime . The pipeline has no support for offline runs!"
tree ./singularity_container_images
4 changes: 2 additions & 2 deletions .github/workflows/fix-linting.yml
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest
steps:
# Use the @nf-core-bot token to check out so we can push later
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
with:
token: ${{ secrets.nf_core_bot_auth_token }}

@@ -32,7 +32,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.nf_core_bot_auth_token }}

# Install and run pre-commit
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: "3.12"

10 changes: 5 additions & 5 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -14,10 +14,10 @@ jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- name: Set up Python 3.12
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: "3.12"

@@ -31,12 +31,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v2

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: "3.12"
architecture: "x64"
@@ -74,7 +74,7 @@ jobs:

- name: Upload linting log file artifact
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4
with:
name: linting-logs
path: |
2 changes: 1 addition & 1 deletion .github/workflows/linting_comment.yml
Original file line number Diff line number Diff line change
@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download lint results
uses: dawidd6/action-download-artifact@bf251b5aa9c2f7eeb574a96ee720e24f801b7c11 # v6
uses: dawidd6/action-download-artifact@80620a5d27ce0ae443b965134db88467fc607b43 # v7
with:
workflow: linting.yml
workflow_conclusion: completed
2 changes: 1 addition & 1 deletion .github/workflows/release-announcements.yml
Original file line number Diff line number Diff line change
@@ -31,7 +31,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: "3.10"
- name: Install dependencies
2 changes: 1 addition & 1 deletion .github/workflows/template_version_comment.yml
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
with:
ref: ${{ github.event.pull_request.head.sha }}

11 changes: 2 additions & 9 deletions .gitpod.yml
Original file line number Diff line number Diff line change
@@ -6,12 +6,5 @@ tasks:
nextflow self-update
vscode:
extensions: # based on nf-core.nf-core-extensionpack
#- esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code
- EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files
- Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar
- mechatroner.rainbow-csv # Highlight columns in csv files in different colors
- nextflow.nextflow # Nextflow syntax highlighting
- oderwat.indent-rainbow # Highlight indentation level
- streetsidesoftware.code-spell-checker # Spelling checker for source code
- charliermarsh.ruff # Code linter Ruff
extensions:
- nf-core.nf-core-extensionpack # https://github.com/nf-core/vscode-extensionpack
6 changes: 1 addition & 5 deletions .nf-core.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
bump_version: null
lint:
nextflow_config:
- config_defaults:
- params.bamtools_filter_se_config
- params.bamtools_filter_pe_config
nf_core_version: 3.0.2
org_path: null
nf_core_version: 3.1.1
repository_type: pipeline
template:
author: Espinosa-Carrasco J, Patel H, Wang C, Ewels P
@@ -15,6 +13,4 @@ template:
name: chipseq
org: nf-core
outdir: .
skip_features: null
version: 2.2.0dev
update: null
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -10,3 +10,4 @@ testing/
testing*
*.pyc
bin/
ro-crate-metadata.json
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"markdown.styles": ["public/vscode_markdown.css"]
}
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) Espinosa-Carrasco J, Patel H, Wang C, Ewels P
Copyright (c) The nf-core/chipseq team

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -3,9 +3,9 @@
<source media="(prefers-color-scheme: dark)" srcset="docs/images/nf-core-chipseq_logo_dark.png">
<img alt="nf-core/chipseq" src="docs/images/nf-core-chipseq_logo_light.png">
</picture>
</h1>
[![GitHub Actions CI Status](https://github.com/nf-core/chipseq/workflows/nf-core%20CI/badge.svg)](https://github.com/nf-core/chipseq/actions?query=workflow%3A%22nf-core+CI%22)
[![GitHub Actions Linting Status](https://github.com/nf-core/chipseq/workflows/nf-core%20linting/badge.svg)](https://github.com/nf-core/chipseq/actions?query=workflow%3A%22nf-core+linting%22)[![AWS CI](https://img.shields.io/badge/CI%20tests-full%20size-FF9900?labelColor=000000&logo=Amazon%20AWS)](https://nf-co.re/chipseq/results)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.3240506-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.3240506)
</h1>[![GitHub Actions CI Status](https://github.com/nf-core/chipseq/actions/workflows/ci.yml/badge.svg)](https://github.com/nf-core/chipseq/actions/workflows/ci.yml)
[![GitHub Actions Linting Status](https://github.com/nf-core/chipseq/actions/workflows/linting.yml/badge.svg)](https://github.com/nf-core/chipseq/actions/workflows/linting.yml)[![AWS CI](https://img.shields.io/badge/CI%20tests-full%20size-FF9900?labelColor=000000&logo=Amazon%20AWS)](https://nf-co.re/chipseq/results)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.3240506-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.3240506)
[![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com)

[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A524.04.2-23aa62.svg)](https://www.nextflow.io/)
[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)
@@ -70,7 +70,7 @@ You can find numerous talks on the [nf-core events page](https://nf-co.re/events
## Usage

> [!NOTE]
> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.
> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow.Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.
To run on your data, prepare a tab-separated samplesheet with your input data. Please follow the [documentation on samplesheets](https://nf-co.re/chipseq/usage#samplesheet-input) for more details. An example samplesheet for running the pipeline looks as follows:

2 changes: 1 addition & 1 deletion conf/base.config
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ process {
maxErrors = '-1'

// Process-specific resource requirements
// NOTE - Please try and re-use the labels below as much as possible.
// NOTE - Please try and reuse the labels below as much as possible.
// These labels are used and recognised by default in DSL2 files hosted on nf-core/modules.
// If possible, it would be nice to keep the same label naming convention when
// adding in your local modules too.
32 changes: 14 additions & 18 deletions docs/usage.md
Original file line number Diff line number Diff line change
@@ -151,9 +151,8 @@ If you wish to repeatedly use the same parameters for multiple runs, rather than

Pipeline settings can be provided in a `yaml` or `json` file via `-params-file <file>`.

:::warning
Do not use `-c <file>` to specify parameters as this will result in errors. Custom config files specified with `-c` must only be used for [tuning process resource specifications](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources), other infrastructural tweaks (such as output directories), or module arguments (args).
:::
> [!WARNING]
> Do not use `-c <file>` to specify parameters as this will result in errors. Custom config files specified with `-c` must only be used for [tuning process resource specifications](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources), other infrastructural tweaks (such as output directories), or module arguments (args).
The above pipeline run specified with a params file in yaml format:

@@ -182,40 +181,37 @@ nextflow pull nf-core/chipseq

### Reproducibility

It is a good idea to specify a pipeline version when running the pipeline on your data. This ensures that a specific version of the pipeline code and software are used when you run your pipeline. If you keep using the same tag, you'll be running the same version of the pipeline, even if there have been changes to the code since.
It is a good idea to specify the pipeline version when running the pipeline on your data. This ensures that a specific version of the pipeline code and software are used when you run your pipeline. If you keep using the same tag, you'll be running the same version of the pipeline, even if there have been changes to the code since.

First, go to the [nf-core/chipseq releases page](https://github.com/nf-core/chipseq/releases) and find the latest pipeline version - numeric only (eg. `2.1.0`). Then specify this when running the pipeline with `-r` (one hyphen) - eg. `-r 2.1.0`. Of course, you can switch to another version by changing the number after the `-r` flag.

This version number will be logged in reports when you run the pipeline, so that you'll know what you used when you look back in the future. For example, at the bottom of the MultiQC reports.

To further assist in reproducbility, you can use share and re-use [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.
To further assist in reproducibility, you can use share and reuse [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.

:::tip
If you wish to share such profile (such as upload as supplementary material for academic publications), make sure to NOT include cluster specific paths to files, nor institutional specific profiles.
:::
> [!TIP]
> If you wish to share such profile (such as upload as supplementary material for academic publications), make sure to NOT include cluster specific paths to files, nor institutional specific profiles.
## Core Nextflow arguments

:::note
These options are part of Nextflow and use a _single_ hyphen (pipeline parameters use a double-hyphen).
:::
> [!NOTE]
> These options are part of Nextflow and use a _single_ hyphen (pipeline parameters use a double-hyphen)
### `-profile`

Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments.

Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Shifter, Charliecloud, Apptainer, Conda) - see below.

:::info
We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
:::
> [!IMPORTANT]
> We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
The pipeline also dynamically loads configurations from [https://github.com/nf-core/configs](https://github.com/nf-core/configs) when it runs, making multiple config profiles for various institutional clusters available at run time. For more information and to see if your system is available in these configs please see the [nf-core/configs documentation](https://github.com/nf-core/configs#documentation).
The pipeline also dynamically loads configurations from [https://github.com/nf-core/configs](https://github.com/nf-core/configs) when it runs, making multiple config profiles for various institutional clusters available at run time. For more information and to check if your system is supported, please see the [nf-core/configs documentation](https://github.com/nf-core/configs#documentation).

Note that multiple profiles can be loaded, for example: `-profile test,docker` - the order of arguments is important!
They are loaded in sequence, so later profiles can overwrite earlier profiles.

If `-profile` is not specified, the pipeline will run locally and expect all software to be installed and available on the `PATH`. This is _not_ recommended, since it can lead to different results on different machines dependent on the computer enviroment.
If `-profile` is not specified, the pipeline will run locally and expect all software to be installed and available on the `PATH`. This is _not_ recommended, since it can lead to different results on different machines dependent on the computer environment.

- `test`
- A profile with a complete configuration for automated testing
@@ -251,13 +247,13 @@ Specify the path to a specific config file (this is a core Nextflow command). Se

### Resource requests

Whilst the default requirements set within the pipeline will hopefully work for most people and with most input data, you may find that you want to customise the compute resources that the pipeline requests. Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the steps in the pipeline, if the job exits with any of the error codes specified [here](https://github.com/nf-core/rnaseq/blob/4c27ef5610c87db00c3c5a3eed10b1d161abf575/conf/base.config#L18) it will automatically be resubmitted with higher requests (2 x original, then 3 x original). If it still fails after the third attempt then the pipeline execution is stopped.
Whilst the default requirements set within the pipeline will hopefully work for most people and with most input data, you may find that you want to customise the compute resources that the pipeline requests. Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the pipeline steps, if the job exits with any of the error codes specified [here](https://github.com/nf-core/rnaseq/blob/4c27ef5610c87db00c3c5a3eed10b1d161abf575/conf/base.config#L18) it will automatically be resubmitted with higher resources request (2 x original, then 3 x original). If it still fails after the third attempt then the pipeline execution is stopped.

To change the resource requests, please see the [max resources](https://nf-co.re/docs/usage/configuration#max-resources) and [tuning workflow resources](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources) section of the nf-core website.

### Custom Containers

In some cases you may wish to change which container or conda environment a step of the pipeline uses for a particular tool. By default nf-core pipelines use containers and software from the [biocontainers](https://biocontainers.pro/) or [bioconda](https://bioconda.github.io/) projects. However in some cases the pipeline specified version maybe out of date.
In some cases, you may wish to change the container or conda environment used by a pipeline steps for a particular tool. By default, nf-core pipelines use containers and software from the [biocontainers](https://biocontainers.pro/) or [bioconda](https://bioconda.github.io/) projects. However, in some cases the pipeline specified version maybe out of date.

To use a different container from the default container or conda environment specified in a pipeline, please see the [updating tool versions](https://nf-co.re/docs/usage/configuration#updating-tool-versions) section of the nf-core website.

6 changes: 3 additions & 3 deletions modules.json
Original file line number Diff line number Diff line change
@@ -217,17 +217,17 @@
},
"utils_nextflow_pipeline": {
"branch": "master",
"git_sha": "3aa0aec1d52d492fe241919f0c6100ebf0074082",
"git_sha": "c2b22d85f30a706a3073387f30380704fcae013b",
"installed_by": ["subworkflows"]
},
"utils_nfcore_pipeline": {
"branch": "master",
"git_sha": "1b6b9a3338d011367137808b49b923515080e3ba",
"git_sha": "51ae5406a030d4da1e49e4dab49756844fdd6c7a",
"installed_by": ["subworkflows"]
},
"utils_nfschema_plugin": {
"branch": "master",
"git_sha": "bbd5a41f4535a8defafe6080e00ea74c45f4f96c",
"git_sha": "2fd2cd6d0e7b273747f32e465fdc6bcc3ae0814e",
"installed_by": ["subworkflows"]
}
}
2 changes: 1 addition & 1 deletion modules/nf-core/fastqc/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions modules/nf-core/fastqc/meta.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

66 changes: 54 additions & 12 deletions nextflow.config
Original file line number Diff line number Diff line change
@@ -88,8 +88,7 @@ params {
show_hidden = false
version = false
pipelines_testdata_base_path = 'https://raw.githubusercontent.com/nf-core/test-datasets/'

// Config options
trace_report_suffix = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')// Config options
config_profile_name = null
config_profile_description = null

@@ -203,6 +202,13 @@ profiles {
executor.name = 'local'
executor.cpus = 4
executor.memory = 8.GB
process {
resourceLimits = [
memory: 8.GB,
cpus : 4,
time : 1.h
]
}
}
test { includeConfig 'conf/test.config' }
test_full { includeConfig 'conf/test_full.config' }
@@ -251,30 +257,65 @@ set -C # No clobber - prevent output redirection from overwriting files.
// Disable process selector warnings by default. Use debug profile to enable warnings.
nextflow.enable.configProcessNamesValidation = false

def trace_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')
timeline {
enabled = true
file = "${params.outdir}/pipeline_info/execution_timeline_${trace_timestamp}.html"
file = "${params.outdir}/pipeline_info/execution_timeline_${params.trace_report_suffix}.html"
}
report {
enabled = true
file = "${params.outdir}/pipeline_info/execution_report_${trace_timestamp}.html"
file = "${params.outdir}/pipeline_info/execution_report_${params.trace_report_suffix}.html"
}
trace {
enabled = true
file = "${params.outdir}/pipeline_info/execution_trace_${trace_timestamp}.txt"
file = "${params.outdir}/pipeline_info/execution_trace_${params.trace_report_suffix}.txt"
}
dag {
enabled = true
file = "${params.outdir}/pipeline_info/pipeline_dag_${trace_timestamp}.html"
file = "${params.outdir}/pipeline_info/pipeline_dag_${params.trace_report_suffix}.html"
}

manifest {
name = 'nf-core/chipseq'
author = """Espinosa-Carrasco J, Patel H, Wang C, Ewels P"""
author = """Espinosa-Carrasco J, Patel H, Wang C, Ewels P""" // The author field is deprecated from Nextflow version 24.10.0, use contributors instead
contributors = [
// TODO nf-core: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
[
name: 'Espinosa-Carrasco J',
affiliation: '',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is missing too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I will leave it for a new PR as I want to see how others do it and also copy paste it from them 👅

email: '',
github: '',
contribution: [], // List of contribution types ('author', 'maintainer' or 'contributor')
orcid: ''
],
[
name: ' Patel H',
affiliation: '',
email: '',
github: '',
contribution: [], // List of contribution types ('author', 'maintainer' or 'contributor')
orcid: ''
],
[
name: ' Wang C',
affiliation: '',
email: '',
github: '',
contribution: [], // List of contribution types ('author', 'maintainer' or 'contributor')
orcid: ''
],
[
name: ' Ewels P',
affiliation: '',
email: '',
github: '',
contribution: [], // List of contribution types ('author', 'maintainer' or 'contributor')
orcid: ''
],
]
homePage = 'https://github.com/nf-core/chipseq'
description = """ChIP-seq peak-calling and differential analysis pipeline."""
mainScript = 'main.nf'
defaultBranch = 'master'
nextflowVersion = '!>=24.04.2'
version = '2.2.0dev'
doi = 'https://doi.org/10.5281/zenodo.3240506'
@@ -287,9 +328,10 @@ plugins {

validation {
defaultIgnoreParams = ["genomes"]
monochromeLogs = params.monochrome_logs
help {
enabled = true
command = "nextflow run $manifest.name -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
command = "nextflow run nf-core/chipseq -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
fullParameter = "help_full"
showHiddenParameter = "show_hidden"
beforeText = """
@@ -299,15 +341,15 @@ validation {
\033[0;34m |\\ | |__ __ / ` / \\ |__) |__ \033[0;33m} {\033[0m
\033[0;34m | \\| | \\__, \\__/ | \\ |___ \033[0;32m\\`-._,-`-,\033[0m
\033[0;32m`._,._,\'\033[0m
\033[0;35m ${manifest.name} ${manifest.version}\033[0m
\033[0;35m nf-core/chipseq ${manifest.version}\033[0m
-\033[2m----------------------------------------------------\033[0m-
"""
afterText = """${manifest.doi ? "* The pipeline\n" : ""}${manifest.doi.tokenize(",").collect { " https://doi.org/${it.trim().replace('https://doi.org/','')}"}.join("\n")}${manifest.doi ? "\n" : ""}
afterText = """${manifest.doi ? "\n* The pipeline\n" : ""}${manifest.doi.tokenize(",").collect { " https://doi.org/${it.trim().replace('https://doi.org/','')}"}.join("\n")}${manifest.doi ? "\n" : ""}
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
* Software dependencies
https://github.com/${manifest.name}/blob/master/CITATIONS.md
https://github.com/nf-core/chipseq/blob/master/CITATIONS.md
"""
}
summary {
6 changes: 6 additions & 0 deletions nextflow_schema.json
Original file line number Diff line number Diff line change
@@ -556,6 +556,12 @@
"description": "Base URL or local path to location of pipeline test dataset files",
"default": "https://raw.githubusercontent.com/nf-core/test-datasets/",
"hidden": true
},
"trace_report_suffix": {
"type": "string",
"fa_icon": "far calendar",
"description": "Suffix to add to the trace report filename. Default is the date and time in the format yyyy-MM-dd_HH-mm-ss.",
"hidden": true
}
}
}
316 changes: 316 additions & 0 deletions ro-crate-metadata.json

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions subworkflows/local/utils_nfcore_chipseq_pipeline/main.nf
Original file line number Diff line number Diff line change
@@ -97,6 +97,7 @@ workflow PIPELINE_COMPLETION {

main:
summary_params = paramsSummaryMap(workflow, parameters_schema: "nextflow_schema.json")
def multiqc_reports = multiqc_report.toList()

//
// Completion email and summary
@@ -110,7 +111,7 @@ workflow PIPELINE_COMPLETION {
plaintext_email,
outdir,
monochrome_logs,
multiqc_report.toList()
multiqc_reports.getVal(),
)
}

@@ -201,7 +202,7 @@ def toolBibliographyText() {
}

def methodsDescriptionText(mqc_methods_yaml) {
// Convert to a named map so can be used as with familar NXF ${workflow} variable syntax in the MultiQC YML file
// Convert to a named map so can be used as with familiar NXF ${workflow} variable syntax in the MultiQC YML file
def meta = [:]
meta.workflow = workflow.toMap()
meta["manifest_map"] = workflow.manifest.toMap()
2 changes: 2 additions & 0 deletions subworkflows/nf-core/utils_nextflow_pipeline/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

89 changes: 23 additions & 66 deletions subworkflows/nf-core/utils_nfcore_pipeline/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions workflows/chipseq.nf
Original file line number Diff line number Diff line change
@@ -518,8 +518,12 @@ workflow CHIPSEQ {
// Collate and save software versions
//
softwareVersionsToYAML(ch_versions)
.collectFile(storeDir: "${params.outdir}/pipeline_info", name: 'nf_core_chipseq_software_mqc_versions.yml', sort: true, newLine: true)
.set { ch_collated_versions }
.collectFile(
storeDir: "${params.outdir}/pipeline_info",
name: 'nf_core_' + 'chipseq_software_' + 'mqc_' + 'versions.yml',
sort: true,
newLine: true
).set { ch_collated_versions }

//
// MODULE: MultiQC