Skip to content

Commit

Permalink
Merge branch 'dev' into feature/python-typing/list.py
Browse files Browse the repository at this point in the history
  • Loading branch information
kedhammar authored Oct 17, 2023
2 parents bba34d2 + 873da41 commit 9608ce9
Show file tree
Hide file tree
Showing 205 changed files with 9,207 additions and 5,105 deletions.
1 change: 1 addition & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"image": "nfcore/gitpod:latest",
"postCreateCommand": "python -m pip install --upgrade -r ../requirements-dev.txt -e ../ && pre-commit install --install-hooks",
"remoteUser": "gitpod",
"runArgs": ["--privileged"],

// Configure tool-specific properties.
"customizations": {
Expand Down
1 change: 0 additions & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,4 +142,3 @@ To get started:
Devcontainer specs:

- [DevContainer config](.devcontainer/devcontainer.json)
- [Dockerfile](.devcontainer/Dockerfile)
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ body:
attributes:
label: System information
description: |
* Nextflow version _(eg. 22.10.1)_
* Nextflow version _(eg. 23.04.0)_
* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
Expand Down
23 changes: 12 additions & 11 deletions .github/RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,21 @@
1. Check issue milestones to see outstanding issues to resolve if possible or transfer to the milestones for the next release e.g. [`v1.9`](https://github.com/nf-core/tools/issues?q=is%3Aopen+is%3Aissue+milestone%3A1.9)
2. Most importantly, pick an undeniably outstanding [name](http://www.codenamegenerator.com/) for the release where _Prefix_ = _Metal_ and _Dictionary_ = _Animal_.
3. Check the [pipeline health page](https://nf-co.re/pipeline_health) to make sure that all repos look sane (missing `TEMPLATE` branches etc)
4. Create a PR to `dev` to bump the version in `CHANGELOG.md` and `setup.py`.
5. Make sure all CI tests are passing!
6. Create a PR from `dev` to `master`
7. Make sure all CI tests are passing again (additional tests are run on PRs to `master`)
8. Request review (2 approvals required)
9. Run `rich-codex` to regenerate docs screengrabs (actions `workflow_dispatch` button)
10. Merge the PR into `master`
11. Wait for CI tests on the commit to passed
12. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR.
13. Create a new release copying the `CHANGELOG` for that release into the description section.
4. Check that modules/subworkflows in tempalte are up to date with the latest releases
5. Create a PR to `dev` to bump the version in `CHANGELOG.md` and `setup.py` and change the gitpod container to `nfcore/gitpod:latest`.
6. Make sure all CI tests are passing!
7. Create a PR from `dev` to `master`
8. Make sure all CI tests are passing again (additional tests are run on PRs to `master`)
9. Request review (2 approvals required)
10. Run `rich-codex` to regenerate docs screengrabs (actions `workflow_dispatch` button)
11. Merge the PR into `master`
12. Wait for CI tests on the commit to passed
13. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR.
14. Create a new release copying the `CHANGELOG` for that release into the description section.

## After release

1. Check the automated template synchronisation has been triggered properly. This should automatically open PRs directly to individual pipeline repos with the appropriate changes to update the pipeline template.
2. Check that the automatic `PyPi` deployment has worked: [pypi.org/project/nf-core](https://pypi.org/project/nf-core/)
3. Check `BioConda` has an automated PR to bump the version, and merge. eg. [bioconda/bioconda-recipes #20065](https://github.com/bioconda/bioconda-recipes/pull/20065)
4. Create a tools PR to `dev` to bump back to the next development version in `CHANGELOG.md` and `setup.py`
4. Create a tools PR to `dev` to bump back to the next development version in `CHANGELOG.md` and `setup.py` and change the gitpod container to `nfcore/gitpod:dev`.
24 changes: 24 additions & 0 deletions .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: "Close user-tagged issues and PRs"
on:
schedule:
- cron: "0 0 * * 0" # Once a week

jobs:
clean-up:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
days-before-stale: 30
days-before-close: 20
days-before-pr-close: -1
any-of-labels: "awaiting-changes,awaiting-feedback"
exempt-issue-labels: "WIP"
exempt-pr-labels: "WIP"
repo-token: ${{ secrets.GITHUB_TOKEN }}
12 changes: 8 additions & 4 deletions .github/workflows/create-lint-wf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,18 @@ jobs:
strategy:
matrix:
NXF_VER:
- "22.10.1"
- "23.04.0"
- "latest-everything"
steps:
# Get the repo code
- uses: actions/checkout@v3
name: Check out source-code repository

# Set up nf-core/tools
- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11

- name: Install python dependencies
run: |
Expand Down Expand Up @@ -84,6 +84,10 @@ jobs:
- name: remove TODO
run: find nf-core-testpipeline -type f -exec sed -i '/TODO nf-core:/d' {} \;

# Replace zenodo.XXXXXX to pass readme linting
- name: replace zenodo.XXXXXX
run: find nf-core-testpipeline -type f -exec sed -i 's/zenodo.XXXXXX/zenodo.123456/g' {} \;

# Run nf-core linting
- name: nf-core lint
run: nf-core --log-file log.txt --hide-progress lint --dir nf-core-testpipeline --fail-ignored --fail-warned
Expand All @@ -108,7 +112,7 @@ jobs:
run: nf-core --log-file log.txt modules install fastqc --dir nf-core-testpipeline/ --force

- name: nf-core modules install gitlab
run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git --branch main install fastqc --force --dir nf-core-testpipeline/
run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git --branch branch-tester install fastp --dir nf-core-testpipeline/

- name: nf-core modules list local
run: nf-core --log-file log.txt modules list local --dir nf-core-testpipeline/
Expand Down
137 changes: 137 additions & 0 deletions .github/workflows/create-test-lint-wf-template.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
name: Create a pipeline from a template and test it
on:
push:
branches:
- dev
paths:
- nf_core/pipeline-template/**
pull_request:
release:
types: [published]

# Cancel if a newer run is started
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
NXF_ANSI_LOG: false

jobs:
RunTestWorkflow:
runs-on: ubuntu-latest
env:
NXF_ANSI_LOG: false
strategy:
matrix:
TEMPLATE:
- "template_skip_all.yml"
- "template_skip_github_badges.yml"
- "template_skip_igenomes.yml"
- "template_skip_ci.yml"
- "template_skip_nf_core_configs.yml"

steps:
- uses: actions/checkout@v3
name: Check out source-code repository

- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install python dependencies
run: |
python -m pip install --upgrade pip
pip install .
- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
with:
version: latest-everything

# Install the Prettier linting tools
- uses: actions/setup-node@v3

- name: Install Prettier
run: npm install -g prettier

# Install the editorconfig linting tools
- name: Install editorconfig-checker
run: npm install -g editorconfig-checker

# Create template files
- name: Create template skip all (except github)
run: |
printf "prefix: my-prefix\nskip: ['ci', 'github_badges', 'igenomes', 'nf_core_configs']" > template_skip_all.yml
- name: Create template skip github_badges
run: |
printf "prefix: my-prefix\nskip: github_badges" > template_skip_github_badges.yml
- name: Create template skip igenomes
run: |
printf "prefix: my-prefix\nskip: igenomes" > template_skip_igenomes.yml
- name: Create template skip ci
run: |
printf "prefix: my-prefix\nskip: ci" > template_skip_ci.yml
- name: Create template skip nf_core_configs
run: |
printf "prefix: my-prefix\nskip: nf_core_configs" > template_skip_nf_core_configs.yml
# Create a pipeline from the template
- name: create a pipeline from the template ${{ matrix.TEMPLATE }}
run: |
nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface" --template-yaml ${{ matrix.TEMPLATE }}
- name: run the pipeline
run: |
nextflow run my-prefix-testpipeline -profile test,docker --outdir ./results
# Remove results folder before linting
- name: remove results folder
run: |
rm -rf ./results
# Try syncing it before we change anything
- name: nf-core sync
run: nf-core --log-file log.txt sync --dir my-prefix-testpipeline/

# Run code style linting
- name: Run Prettier --check
run: prettier --check my-prefix-testpipeline

- name: Run ECLint check
run: editorconfig-checker -exclude README.md $(find my-prefix-testpipeline/.* -type f | grep -v '.git\|.py\|md\|json\|yml\|yaml\|html\|css\|work\|.nextflow\|build\|nf_core.egg-info\|log.txt\|Makefile')

# Remove TODO statements
- name: remove TODO
run: find my-prefix-testpipeline -type f -exec sed -i '/TODO nf-core:/d' {} \;

# Replace zenodo.XXXXXX to pass readme linting
- name: replace zenodo.XXXXXX
run: find my-prefix-testpipeline -type f -exec sed -i 's/zenodo.XXXXXX/zenodo.123456/g' {} \;

# Run nf-core linting
- name: nf-core lint
run: nf-core --log-file log.txt --hide-progress lint --dir my-prefix-testpipeline --fail-warned

# Run bump-version
- name: nf-core bump-version
run: nf-core --log-file log.txt bump-version --dir my-prefix-testpipeline/ 1.1

# Run nf-core linting in release mode
- name: nf-core lint in release mode
run: nf-core --log-file log.txt --hide-progress lint --dir my-prefix-testpipeline --fail-warned --release

- name: Tar files
run: tar -cvf artifact_files.tar log.txt template_skip*.yml

- name: Upload log file artifact
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: nf-core-log-file
path: artifact_files.tar
6 changes: 3 additions & 3 deletions .github/workflows/create-test-wf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,16 @@ jobs:
strategy:
matrix:
NXF_VER:
- "22.10.1"
- "23.04.0"
- "latest-everything"
steps:
- uses: actions/checkout@v3
name: Check out source-code repository

- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11

- name: Install python dependencies
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ jobs:
- uses: actions/checkout@v3
name: Check out source-code repository

- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11

- name: Install python dependencies
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/fix-linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ jobs:
# Override to remove the default --check flag so that we make changes
options: "--color"

- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11
- name: python-isort
uses: isort/[email protected]
with:
Expand Down
23 changes: 21 additions & 2 deletions .github/workflows/lint-code.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,31 @@ jobs:
- name: Check out source-code repository
uses: actions/checkout@v3

- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11
- name: python-isort
uses: isort/[email protected]
with:
isortVersion: "latest"
requirementsFiles: "requirements.txt requirements-dev.txt"

static-type-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v3
with:
python-version: "3.11"
- run: pip install mypy types-PyYAML
- name: Get Python changed files
id: changed-py-files
uses: tj-actions/changed-files@v23
with:
files: |
*.py
**/*.py
- name: Run if any of the listed files above is changed
if: steps.changed-py-files.outputs.any_changed == 'true'
run: mypy ${{ steps.changed-py-files.outputs.all_changed_files }} --ignore-missing-imports
52 changes: 0 additions & 52 deletions .github/workflows/pytest-frozen-ubuntu-20.04.yml

This file was deleted.

Loading

0 comments on commit 9608ce9

Please sign in to comment.