diff --git a/README.md b/README.md index 18d171e..a3f6956 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ +[](https://gitpod.io/#https://github.com/AngryMaciek/warlock) [](https://github.com/AngryMaciek/warlock/actions?query=workflow%3Aci-pipeline) [](https://www.codefactor.io/repository/github/angrymaciek/warlock/overview/master) +[](https://mybinder.org/v2/gh/AngryMaciek/warlock/master?filepath=notebook.ipynb) [](https://hub.docker.com/r/angrymaciek/warlock) [](https://github.com/AngryMaciek/warlock/issues) [](https://github.com/AngryMaciek/warlock/blob/master/LICENSE) @@ -123,7 +125,7 @@ Running large workflows with hundreds of cluster jobs might take very long; cons ## Output -After each pipeline run the main output directory will contain three subdirectories: `configfiles`, `simulations` and `logs`. Each simulation run with a specific set of parameters is encoded by a 4-letter code. The first directory contains configuration files for each of the simulation runs; `simulations` contain all _demon_ output files; `logs` keep captured standard output and error streams for the commands. +After each pipeline run the main output directory will contain four subdirectories: `configfiles`, `simulations`, `analyses` and `logs`. Each simulation run with a specific set of parameters is encoded by a 8-letter code. The first directory contains configuration files for each of the simulation runs; `simulations` contain all _demon_ output files; `analyses` contains plots generated with _demonanalysis_; `logs` keep captured standard output and error streams for the commands. Feel free to run the pipeline and inspect the results yourself in an [interactive jupyter notebook](https://mybinder.org/v2/gh/AngryMaciek/warlock/master?labpath=notebook.ipynb) we prepared. ## Community guidelines For guidelines on how to contribute to the project or report issues, please see [contributing instructions](/CONTRIBUTING.md). diff --git a/binder/environment.yml b/binder/environment.yml new file mode 100644 index 0000000..8a6535a --- /dev/null +++ b/binder/environment.yml @@ -0,0 +1,47 @@ +############################################################################### +# +# Software to be installed in the main environment +# +# AUTHOR: Maciej_Bak +# CONTACT: wsciekly.maciek@gmail.com +# CREATED: 15-02-2022 +# LICENSE: Apache_2.0 +# +############################################################################### +--- + + name: warlock + + channels: + - conda-forge + - bioconda + + dependencies: + - bioconductor-graph>=1.76.0 + - bioconductor-rgraphviz>=2.42.0 + - boost-cpp>=1.74.0 + - compilers>=1.7.0 + - graphviz>=8.0.3 + - jinja2>=3.1.2 + - python>=3.8.0 + - pyyaml>=6.0 + - r-ape>=5.8 + - r-base>=4.0.3 + - r-data.table>=1.15.4 + - r-devtools>=2.4.5 + - r-dplyr>=0.7.0 + - r-ggmuller>=0.5.6 + - r-ggplot2>=3.5.1 + - r-gridextra>=2.3 + - r-lazyeval>=0.2.2 + - r-moments>=0.14.1 + - r-optparse>=1.7.5 + - r-rcolorbrewer>=1.1_3 + - r-readr>=2.1.5 + - r-rlang>=1.1.4 + - r-rvaidememoire>=0.9_83_7 + - r-scales>=1.3.0 + - r-teachingdemos>=2.12.1 + - snakemake>=7.18.2 + +... diff --git a/binder/postBuild b/binder/postBuild new file mode 100644 index 0000000..ae7dc4f --- /dev/null +++ b/binder/postBuild @@ -0,0 +1,4 @@ +#!/bin/bash + +set -ex +Rscript -e "devtools::install_github('robjohnnoble/demonanalysis', upgrade=FALSE)" \ No newline at end of file diff --git a/notebook.ipynb b/notebook.ipynb new file mode 100644 index 0000000..5f18cc9 --- /dev/null +++ b/notebook.ipynb @@ -0,0 +1,236 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "1928a2a5", + "metadata": { + "tags": [] + }, + "source": [ + "### _warlock_ vignette" + ] + }, + { + "cell_type": "markdown", + "id": "4450695b", + "metadata": {}, + "source": [ + "The following notebook demonstrates workflow execution with a simple set of simulation parameters." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "89b9ced8", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from IPython.display import Image" + ] + }, + { + "cell_type": "markdown", + "id": "0a4b57d1", + "metadata": {}, + "source": [ + "## Prepare files for the workflow" + ] + }, + { + "cell_type": "markdown", + "id": "9f7ed01b", + "metadata": {}, + "source": [ + "In the example below we will prepare a configuration file varying one specific parameter: deme carrying capacity.\n", + "\n", + "All dependencies have been already installed while building a container for this notebook." + ] + }, + { + "cell_type": "markdown", + "id": "63403efe", + "metadata": {}, + "source": [ + "Compile _demon_ (skip this step unless you use MyBinder service for notebook exploration)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "90a4c46c", + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "\n", + "g++ resources/demon_model/src/demon.cpp -o resources/demon_model/bin/demon -I/srv/conda/envs/notebook/include/ -lm" + ] + }, + { + "cell_type": "markdown", + "id": "cb706573", + "metadata": {}, + "source": [ + "Prepare configfile for the workflow (based on a template)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3aa3277f", + "metadata": {}, + "outputs": [], + "source": [ + "with open(\"tests/binder/config-template.yml\") as f:\n", + " configlines = f.read().splitlines()\n", + "\n", + "configlines[17] = \"workflow_repo_path: \\\"\" + os.getcwd() + \"\\\"\"\n", + "\n", + "outdir = os.path.join(\n", + " os.getcwd(),\n", + " \"tests\",\n", + " \"binder\",\n", + " \"output\"\n", + ")\n", + "\n", + "configlines[22] = \"workflow_analysis_outdir: \\\"\" + outdir + \"\\\"\"\n", + "\n", + "with open(\"tests/binder/config.yml\", \"w\") as f:\n", + " for line in configlines:\n", + " f.write(line + os.linesep)" + ] + }, + { + "cell_type": "markdown", + "id": "5298889a", + "metadata": {}, + "source": [ + "## Execute the workflow" + ] + }, + { + "cell_type": "markdown", + "id": "59834894", + "metadata": {}, + "source": [ + "We trigger the snakemake workflow via our main bash script (~5min runtime)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6f9a1e6a", + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "\n", + "time \\\n", + "bash warlock.sh \\\n", + "--configfile tests/binder/config.yml \\\n", + "--environment local \\\n", + "--cores 3" + ] + }, + { + "cell_type": "markdown", + "id": "7317904e", + "metadata": {}, + "source": [ + "## Inspect summary plots:" + ] + }, + { + "cell_type": "markdown", + "id": "01c72cb9", + "metadata": {}, + "source": [ + "$\\log_2$[deme carrying capacity] = 8" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2b0f8217", + "metadata": {}, + "outputs": [], + "source": [ + "Image(filename=\"tests/binder/output/analyses/00000000/muller.png\") " + ] + }, + { + "cell_type": "markdown", + "id": "518dde2c", + "metadata": {}, + "source": [ + "$\\log_2$[deme carrying capacity] = 9" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e4a74b6", + "metadata": {}, + "outputs": [], + "source": [ + "Image(filename=\"tests/binder/output/analyses/00000001/muller.png\") " + ] + }, + { + "cell_type": "markdown", + "id": "123529a1", + "metadata": {}, + "source": [ + "$\\log_2$[deme carrying capacity] = 10" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "018e1056", + "metadata": {}, + "outputs": [], + "source": [ + "Image(filename=\"tests/binder/output/analyses/00000002/muller.png\") " + ] + }, + { + "cell_type": "markdown", + "id": "dfdd7551", + "metadata": {}, + "source": [ + "**The simple example above highlights that the higher the deme carrying capacity is the easier the tumor evolves within the population.**" + ] + }, + { + "cell_type": "markdown", + "id": "782ed072", + "metadata": {}, + "source": [ + "---" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}