Jupyter Notebook Report

Report

Instead of plotting individual plots using the Python mjolnir script, there is a jupyter notebook that can be run to automatically create a page with plots and animated plots to get an overview of the output of the simulation.

Installation

These are the packages required to run the script. called with pip3, they pull in the other needed dependencies:

$ pip3 install --user jupyter_contrib_nbextensions nbformat matplotlib h5py imageio nbparameterise scipy pyshtools pycuda imageio-ffmpeg

ffmpeg program is also necessary to create the videos. That can be installed through your package manager, probably with something similar to this on Ubuntu and Debian:

$ sudo apt-get install ffmpeg

Usage

Open the notebook

This badboy lives at "THOR/mjolnir/mjolnyr.ipynb". In order to access videos in the notebook, you'll have to initialize the jupyter in a directory upstream from both mjolnir and the path to video output (output_path, under "Configuration" below). Find such a directory and start the jupyter server with:

$ jupyter notebook

This should open in a browser, where you can then navigate to "mjolnir/mjolnyr.ipynb".

Configuration

The first code block of the notebook contains all the variables on how to run, that can be set in the notebook when running it or from the automatic report creation script:

# where to find the dataset
data_path = "../Alfrodull/experiments/ubelix/walf_spinup_albedo1"
# where to output the images and movies
output_path = "../Alfrodull/experiments/ubelix/walf_spinup_albedo1/report"

# path to mjolnir from execution directory, to find all the python tools, used if ran in another folder, e.g. with 'muninn' script
mjolnir_path = "./"

# size in inch for the matplotlib figures, using also dpi (dots per inch) resolution below
FIGSIZE_x = 10
FIGSIZE_y = 7

# size in pixels of videos, only X size, Y size is computed from aspect ratio from FIGSIZE_*
# size of output video file
VIDEO_x = 1280
# size of displayed video in notebook
VIDEO_DISPLAY_x = 800

# resolution in pixels per inch for matplotlib 
dpi = 296

Standard Jupyter Notebook

Configure the notebook variables. In this case, the paths are relative to where the notebook mjolnyr.ipynb file is. Run the notebook in your browser like you'd run any jupyter notebook.

Reporting creation script

Don't configure the notebook path variables in the notebook, those will be set by calling muninn script. In this case, the paths will need to be relative to where you run the muninn script from. Simplest is to run it from THOR base directory.

muninn usage tells us:

usage: muninn [-h] [-j JUPNB] [-t TIMEOUT] data_folder

THOR report builder

positional arguments:
  data_folder           folder to look for data

optional arguments:
  -h, --help            show this help message and exit
  -j JUPNB, --jupnb JUPNB
                        jupyter notebook to run
  -t TIMEOUT, --timeout TIMEOUT
                        notebook cell execution timeout

If you run from THOR folder, default parameters are good. Run it with:

$ muninn <datafolder>

or if mjolnir folder is not on your path:

$ mjolnir/muninn <datafolder>

It will output the report to the report subdirectory of the datafolder.

Run on slurm

If you run THOR on a slurm cluster with slurm_batch_script.py, you can add a jobid to run the reporting by using the -r flag to slurm_batch_script.py.

If you want to run it on a dataset without running THOR, just tell it to run 0 THOR jobs:

user@cluser:~/THOR $ python3 tools/slurm_batch_run.py -jn <jobname> -n 0 -o <output folder> ifile/<initial conditions>.thr -r

And if you started a slurm job, you can add a dependency to this job to wait for it to be finished using -d:

user@cluser:~/THOR $ python3 tools/slurm_batch_run.py -jn <jobname> -n 0 -o <output folder> ifile/<initial conditions>.thr -d 31415962 -r