Multi-Scale Toolkit

Build

Dependencies

To build JSI-Toolkit, you need the following dependencies installed:

• Python 2.7
• CMake >= 3.5
• DynInst >= 10.0.0
• Libunwind
• PAPI
• GCC >= 12.3.0
• Libdwarf
• MPI (Should be fit to the MPI the target cluster used)
• OTF2 3.0.3
• Boost 1.86.0
• range-v3 0.12.0
• sqlite 3.40.1
• fmt 10.2.1
• spdlog 1.14.1
• magic-enum 0.9.6

For convenience, you can build and install the dependencies via spack:

git clone https://github.com/spack/spack.git
. ./spack/share/spack/setup-env.sh
spack install dyninst papi libunwind otf2
spack install [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
spack load dyninst papi libunwind otf2
spack load [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
cd ..

Install of libdwarf

Libdwarf need to be installed manually (e.g., version 0.11.1). You can get instructions from: https://www.prevanders.net/dwarf.html#releases

cd libdwarf-x.x.x
mkdir build
cd build
../configure --prefix=<path-to-install> --enable-shared
make -j 
make install
export PKG_CONFIG_PATH=<path-to-libdwarf>/lib/pkgconfig:$PKG_CONFIG_PATH

Build & Install

git clone https://github.com/buaa-hipo/JSI-Toolkit.git
# the compiled binaries and libraries are installed in install/
./build.sh

Load the environment to use the installed JSI-Toolkit:

# make sure we are in the root directory of this repo
cd JSI-Toolkit
# load the environment variables
source env.sh
export OTF2_PATH=<path-to-otf2>

How to use JSI-Toolkit

To use JSI Toolkit, you neet to complete each step of the Build & Install Usage:

jsirun -o <measurement directory> --backtrace -- <EXE> <ARGS>
#
# -o DIR
#   PATH_DIR: the path of output file you want
# --backtrace: 
#   enable the backtrace function

Sampling Module

The following parameters of jsirun need to be set:

• --enable_sample: Enable sampling module adds the sampling dynamic library to the LD_PRELOAD environment variable.
• --events <(string)PMU events>: Define the PMU events to be collected.
• --samp_interval <(double)interval while sampling>: Sampling time interval, in seconds, expressed as a float number.
• -- <target program> <args>: Target program.

Example startup command line is as follows:

$JSI-Toolkit/install/bin/jsirun --enable_sample --samp_interval 1.2 --samp_events PAPI_TOT_INS,PAPI_TOT_CYC -- <target program> <args> 

Analysis Driver Commands

graph_analysis

Read the measurement directory generated by jsirun and generate chrome tracing file with dependancy graph annotation.
Usage:
  graph_analysis [OPTION...]

  -f, --force       Force to overwrite the output file if already exists.
  -h, --help        Print help
  -i, --input arg   Input measurement directory generated by jsirun
  -o, --output arg  Output file to store the chrome trace json outputs 
                    (default: timeline.json)

The generated JSON output can be opened by chrome tracing tool (chrome://tracing or edge://tracing for Google Chrome or Microsoft Edge).

variance_analysis

Read the measurement directory generated by jsirun and generate variance analysis results.
Usage:
  variance_analysis [OPTION...]

  -f, --force       Force to overwrite the output file if already exists.
  -h, --help        Print help
  -i, --input arg   Input measurement directory generated by jsirun
  -o, --output arg  Output directory to store the variance csv outputs. 
                    (default: variance)

The generated csv outputs are located in a output directory specified in -o/--output, which can be further processed to generate heatmap figures with the script located in script/analysis/variance/var_heatmap.py:

usage: var_heatmap.py [-h] [--input INPUT] [--output OUTPUT] [--time TIME]

Draw heatmap from csv generated by variance_anaylsis.

optional arguments:
  -h, --help       show this help message and exit
  --input INPUT    the variance directory containing csv generated by variance_analysis
  --output OUTPUT  output directory
  --time TIME      time threshold (ms) to cutoff the variance heatmap. 0 for full trace.

Convert to/from other formats

jsiextract

Convert jsi-format records to OTF2
  jsiextract [OPTION...]

  -b, --backtrace             Read backtrace information
  -e, --event_file arg        events that need to extract
  -h, --help                  Print help
  -l, --line_info_dump arg    Input line info directory dumped by 
                              dwarf_line_info_dump
  -i, --input arg             Input measurement directory generated by 
                              jsirun
  -o, --output arg            Output directory to store the readable 
                              outputs (default: readable)
  -t, --time_offset arg       Time offset of starting timestamps(ms). 
                              Default is 0 (default: 0)
      --interval_length arg   Time interval length of the trace (ms). 
                              Default is -1(inf). (default: -1)
  -n, --number_of_events arg  Number of events per rank. Default is -1(inf) 
                              (default: -1)

// events.txt
MPI_Recv
MPI_Send
MPI_Init
...

Contribution

1. The pull request must request one reviewer for code review before merging into the main branch.