20170816

[Setup]

In a dedicated terminal window, begin an interactive bash session:

qlogin

Run the following commands:

PIPEWS=/data/joy/BBL-extend/share/pipeline_workshop
cd $PIPEWS
source $PIPEWS/software/configureAccelerator.sh
FCv6=$PIPEWS/software/ACCELERATOR-prerelease/designs/example.dsn
FCv6local=$PIPEWS/output/accelerator_r2/100031/3818/100031_3818.dsn
FCv6out=$PIPEWS/output/accelerator_r2/100031/3818/
FCv5=$PIPEWS/software/xcpEngine/designs/example.dsn
STRv5=$PIPEWS/software/xcpEngine/designs/struct.dsn
ls -d1 $PIPEWS/software/*/

Two versions of the processing software are installed in the software directory:

• xcpEngine contains v0.5.1 -- it's similar to the version that you have been using (v0.5.0), with the addition of a structural processing stream contributed by @adrose.
• ACCELERATOR-prerelease contains v0.6.0, which includes substantial changes to the code base. This is the version that will be supported moving forward. This version is not yet complete, but it can support an elementary connectivity pipeline.
• In this session, we will focus on v0.6.0 for functional processing and v0.5.1 for structural processing, but support for v0.5.x will cease in the future.

[Modules]

View all of the modules that currently exist: ls $XCPEDIR/modules/*/*.mod|rev|cut -d'/' -f1|rev

View the code of an example (v0.6.0) module: cat ${XCPEDIR}/modules/prestats/prestats.mod

Modules are not designed to run as autonomous scripts; instead, they are given meaning in the context of a pipeline designed by the user.

[xcpEngine]

Use the following command to bring up usage for the pipeline engine:

${XCPEDIR}/xcpEngine

We will now discuss the various arguments to the engine.

[The design file]

View some example design files:

• v0.5.1 [structural]: cat ${STRv5}
• v0.5.x [functional]: cat ${FCv5}
• v0.6.0 [functional]: cat ${FCv6}

The sequence of modules that will be run is specified in the pipeline variable. Find the pipeline variable in the structural and functional design files:

• Functional: grep '^pipeline=' ${FCv6}
• Structural: grep '^pipeline=' ${STRv5}

[The cohort file]

The format has not changed between versions: Update 20170821: The format has now changed. A v0.6.0-conformed design file can be found here: `cat /data/joy/BBL-extend/share/pipeline_workshop/config/cohort/cohort-v6.csv

This is a cohort file for functional connectivity processing. Here, the format is as follows (begin numbering from 0):

• Column 0 is the subject identifier.
• Column 1 is the timepoint identifier.
• Column 2 is the (distortion-corrected) functional image.
• Column 3 is the ANTsCT output directory. For structural processing, the only required inputs in the cohort file are identifiers and the raw anatomical scan.

Columns from the cohort file can be used to define variables in the design file.

• Run grep '^confound_csf_path' ${FCv6}
• This will give you the path to an image containing a map of the subject's cerebrospinal fluid.
• Notice that the path contains a variable, antsct[sub]. When the design file is parsed, it replaces that variable (antsct[sub]) with the column of the cohort file marked antsct. (sub is the syntax for a subject-level variable.)
• What is the cohort file? The cohort file is really a way of providing subject-specific input to your pipeline without having to write a new design file for every subject.

The cohort file is specified differently in v0.6.0 to make design files more portable.

• In v0.5.x, the cohort file is specified as a variable in the design file (cohort=cohort.csv).
• In v0.6.0, the cohort file is supplied as a separate argument to xcpEngine (-c cohort.csv).

[Localiser]

The localiser handles subject-specific rewriting of the design file. It creates a local design file for each subject.

• For an example of what the localiser does, see diff $FCv6 $FCv6local|head -n14.
• Note that the generalised subject-specific variables have been evaluated in the localised version of the design file. subject[2] and subject[3] have been replaced by columns 2 and 3 from the cohort file, respectively.
• In v0.6.0, the localiser will also conform all input images to the format used by our pipeline (NIFTI_GZ).

[Pipeline configuration]

Sorry -- our pipeline configuration system is offline for a much-needed overhaul! When v0.6.0 is released, pipeline configuration will work through direct edits to the design file and consultation of the built-in documentation system (also unfinished) until our new configuration system comes online.

[Running your analysis]

$XCPEDIR/xcpEngine -d $XCPEDIR/designs/example.dsn -o $PIPEWS/output/accelerator_$(whoami) -c $PIPEWS/config/cohort/cohort_example.csv -i $TMPDIR -t 1

Then, start a new terminal session and run the following commands again:

PIPEWS=/data/joy/BBL-extend/share/pipeline_workshop
cd $PIPEWS
source $PIPEWS/software/configureAccelerator.sh
FCv6=$PIPEWS/software/ACCELERATOR-prerelease/designs/example.dsn
FCv6local=$PIPEWS/output/accelerator_r2/100031/3818/100031_3818.dsn
FCv6out=$PIPEWS/output/accelerator_r2/100031/3818/
FCv5=$PIPEWS/software/xcpEngine/designs/example.dsn
STRv5=$PIPEWS/software/xcpEngine/designs/struct.dsn

[prestats -- Functional preprocessing]

• View the input variables for prestats: grep '^prestats_' ${FCv6}
• View the output of prestats: ls ${FCv6out}/prestats

[coreg -- Coregistration]

• View the input variables for coreg: grep '^coreg_' ${FCv6}
• View the output of coreg: ls ${FCv6out}/coreg

[confound -- Confound modelling]

• View the input variables for confound: grep '^confound_' ${FCv6}
• View the output of confound: ls ${FCv6out}/confound

[aroma -- ICA-AROMA]

• Not covered in this session.

[regress -- Confound regression]

• View the input variables for regress: grep '^regress_' ${FCv6}
• View the output of regress: ls ${FCv6out}/regress

[seed -- Seed-based connectivity]

• View the input variables for seed: grep '^seed_' ${FCv6}
• The inputs include a seed correlation library. Let's look at that library in a little more detail: sclib=$(grep '^seed_lib' ${FCv6}|cut -d'=' -f2); cat ${sclib}
  • The seed library is formatted as a hash-delimited file. (This format may change before release.)
  • The fields in the first seed's definition are as follows: #SEED_NAME#SEED_COORDINATES_IN_VOXELS#SEED_RADIUS_IN_MM#. This seed represents a sphere of radius 8mm centred on the posterior cingulate cortex. The space of the seed is specified in the header of the library file: SPACE::PNC%9375x9375x1. This corresponds to the PNC template with voxel dimensions 0.9375x0.9375x1 mm: cat $XCPEDIR/space/PNC/PNC_space.json|$XCPEDIR/thirdparty/jq/jq-linux64.
  • The fields in the second and third seeds' definitions are as follows: #SEED_NAME#PATH_TO_SEED_FILE#SEED_SPACE. Either format is acceptable.
  • New seeds can be added by adding a new line to the coordinates library. Find the voxel coordinates by viewing the map of the library space: fslview ${XCPEDIR}/space/PNC/PNC-9375x9375x1.nii.gz.
• View the output of seed: ls ${FCv6out}/seed
  • ls ${FCv6out}/seed/PCC8. For each seed, there is an output directory containing the map of the seed in analyte space (~seed.nii.gz), the mean activity time series in the seed region (~ts.1D), and voxelwise maps of correlations with the seed.

[reho -- Regional homogeneity]

• View the input variables for reho: grep '^reho_' ${FCv6}
• View the output of reho: ls ${FCv6out}/reho

[alff -- Amplitude of low-frequency fluctuations]

• View the input variables for alff: grep '^alff_' ${FCv6}
• View the output of alff: ls ${FCv6out}/alff

[roiquant -- Regional or atlas-based quantification]

• View the input variables for roiquant: grep '^roiquant_' ${FCv6}
• The inputs include an atlas file. Let's take a closer look at this file:

atlas=$(grep '^roiquant_atlas' ${FCv6}|cut -d'=' -f2); eval cat ${atlas}|$XCPEDIR/thirdparty/jq/jq-linux64

• The atlas file is formatted as JSON metadata. This atlas file includes 4 objects: Power264, Gordon, Glasser, and Schaefer400. Each object corresponds to a brain parcellation or an atlas of regions of interest.
• Each object has a number of properties. The Map property, for instance, specifies the path to the image containing the parcellation. The NodeNames property specifies the names of the nodes of the parcellation. The Space property tells us that each parcellation is in PNC space.
• As the atlas is processed, each parcellation is also added to a subject-specific atlas: cat ${FCv6out}/*atlas/*atlas.json|$XCPEDIR/thirdparty/jq/jq-linux64. Note that these parcellations are in the subject's native space (100031_3818_rest). Also note that they have a number of properties that the original parcellations didn't have, including connectivity matrices (MatrixFC).
• View the output of roiquant: ls ${FCv6out}/roiquant
  • Each parcellation has a separate output directory. ls ${FCv6out}/roiquant/Schaefer400/
  • View an example of regionally quantified data: cat ${FCv6out}/roiquant/Schaefer400/*_mean_reho.csv

[fcon -- Functional connectomics]

• View the input variables for fcon: grep '^fcon_' ${FCv6}
• View the output of fcon: ls ${FCv6out}/fcon
• Note that this is a new module in v0.6.0, but the functionality should be familiar if you've used v0.5.x -- it includes everything from the net module except for community detection. We made the decision to split the network module into two -- one (fcon) computes the connectivity matrix, while the other (net) performs network-based analyses. This decision will enable us to use the same module (net) on connectivity matrices derived from other modalities (like diffusion) as we expand the pipeline.
• This is the module that added most of the new properties to each parcellation.

[net -- Network analysis]

• View the input variables for net: grep '^net_' ${FCv6}
• View the output of net: ls ${FCv6out}/net

[norm -- Normalisation to a template]

• View the input variables for norm: grep '^norm_' ${FCv6}
• View the output of norm: ls ${FCv6out}/norm

[Output: on the file system]

ls $FCv6out

• View the processed design file: cat ${FCv6out}/*.dsn
• View quality metrics (slightly bugged): cat ${FCv6out}/*quality.csv
• View native-space derivatives: cat ${FCv6out}/*derivatives*rest.json|$XCPEDIR/thirdparty/jq/jq-linux64
• View spatial metadata: cat ${FCv6out}/*spaces.json|$XCPEDIR/thirdparty/jq/jq-linux64

[Processing application directory]

• ls ${XCPEDIR}

[v0.5.1 features]

[Sample structural output]

• /data/joy/BBL-extend/share/pipeline_workshop/output/structural-pipeline-v5/100088/7944/

[Template construction prototype]

• $XCPEDIR/utils/templateConstruct