MagellanMapper v1.3.0

MagellanMapper v1.3.0

This release brings many changes including a unified graphical interface. We also cleaned up the command-line interface (CLI) to make it slightly simpler and more flexible. Please note that several options have changed, which may require updating custom scripts. The sample commands script (bin/sample_cmds.sh) has been updated to illustrate this usage.

Summary of CLI usage changes:

Old	New	Purpose
python -m magmap.xx.yy	run_cli.py	All command-line based entry points can be accessed through the CLI using this script
Use atlas profile for registration	No longer needed	Atlases should be fully imported before image registration, and the atlas's profile should no longer be typically given when registering an image
--stats	--df	Run data-frame (eg CSV file) tasks
--roc	--grid_search <name1>[,name2]	Its main task is to perform Grid Search based hyperparameter tuning; specify profile names or YAML files
--rescale	--transform rescale=x	Grouped with other transformation tasks
--microscope <name1>[_name2]	--roi_profile <name1>[,name2]	Specifies profiles to process by regions of interest; delimit by , to allow underscores especially in file paths
--reg_profile <name1>[_name2]	--atlas_profile <name1>[,name2]	Specifies profiles for atlases; delimit by , to allow underscores especially in file paths
--saveroi	--save_subimg	Consistency with "sub-images" as parts of images that can contain ROIs
--chunk_size	None	Obsolete
finer atlas profile	None	Its settings are now default
--res	--set_meta resolutions=x,y,z	Grouped custom metadata settings into --set_meta
--mag	--set_meta magnification=x.y	Grouped custom metadata settings into --set_meta
--zoom	--set_meta zoom=x,y,z	Grouped custom metadata settings into --set_meta
--no_show	--show 0	Show with 1
--no_scale_bar	--plot_labels scale_bar=1	Grouped with other plot labels controls
--padding_2d	--plot_labels margin=x,y,z	Grouped with other plot labels controls, adding margin as space outside the ROI
--border	--plot_labels padding=x,y,z	Duplicated by the padding argument

Changes

Installation

• New Windows install script
• The install scripts are now the recommended installation pathway
• Install scripts perform silent Miniconda installs after prompting
• Fixed the URL for Miniconda download
• Fixed the run script to include command-line arguments

GUI

• All new integrated graphical interface with unified ROI Editor, Atlas Editor, and 3D viewer in separate tabs alongside the controls panel
• ROI Editor
  • Overview plot zooming scales to the size of the ROI
  • Orange border highlights the ROI z-plane corresponding to the overview plots
  • Shows similar pixel information as in the Atlas Editor
  • Layout now respects labels
  • Title now specifies axes
  • Title and empty ROI plots compatible with dark theme
  • Halve the size of truth blobs in verification plots to avoid obscuring the underlying image
  • Easier to flag detections in the ROI Editor
• GUI image loading
  • Load sub-images through the GUI
  • Fixed loading the GUI without an image
  • Fixed loading images through the GUI
  • Fixed image coordinate limits after loading an image
• Region selection
  • Option to select multiple regions, separating IDs by ,
  • Selecting a region ID shows its basic measurements
  • Fixed potential for label boundaries to exceed image boundaries
• Isotropic rescaling is turned on by default for 3D visualizations and incorporates image resolution
• NaN values can be used for invisible pixels, which reduces opacification when highlighting an atlas label in the ROI Editor
• 3D surface rendering is now default
• Unified save button for all viewers
• Auto-select a channel when restoring an ROI based on saved blobs' channel
• Fixed hang when opening the Atlas Editor for large images (downsamples images if necessary)
• Fixed error when showing an image with the z-offset set to the maximum value
• Fixed clearing picked colors during atlas painting
• Fixed the aspect ratio for images rotated 90 degrees
• Fixed display of RGB values for labels
• Fixed over darking some text boxes when hovering
• Fixed blob alignment in 3D surface rendering

CLI

• The CLI now serves as a unified entry point to the Command Line Interface, incuding register, plot_2d, and other tasks,
• The CLI can be accessed through the run_cli.py script, which benefits from the environment setup in run.py without loading a window
• --proc preprocess option to preprocess whole images by tasks specified in a preprocess ROI profile setting
• Many more sample commands for common tasks
• Better support for using the sample commands script without modification
• Help information added for command-line arguments
• Command-line arguments are checked for valid options when available
• Task to export image planes to separate files (--proc export_planes)
• Fixed sample commands for sub-images
• Fixed sample commands path setup for older versions of Bash (< 4.3)

Atlas refinement

• Resize images using the --size argument as an alternative to a profile setting
• Atlas operations expecting symmetry have been generalized across any axis
• Apply adaptive histogram equalization (access as a preprocessing task)
• Records total volumes of atlas and labels during atlas import
• Fixed rotation with resizing for non-z axes
• Fixed storing image plane boundaries for label contour interpolation

Atlas registration

• The atlas is assumed to be pre-imported, which avoids redundant atlas import tasks
• Support for more pre-registration atlas pre-processing tasks, such as 3D rotation, inversion, cropping, and rescaling
• Saves a truncated labels and pre-curated images only if the corresponding options are set
• Option to rescale units (eg mm to microns)
• Settings are customizable for each registration transformation task (eg translation, affine) rather than globally
• Defaults to increased b-spline iterations
• Fixed display of images through SimpleITK after registration

Volumetric image processing

• Grid Search profiles
  • Configurable as YAML files
  • Fixed Grid Searches without sub-image parameters
• Provides basic detection accuracy stats when saving blobs
• Option for whole-image contrast limited adaptive histogram equalization (using scikit-image)
• Fixed retrieving saved ROIs from the database

I/O

• Import from a directory of images
  • Option to convert RGB images to grayscale when importing images from a directory
  • Import multi-channel images
  • Fixed importing large images
• Option to specify output paths when importing a image directory or multi-page TIFF
• Can load multi-page TIFF files without channel indicator
• Defaults to saving figures as PNG
• Library functions for listing, downloading, and uploading files in AWS S3
• Library functions for de/compressing and testing files using ZSTD
• Allows loading the main image from a registered image path
• Avoids loading a sub-image when saving it to avoid a hang
• Sample YAML profiles for blob detection, registration
• profiles folder is checked automatically when loading YAML profiles

Python stats and plots

• Command-line options for configuring markers (--plot_labels marker) and annotation columns (--plot_labels annot_col)

R stats and plots

• Simple R script to load and run stats

Code base and docs

• Licensed under the BSD-3 open source license
• Moved AWS cloud management to a separate document

Dependency Updates

Python Dependency Changes

• Javabridge custom binary updated for import fix on MacOS
• Removed redundant PyQt5 installation during Conda installs
• Workaround for VTK 9 incompatibility with currently Mayavi dependency

MagellanMapper v1.2.2

GUI

• Fixed ignoring the Atlas Editor opacity slider setting when scrolling through planes

CLI

• Sample commands script uses a separate file for paths to allow updating the command script without overwriting users' custom paths
• Files are found from custom data directories more reliably

Atlas registration

• Fixed preserving negative labels

MagellanMapper v1.2.1

Changes

GUI

• Atlas Editor uses command-line specified alpha values

CLI

• Option to load microscope and atlas profiles through YAML files
• Script with sample commands for various common tasks

Atlas refinement

• Option to turn off negative labeling used to distinguish hemispheres
• The origin will update after atlas cropping
• Cleaned up profile settings for label edge extension and mirroring

I/O

• Fixed image stack export to use the new sub-image settings
• Applied workaround for exporting image labels where the first labeled plane is empty

Dependency Additions

• PyYaml (already included with other dependencies)

MagellanMapper v1.2.0

Installation

• These release notes are now included in the repository

GUI

• Setting transparency in overlaid images is more configurable

CLI

• Multiprocessing supported cross-platform by adding spawn support for Windows, which is also available and the new default in Mac; currently implemented for blob detection and image downsampling
• Fixed shebang for most Python files

I/O

• File naming simplifications: removed series string (eg 00000), fixed some extensions (eg NPY instead of NPZ for single value archives), clearer suffixes (eg blobs instead of info_proc)
• Distinguished sub-images (specify by --subimg_offset and --subimg_size) vs ROIs (--offset and --size), which fixes many issues with saving/loading image and blob subsets

Server pipelines

• Tile stitching script uses JAVA_HOME for Java by default
• Fixed attempting to upload to S3 when directory is not set

MagellanMapper v1.1.4

GUI

• Theming capabilities, including default and dark themes
• Workaround for error if 3D points contain invalid values
• Workaround to show labels when the first image displayed has no labels
• Detection sphere slider is larger by making the max value compact
• Fixed settings for overlaid multichannel images, allowing scalars in addition to sequences
• Larger radius to select each ROI Editor draggable circle
• Customize the number of serial 2D plot columns using the --plot_labels layout option
• ROIs without any blobs can be saved
• Truth blobs in verified plots are shown with their actual radius rather than as a fixed size and placed below detection blobs

CLI

• Added "".py"" extension to run script to allow cross-platofrm launch by double-click
• Fixed detecting and activating Conda and Venv environments in the run script
• Explicitly set truth database takes priority over the default path

Atlas registration

• No longer specifies left/right for each label since the laterality may change with mirroring
• Test profile without any registration iterations
• Alternate output paths can be given as directories in addition to full paths
• Fixed regression in generating a new atlas

Nuclei detection

• Simplified setup for varying values at a single index during a grid search of a hyperparameter as an array
• Total variation denoising weight is configurable

I/O

• Default zoom and magnification values are now 1 rather than -1, and total magnification is always shown as a non-negative value

Server pipelines

• Option to specify the plane orientation

Python stats and plots

• Shift ROC legends to the lower right to reduce the chance of obscuring data points

MagellanMapper v1.1.3

Changes

GUI

• Open multiple Atlas Editors at the same time, including synchronized annotation updates
• Fixed inability to open an ROI Editor when a non-editable window is open
• Fixed error when attempting to show 3D blob locations without any blobs
• Fixed error when showing some ROI Editor overview images

Nuclei detection

• Grid search hyperparameters groups have been reorganized into selectable profiles
• Fixed running grid searches/ROC curves with Pandas 1.0
• Unsharp filtering and erosion can be turned off during image preprocessing
• The vmin saturation settings is now configurable, similar to the vmax setting
• The lower threshold factor for max scaling is now configurable to reduce false detections in low signal areas
• Microscope profiles for minimal preprocessing and low resolution images

Additional Assets

Sample 3D microscopy nuclei-stained region at cell resolution and downsampled whole brain are included here as separate assets

MagellanMapper v1.1.2

Changes

Installation

• No longer requires Git, C compiler, or preinstalled Java
• Startup script attempts to activate an environment if necessary, allowing the script to work from a double-click
• Moved installation docs to a separate file and added table of specialized dependency requirements
• Fixed Venv environment setup script
• Fixed SimpleElastix Windows build script to use the Conda environment Python executable if available

GUI

• Performance enhancement when viewing labeled images, especially for images with many colors
• Shows path of image loaded during startup
• Allows loading the ROI Selector without any image
• Fixed VTK error window display on Windows

CLI

• Removed unnecessary pipelines script options
• Fixed shutdown error on Windows and explicitly shut down the JVM if necessary
  Atlas refinement:
• Label fill function now interpolates more smoothly and no longer adjusts the originally edited planes

Nuclei detection

• Fixed error when no blobs are found during verification against truth sets

I/O

• Applies metadata after importing image to make it immediately available for use
• Fixed regression in loading registered images

Dependency Updates

Python Dependency Changes

• OpenJDK 8 is installed through the Conda pathway
• Uses prebuilt Javabridge

Server dependency Changes

• Git is no longer required since not accessing Javabridge from Git
• C compiler is no longer necessary as long as prebuilt Javabridge works
• Java is no longer needs to be preinstalled if following Conda pathway

MagellanMapper v1.1.1

Changes

GUI

• show the file path in the Atlas Editor window
• fixed loss of label visibility after setting the opacity to 0 and changing planes

Atlas refinement

• profile for Allen CCFv3
• use the resolution of the fixed image during registration if the initial registration fails to start

I/O

• use FFMpeg for export to MP4 files for smaller files that can be opened by QuickTime
• fixed TIFF import to match only files with TIFF extensions (eg .tif or .tiff)
• fixed loading images by SimpleITK to select a single channel

Python stats and plots

• option for image padding size

MagellanMapper v1.1.0

Changes

Installation

• Changed name from "Clrbrain" to "MagellanMapper"
• Default virtual environment name is mag
• Support new style Conda initialization during Conda environment setup
• Update reference Conda environment and Pip package specs

Atlas refinement

• Default to use symmetric colormaps rather than separate colors for labels on opposite hemispheres
• Export RGB values with colored cells in Excel file of region IDs for a given atlas
  Atlas registration:
• Fix b-spline grid voxels setting in NCC profile

Python stats and plots

• Merge Excel files into separate sheets of single Excel file

Code base

• Reorganized all source modules into subpackages
• Moved all shell scripts in root folder into bin folder

MagellanMapper v1.0.0

Changes

Installation

• Updated requirements and environment scripts

GUI

• More zoom levels in ROI editor
• Customize position of ROI with overview plots

CLI

• More CSV manipulations by command-line: append columns to a CSV, normalize metrics
• Generate generic line plots including error bars

Atlas refinement

• Increase default smoothing for P14 atlas

Atlas registration

• Increase bspline grid size when using correlation coefficient

I/O

• Specify alternate metadata files by --meta command-line argument
• Load blobs directly rather than through image setup

Python stats and plots

• Nuclei clustering stats using DBSCAN and k-nearest-neighbors from scikit-learn
• further customize generic bar plots
• Option to draw a horizontal line on bar plots based on a summary function
• Option to show labels as a binary image
• Fix DSC measurements for labels

R stats and plots

• Record n for each subgroup
• Profile for basic stats

Dependency Updates

Python Dependency Changes

• Scikit-learn for clustering