Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support selecting specific channels for match-based colocalization #451

Merged
merged 6 commits into from
Mar 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 73 additions & 4 deletions bin/sample_cmds_bash.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,15 @@
]
},
{
"cell_type": "raw",
"cell_type": "code",
"execution_count": null,
"id": "5ff91655-82d3-4007-8854-fda802a298b9",
"metadata": {},
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"pip install jupyterlab\n",
"pip install bash_kernel\n",
Expand All @@ -46,9 +52,15 @@
]
},
{
"cell_type": "raw",
"cell_type": "code",
"execution_count": null,
"id": "f28f8b69-fd31-4922-b371-a8e4edfa782d",
"metadata": {},
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"jupyter-lab"
]
Expand Down Expand Up @@ -521,6 +533,63 @@
"TODO"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ce3ca4c9",
"metadata": {},
"source": [
"### Channel colocalization\n",
"\n",
"Blobs detected in several channels may represent the same cells. To find which blobs are colocalized across channels, MagellanMapper offers several colocalization detection methods:\n",
"* Intensity-based colocalization\n",
"* Match-based colocalization\n",
"\n",
"#### Intensity-based colocalization\n",
"\n",
"For each blob detected in one channel, the corresponding positions are checked in all remaining channels. Channels above threshold at that position are considered to be colocalized with the blob. This process is performed during blob detection since it uses the same preprocessed versions of the image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4cba0b3",
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"mm \"$img\" --proc detect_coloc --roi_profile lightsheet"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8a935d2b",
"metadata": {},
"source": [
"#### Match-based colocalization\n",
"\n",
"After detecting blobs in each channel, blobs in one channel are paired with blobs in another channel. Matches are optimized by distance to maximize the number of pairs. This approach does not rely on thresholding and be performed after the detection step.\n",
"\n",
"The ROI profile specifies the block sizes used for colocalization. Only the first ROI parameter will be used (ie for channel 0). Relevant parameters are:\n",
"* `segment_size`: adjusts block size, where larger blocks reduce the number of blocks to process but require more memory\n",
"* `prune_tol_factor`: adjusts the overlap between blocks\n",
"* `verify_tol_factor`: tolerance for a match, where larger values allow blobs farther apart to be considered matches\n",
"* `resize_blobs`: rescales blob coordinates, which may be important for anisotropic images"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ed0c16b6",
"metadata": {},
"source": [
"mm \"$img\" --proc coloc_match --roi_profile lightsheet"
]
},
{
"cell_type": "markdown",
"id": "92bedce5-8fec-4310-8d18-f56671eb6f3b",
Expand Down
1 change: 1 addition & 0 deletions docs/release/release_v1.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@

- Match-based colocalization can run without the main image, using just its metadata instead (#117)
- These colocalizations are now displayed in the 3D viewer (#121)
- Specific match-based colocalizations channels can be set, eg `--channels 0 2` (#451)
- Fixed match-based colocalizations when no matches are found (#117, #120)
- Fixed slow loading of match-based colocalizations (#119, #123)

Expand Down
101 changes: 63 additions & 38 deletions magmap/cv/colocalizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,60 +169,75 @@ class StackColocalizer(object):
pickling in forked multiprocessing.

"""
blobs = None
match_tol = None
blobs: Optional["detector.Blobs"] = None
match_tol: Optional[Sequence[float]] = None
#: Channels to match; defaults to None.
channels: Optional[Sequence[int]] = None

@classmethod
def colocalize_block(cls, coord, offset, shape, blobs=None,
tol=None, setup_cli=False):
def colocalize_block(
cls, coord: Sequence[int], offset: Sequence[int],
shape: Sequence[int], blobs: Optional["detector.Blobs"] = None,
tol: Optional[Sequence[float]] = None, setup_cli: bool = False,
channels: Optional[Sequence[int]] = None
) -> Tuple[Sequence[int], Dict[Tuple[int, int], "BlobMatch"]]:
"""Colocalize blobs from different channels within a block.

Args:
coord (Tuple[int]): Block coordinate.
offset (List[int]): Block offset within the full image in z,y,x.
shape (List[int]): Block shape in z,y,x.
blobs (:obj:`np.ndarray`): 2D blobs array; defaults to None to
use :attr:`blobs`.
tol (List[float]): Tolerance for colocalizing blobs; defaults
coord: Block coordinate.
offset: Block offset within the full image in z,y,x.
shape: Block shape in z,y,x.
blobs: Blobs; defaults to None, which will use :attr:`blobs`.
tol: Tolerance for colocalizing blobs; defaults
to None to use :attr:`match_tol`.
setup_cli (bool): True to set up CLI arguments, typically for
setup_cli: True to set up CLI arguments, typically for
a spawned (rather than forked) environment; defaults to False.
channels: Channels to match; defaults to None, where
:attr:`channels` will be used.

Returns:
Tuple[int], dict[Tuple[int], Tuple]: ``coord`` for tracking
multiprocessing and the dictionary of matches.
Tuple of:
- ``coord``: ``coord``, for tracking multiprocessing
- ``matches``: dictionary of matches

"""
if blobs is None:
blobs = cls.blobs
if tol is None:
tol = cls.match_tol
if channels is None:
channels = cls.channels
if setup_cli:
# reload command-line parameters
cli.process_cli_args()
_logger.debug(
"Match-based colocalizing blobs in ROI at offset %s, size %s",
offset, shape)
matches = colocalize_blobs_match(blobs, offset[::-1], shape[::-1], tol)
matches = colocalize_blobs_match(
blobs, offset[::-1], shape[::-1], tol, channels=channels)
return coord, matches

@classmethod
def colocalize_stack(cls, shape, blobs):
def colocalize_stack(
cls, shape: Sequence[int], blobs: "detector.Blobs",
channels: Optional[Sequence[int]] = None
) -> Dict[Tuple[int, int], "BlobMatch"]:
"""Entry point to colocalizing blobs within a stack.

Args:
shape (List[int]): Image shape in z,y,x.
blobs (:obj:`np.ndarray`): 2D Numpy array of blobs.
shape: Image shape in z,y,x.
blobs: Blobs.
channels: Channels to match; defaults to None.

Returns:
dict[tuple[int, int], :class:`BlobMatch`]: The
dictionary of matches, where keys are tuples of the channel pairs,
and values are blob match objects.
Dictionary of matches, where keys are tuples of the channel pairs,
and values are blob match objects.

"""
chls = ("each pair of channels" if channels is None
else f"channels: {channels}")
_logger.info(
"Colocalizing blobs based on matching blobs in each pair of "
"channels")
"Colocalizing blobs based on matching blobs in %s", chls)
# scale match tolerance based on block processing ROI size
blocks = stack_detect.setup_blocks(config.roi_profile, shape)
match_tol = np.multiply(
Expand Down Expand Up @@ -256,12 +271,12 @@ def colocalize_stack(cls, shape, blobs):
args=(coord, offset, shape)))
else:
# pickle full set of variables
blobs_roi = detector.Blobs(detector.get_blobs_in_roi(
blobs.blobs, offset, shape)[0])
pool_results.append(pool.apply_async(
StackColocalizer.colocalize_block,
args=(coord, offset, shape,
detector.get_blobs_in_roi(
blobs, offset, shape)[0], match_tol,
True)))
args=(coord, offset, shape, blobs_roi, match_tol,
True, channels)))

# dict of channel combos to blob matches data frame
matches_all = {}
Expand Down Expand Up @@ -422,19 +437,22 @@ def colocalize_blobs(roi, blobs, thresh=None):


def colocalize_blobs_match(
blobs: np.ndarray, offset: Sequence[int], size: Sequence[int],
tol: Sequence[float], inner_padding: Optional[Sequence[int]] = None
blobs: "detector.Blobs", offset: Sequence[int], size: Sequence[int],
tol: Sequence[float], inner_padding: Optional[Sequence[int]] = None,
channels: Optional[Sequence[int]] = None
) -> Optional[Dict[Tuple[int, int], "BlobMatch"]]:
"""Co-localize blobs in separate channels but the same ROI by finding
optimal blob matches.

Args:
blobs: Blobs from separate channels.
blobs: Blobs.
offset: ROI offset given as x,y,z.
size: ROI shape given as x,y,z.
tol: Tolerances for matching given as x,y,z
inner_padding: ROI padding given as x,y,z; defaults
to None to use the padding based on ``tol``.
channels: Indices of channels to colocalize. Defaults to None, which
will use all channels in ``blobs``.

Returns:
Dictionary where keys are tuples of the two channels compared and
Expand All @@ -443,31 +461,38 @@ def colocalize_blobs_match(
"""
if blobs is None:
return None
thresh, scaling, inner_pad, resize, blobs = verifier.setup_match_blobs_roi(
tol, blobs)
thresh, scaling, inner_pad, resize, blobs_roi = \
verifier.setup_match_blobs_roi(tol, blobs)
if inner_padding is None:
inner_padding = inner_pad
matches_chls = {}
channels = np.unique(detector.Blobs.get_blobs_channel(blobs)).astype(int)
for chl in channels:

# get channels in blobs
blob_chls = np.unique(blobs.get_blobs_channel(blobs_roi)).astype(int)
if channels is not None:
# filter blob channels to only include specified channels
blob_chls = [c for c in blob_chls if c in channels]

for chl in blob_chls:
# pair channels
blobs_chl = detector.Blobs.blobs_in_channel(blobs, chl)
for chl_other in channels:
blobs_chl = blobs.blobs_in_channel(blobs_roi, chl)
for chl_other in blob_chls:
# prevent duplicates by skipping other channels below given channel
if chl >= chl_other: continue
# find colocalizations between blobs from one channel to blobs
# in another channel
blobs_chl_other = detector.Blobs.blobs_in_channel(blobs, chl_other)
blobs_chl_other = blobs.blobs_in_channel(blobs_roi, chl_other)
blobs_inner_plus, blobs_truth_inner_plus, offset_inner, \
size_inner, matches = verifier.match_blobs_roi(
blobs_chl_other, blobs_chl, offset, size, thresh, scaling,
inner_padding, resize)

# reset truth and confirmation blob flags in matches
chl_combo = (chl, chl_other)
matches.update_blobs(detector.Blobs.set_blob_truth, -1)
matches.update_blobs(detector.Blobs.set_blob_confirmed, -1)
matches.update_blobs(blobs.set_blob_truth, -1)
matches.update_blobs(blobs.set_blob_confirmed, -1)
matches_chls[chl_combo] = matches

return matches_chls


Expand Down
Loading