Mixed subtree processing

.github/workflows/publish-sdist.yml

@@ -11,10 +11,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@master
-      - name: Set up Python 3.6
+      - name: Set up Python 3.9
         uses: actions/setup-python@v2
         with:
-          python-version: 3.6
+          python-version: 3.9
       - name: Build a source tarball
         run:
             python setup.py sdist

.github/workflows/run-tox.yml

@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7, 3.8]
+        python-version: ['3.7', '3.8', '3.9', '3.10']
 
     steps:
     - uses: actions/checkout@v2

CHANGELOG.rst

@@ -3,10 +3,35 @@ Changelog
 
 Version 3.2.0
 -------------
+
+- Add ``neurom.features.morphology.length_fraction_above_soma`` feature.
+- List of multiple kwargs configurations are now allowed in``neurom.apps.morph_stats``.
+- ``neurom.features.neurite.principal_direction_extents`` directions correspond to extents
+  ordered in a descending order.
+- Add features ``neurom.features.morphology.(aspect_ration, circularity, shape_factor)```
+- Fix ``neurom.morphmath.principal_direction_extent`` to calculate correctly the pca extent.
+- Fix ``neurom.features.neurite.segment_taper_rates`` to return signed taper rates.
+- Fix warning system so that it doesn't change the pre-existing warnings configuration
+- Fix ``neurom.features.bifurcation.partition_asymmetry`` Uylings variant to not throw
+  for bifurcations with leaves.
+- Fix ``neurom.features.neurite.principal_direction_extents`` to remove duplicate points
+  when calculated.
+- Add ``neurom.features.morphology.volume_density`` feature so that it is calculated
+  correctly when the entire morphology is taken into account instead of summing the per
+  neurite volume densities.
+- Add support for py39 and py310 testing.
+- Fix ``neurom.features.morphology.sholl_frequency`` to return an empty list when a
+  neurite_type that is not present in the morphology is specified.
+- Fix ``neurom.features.morphology.trunk_origin_radii`` to warn and use only the root
+  section for the calculation of the path distances. Edge cases from the combination
+  of ``min_length_filter`` and ``max_length_filter`` are addressed.
+- Fix ``neurom.features.morphology.sholl_frequency`` to use soma center in distance
+  calculation, instead of using the origin.
 - Add ``neurom.features.morphology.trunk_angles_inter_types`` and
   ``neurom.features.morphology.trunk_angles_from_vector`` features, make
   ``neurom.features.morphology.trunk_angles`` more generic and add length filter to
   ``neurom.features.morphology.trunk_origin_radii``.
+- Deprecate python3.6
 - Add doc on spherical coordinates.
 
 Version 3.1.0

doc/source/heterogeneous.rst

@@ -0,0 +1,226 @@
+.. Copyright (c) 2022, Ecole Polytechnique Federale de Lausanne, Blue Brain Project
+   All rights reserved.
+
+   This file is part of NeuroM <https://github.com/BlueBrain/NeuroM>
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions are met:
+
+       1. Redistributions of source code must retain the above copyright
+          notice, this list of conditions and the following disclaimer.
+       2. Redistributions in binary form must reproduce the above copyright
+          notice, this list of conditions and the following disclaimer in the
+          documentation and/or other materials provided with the distribution.
+       3. Neither the name of the copyright holder nor the names of
+          its contributors may be used to endorse or promote products
+          derived from this software without specific prior written permission.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+   ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+   WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+   DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
+   DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+   LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+   ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. _heterogeneous:
+
+Heterogeneous Morphologies
+**************************
+
+.. image:: images/heterogeneous_neuron.png
+
+Definition
+----------
+
+A heterogeneous morphology consists of zero or more homogeneous and at least one heterogeneous neurite trees extending from the soma.
+A heterogeneous neurite tree consists of multiple sub-neurites with different types (ie: basal and axon).
+
+A typical example of a heterogeneous neurite is the axon-carrying dendrite, in which the axon sprouts from the basal dendrite.
+
+
+Identification
+--------------
+
+Heterogeneous neurites can be identified using the ``Neurite::is_heterogeneous`` method:
+
+.. code:: python
+
+    from neurom import load_morphology
+    from neurom.core.morphology import iter_neurites
+
+    m = load_morphology('tests/data/swc/heterogeneous_morphology.swc')
+
+    print([neurite.is_heterogeneous() for neurite in m])
+
+which would return ``[False, True, False]``, meaning the 2nd neurite extending from the soma contains multiple neurite types.
+
+
+Sub-neurite views of heterogeneous neurites
+--------------------------------------------
+
+Default mode
+~~~~~~~~~~~~
+
+NeuroM does not take into account heterogeneous sub-neurites by default.
+A heterogeneous neurite is treated as a homogeneous one, the type of which is determined by the first section of the tree.
+For example:
+
+.. code-block:: python
+
+    from neurom import load_morphology
+    from neurom.core.morphology import iter_neurites
+
+    m = load_morphology('tests/data/swc/heterogeneous_morphology.swc')
+
+    basal, axon_carrying_dendrite, apical = list(iter_neurites(m))
+
+    print(basal.type, axon_carrying_dendrite.type, apical.type)
+
+Prints::
+
+    NeuriteType.basal_dendrite NeuriteType.basal_dendrite NeuriteType.apical_dendrite
+
+I.E. the axon-carrying dendrite would be treated as a basal dendrite.
+For feature extraction and checks, the axon-carrying dendrite is treated as a basal dendrite.
+Features, for which an axon neurite type is passed, do not have access to the axonal part of the neurite.
+For instance, the number of basal and axon neurites will be two and zero respectively.
+A features such as ``total_volume`` would include the entire axon-carrying dendrite, without separating between basal and axon types.
+
+Sub-neurite mode
+~~~~~~~~~~~~~~~~
+
+NeuroM provides an immutable approach (without modifying the morphology) to access the homogeneous sub-neurites of a neurite.
+Using ``iter_neurites`` with the flag ``use_subtrees`` returns a neurite view for each homogeneous sub-neurite.
+
+.. code-block:: python
+
+    basal1, basal2, axon, apical = list(iter_neurites(m, use_subtrees=True))
+
+    print(basal1.type, basal2.type, axon.type, apical.type)
+
+In the example above, two views of the axon-carrying dendrite have been created: the basal and axon dendrite views.
+
+.. image:: images/heterogeneous_neurite.png
+
+Given that the morphology is not modified, the sub-neurites specify as their ``root_node`` the section of the homogeneous sub-neurite.
+They are just references to where the sub-neurites start.
+
+.. note::
+    Creating neurite instances for the homogeneous sub-neurites breaks the assumption of root nodes not having a parent.
+
+
+.. warning::
+    Be careful while using sub-neurites.
+    Because they just point to the start sections of the sub-neurite, they may include other sub-neurites as well.
+    In the figure example above, the basal sub-neurite includes the entire tree, including the axon sub-neurite.
+    An additional filtering of the sections is needed to leave out the axonal part.
+    However, for the axon sub-neurite this filtering is not needed because it is downstream homogeneous.
+
+
+Extract features from heterogeneous morphologies
+------------------------------------------------
+
+Neurite
+~~~~~~~
+
+Neurite features have been extended to include a ``section_type`` argument, which can be used to apply a feature on a heterogeneous neurite.
+
+.. code-block:: python
+
+    from neurom import NeuriteType
+    from neurom import load_morphology
+    from neurom.features.neurite import number_of_sections
+
+    m = load_morphology('tests/data/swc/heterogeneous_morphology.swc')
+
+    axon_carrying_dendrite = m.neurites[1]
+
+    total_sections = number_of_sections(axon_carrying_dendrite)
+    basal_sections = number_of_sections(axon_carrying_dendrite, section_type=NeuriteType.basal_dendrite)
+    axon_sections = number_of_sections(axon_carrying_dendrite, section_type=NeuriteType.axon)
+
+    print(total_sections, basal_sections, axon_sections)
+
+Not specifying a ``section_type`` is equivalent to passing ``NeuriteType.all`` and it will use all sections as done historically.
+
+Morphology
+~~~~~~~~~~
+
+Morphology features have been extended to include the ``use_subtrees`` flag, which allows to use the sub-neurites.
+
+.. code-block:: python
+
+    from neurom import NeuriteType
+    from neurom import load_morphology
+    from neurom.features.morphology import number_of_neurites
+
+    m = load_morphology('tests/data/swc/heterogeneous_morphology.swc')
+
+    total_neurites_wout_subneurites = number_of_neurites(m)
+    total_neurites_with_subneurites = number_of_neurites(m, use_subtrees=True)
+
+    print("A:", total_neurites_wout_subneurites, total_neurites_with_subneurites)
+
+    number_of_axon_neurites_wout = number_of_neurites(m, neurite_type=NeuriteType.axon)
+    number_of_axon_neurites_with = number_of_neurites(m, neurite_type=NeuriteType.axon, use_subtrees=True)
+
+    print("B:", number_of_axon_neurites_wout, number_of_axon_neurites_with)
+
+    number_of_basal_neurites_wout = number_of_neurites(m, neurite_type=NeuriteType.basal_dendrite)
+    number_of_basal_neurites_with = number_of_neurites(m, neurite_type=NeuriteType.basal_dendrite, use_subtrees=True)
+
+    print("C:", number_of_basal_neurites_wout, number_of_basal_neurites_with)
+
+Prints::
+
+    A: 3 4
+    B: 0 1
+    C: 2 2
+
+In the example above, the total number of neurites increases from 3 to 4 when the subtrees are enabled (see ``A`` in the print out.)
+This is because the axonal and basal parts of the axon-carrying dendrite are counted separately in the second case.
+
+Specifying a ``neurite_type``, allows to count sub-neurites.
+Therefore, the number of axons without subtrees is 0, whereas it is 1 when subtrees are enabled (see ``B`` in the print out.)
+However, for basal dendrites the number does not change (2) because the axon-carrying dendrite is perceived as basal dendrite in the default case (see ``C``.)
+
+features.get
+~~~~~~~~~~~~
+
+``features.get`` can be used with respect to what has been mentioned above for neurite and morphology features.
+
+.. code-block:: python
+
+    from neurom import features
+    from neurom import load_morphology
+
+    m = load_morphology('tests/data/swc/heterogeneous_morphology.swc')
+
+    features.get("number_of_neurites", m, use_subtrees=True)
+    features.get("number_of_sections", m, section_type=NeuriteType.axon)
+
+Conventions & Incompatibilities
+-------------------------------
+
+Heterogeneous Forks
+~~~~~~~~~~~~~~~~~~~
+
+A heterogeneous bifurcation/fork, i.e. a section with children of different types, is ignored when features on bifurcations are calculated.
+It is not meaningful to calculate features, such as bifurcation angles, on transitional forks where the downstream subtrees have different types.
+
+Incompatible features with subtrees
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following features are not compatible with subtrees:
+
+* trunk_origin_azimuths
+* trunk_origin_elevations
+* trunk_angles
+
+Because they require the neurites to be rooted at the soma.
+This is not true for sub-neurites.
+Therefore, passing a ``use_subtrees`` flag will result in an error.

doc/source/index.rst

@@ -46,6 +46,7 @@ NeuroM is a Python-based toolkit for the analysis and processing of morphologies
    features
    spherical_coordinates
    examples
+   heterogeneous
    cli
    definitions
    api

doc/source/morph_stats.rst

@@ -58,7 +58,7 @@ Short format (prior version 3.0.0)
 An example config:
 
 .. code-block:: yaml
-    
+
     neurite:
         section_lengths:
             - max
@@ -67,13 +67,13 @@ An example config:
             - total
         section_branch_orders:
             - max
-    
+
     neurite_type:
         - AXON
         - APICAL_DENDRITE
         - BASAL_DENDRITE
         - ALL
-    
+
     neuron:
         soma_radius:
             - mean
@@ -93,7 +93,7 @@ function, e.g.
 * ``raw``: array of raw values
 * ``max``, ``min``, ``mean``, ``median``, ``std``: self-explanatory.
 * ``total``: sum of the raw values
-  
+
 An additional field ``neurite_type`` specifies the neurite types into which the morphometrics
 are to be split. It applies only to ``neurite`` features. A sample output using the above
 configuration:
@@ -198,6 +198,65 @@ So the example config from `Short format (prior version 3.0.0)`_ looks:
                - mean
 
 
+List of features format (starting version 3.2.0)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The kwargs entry was converted into a list to allow running the same feature with different arguments. The ``partition_asymmetry`` feature in the example above can be specified multiple times with different arguments as follows:
+
+.. code-block:: yaml
+
+    neurite:
+      partition_asymmetry:
+        kwargs:
+        - method: petilla
+          variant: length
+        - method: uylings
+          variant: branch-order
+        modes:
+        - max
+        - sum
+
+To allow differentiation between the feature multiples, the keys and values of the kwargs are appended at the end of the feature name:
+
+.. code-block::
+
+    partition_asymmetry__variant:length__method:petilla
+    partition_asymmetry__variant:branch-order__method:uylings
+
+The example config from `Short format (prior version 3.0.0)`_ becomes:
+
+.. code-block:: yaml
+
+    neurite:
+      section_branch_orders:
+        kwargs:
+        - {}
+        modes:
+        - max
+      section_lengths:
+        kwargs:
+        - {}
+        modes:
+        - max
+        - sum
+      section_volumes:
+        kwargs:
+        - {}
+        modes:
+        - sum
+    morphology:
+      soma_radius:
+        kwargs:
+        - {}
+        modes:
+        - mean
+    neurite_type:
+    - AXON
+    - APICAL_DENDRITE
+    - BASAL_DENDRITE
+    - ALL
+
+
 Features
 --------