Ravel model for the illusion of 2D tables

[SolarDrew]

Closes #168 (once dkist-inventory is updated to use these changes).

Adds a new model to take a 2D index and return the corresponding correct index for a 1D array, and the inverse model for the reverse operation. To be used as a compound with Tabular1D so that it looks like a Tabular2D but the compound model can still be inverted.

Should be straightforward to expand to 3D also.

[codecov]

Codecov Report

Merging #227 (feed4a9) into main (69415b9) will decrease coverage by 0.03%.
The diff coverage is 96.55%.

@@            Coverage Diff             @@
##             main     #227      +/-   ##
==========================================
- Coverage   97.18%   97.16%   -0.03%     
==========================================
  Files          37       37              
  Lines        2023     2078      +55     
==========================================
+ Hits         1966     2019      +53     
- Misses         57       59       +2     

Impacted Files	Coverage Δ
dkist/wcs/models.py	99.48% <95.12%> (-0.52%)	⬇️
dkist/io/asdf/converters/__init__.py	100.00% <100.00%> (ø)
dkist/io/asdf/converters/models.py	97.95% <100.00%> (+0.39%)	⬆️
dkist/io/asdf/entry_points.py	100.00% <100.00%> (ø)
dkist/io/asdf/tests/test_models.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[Cadair]

pre-commit.ci autofix

[Cadair]

Fixed that for you 😉

[Cadair]

I am in your code, finding your bugs.

I made an asdf, and the first thing I noticed is this:

NotImplementedError: The "separable" property is not defined for model Ravel

In [14]: print(ds)
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[14], line 1
----> 1 print(ds)

File ~/Git/DKIST/dkist/dkist/dataset/dataset.py:224, in Dataset.__str__(self)
    223 def __str__(self):
--> 224     return dataset_info_str(self)

File ~/Git/DKIST/dkist/dkist/dataset/utils.py:92, in dataset_info_str(ds)
     83 s += (' ' * world_dim_width + '  ' +
     84         ('{0:^' + str(wcs.pixel_n_dim * 5 - 2) + 's}').format('Pixel Dim') +
     85         '\n')
     87 s += (('{0:' + str(world_dim_width) + 's}').format('World Dim') +
     88         ''.join(['  ' + ('{0:' + str(pixel_dim_width) + 'd}').format(ipix)
     89                 for ipix in range(wcs.pixel_n_dim)]) +
     90         '\n')
---> 92 matrix = wcs.axis_correlation_matrix[::-1, ::-1]
     93 matrix_str = np.empty(matrix.shape, dtype='U3')
     94 matrix_str[matrix] = 'yes'

File ~/.virtualenvs/dkist/lib/python3.11/site-packages/gwcs/api.py:249, in GWCSAPIMixin.axis_correlation_matrix(self)
    239 @property
    240 def axis_correlation_matrix(self):
    241     """
    242     Returns an (`~BaseLowLevelWCS.world_n_dim`,
    243     `~BaseLowLevelWCS.pixel_n_dim`) matrix that indicates using booleans
   (...)
    247     would be `True` and all other entries `False`.
    248     """
--> 249     return separable.separability_matrix(self.forward_transform)

File ~/Git/astropy/astropy/modeling/separable.py:98, in separability_matrix(transform)
     96 if transform.n_inputs == 1 and transform.n_outputs > 1:
     97     return np.ones((transform.n_outputs, transform.n_inputs), dtype=np.bool_)
---> 98 separable_matrix = _separable(transform)
     99 separable_matrix = np.where(separable_matrix != 0, True, False)
    100 return separable_matrix

File ~/Git/astropy/astropy/modeling/separable.py:307, in _separable(transform)
    305 elif isinstance(transform, CompoundModel):
    306     sepleft = _separable(transform.left)
--> 307     sepright = _separable(transform.right)
    308     return _operators[transform.op](sepleft, sepright)
    309 elif isinstance(transform, Model):

File ~/Git/astropy/astropy/modeling/separable.py:302, in _separable(transform)
    287 def _separable(transform):
    288     """
    289     Calculate the separability of outputs.
    290 
   (...)
    299         Each element represents the separablity of the corresponding output.
    300     """
    301     if (
--> 302         transform_matrix := transform._calculate_separability_matrix()
    303     ) is not NotImplemented:
    304         return transform_matrix
    305     elif isinstance(transform, CompoundModel):

File ~/Git/DKIST/dkist/dkist/wcs/models.py:672, in CoupledCompoundModel._calculate_separability_matrix(self)
    670 def _calculate_separability_matrix(self):
    671     sepleft = separable._separable(self.left)
--> 672     sepright = separable._separable(self.right)
    674     noutp = separable._compute_n_outputs(sepleft, sepright)
    676     cleft = np.zeros((noutp, sepleft.shape[1]))

File ~/Git/astropy/astropy/modeling/separable.py:306, in _separable(transform)
    304     return transform_matrix
    305 elif isinstance(transform, CompoundModel):
--> 306     sepleft = _separable(transform.left)
    307     sepright = _separable(transform.right)
    308     return _operators[transform.op](sepleft, sepright)

File ~/Git/astropy/astropy/modeling/separable.py:306, in _separable(transform)
    304     return transform_matrix
    305 elif isinstance(transform, CompoundModel):
--> 306     sepleft = _separable(transform.left)
    307     sepright = _separable(transform.right)
    308     return _operators[transform.op](sepleft, sepright)

File ~/Git/astropy/astropy/modeling/separable.py:310, in _separable(transform)
    308     return _operators[transform.op](sepleft, sepright)
    309 elif isinstance(transform, Model):
--> 310     return _coord_matrix(transform, "left", transform.n_outputs)

File ~/Git/astropy/astropy/modeling/separable.py:200, in _coord_matrix(model, pos, noutp)
    198         mat[-model.n_outputs :, -model.n_inputs :] = m
    199     return mat
--> 200 if not model.separable:
    201     # this does not work for more than 2 coordinates
    202     mat = np.zeros((noutp, model.n_inputs))
    203     if pos == "left":

File ~/Git/astropy/astropy/modeling/core.py:1589, in Model.separable(self)
   1587 if self._separable is not None:
   1588     return self._separable
-> 1589 raise NotImplementedError(
   1590     'The "separable" property is not defined for '
   1591     f"model {self.__class__.__name__}"
   1592 )

NotImplementedError: The "separable" property is not defined for model Ravel

which breaks the repr / str of the dataset class amongst other things. I think it should be a simple case of setting _separable = False.

[Cadair]

I added the fix, because I needed it for further testing.

[SolarDrew]

Well I thought I'd found and fixed that bug myself but thanks I guess 😆