ARINA read

[smribet]

No description provided.

[smribet]

This PR includes

• two new file readers: one for abTEM data and one for arina data
• a h5 file parser
• reformatting of some existing files

Reading 2D calibrations requires the patches in PR 462

[bsavitzky]

Looks good overall! Just a few things that could use editing. There are going to be merge conflicts, I can take a stab at handling those and building tests for the new readers

[bsavitzky]

Here and on line 71, I'd recommend using DiffractionSlice and RealSlice rather than VirtualDiffraction and VirtualImage. The former are the more general classes for diffraction and real space shaped objects, and the latter are these (i.e. they inherit from the Slices) plus are more specifically generated by our virtual imaging and virtual diffraction methods

[bsavitzky]

are there any syntactically meaningful edits to this file, or is it all just reformatting? It appears to be done with an autoformatter...

[smribet]

just autoformatter

[bsavitzky]

same comment as for virtualdiffraction.py

[bsavitzky]

Ok, I've merged the dev head into this branch, fixed consequent merge conflicts and import errors, added the arina test file to the test suite test file downloader with the rest of the io test bundle, and added a unit test file for the arina reader. So the infrastructure for the arina file reader should be in place. Do have have one or two test files you'd like to use for the abTEM reader? (two if you'd like to test both the 4D and 2D use cases). If so we can add those to the test suite as well.

When I ran the arina reader I hit an error - here's the traceback:

    def test_read_arina():

        # read
>       data = py4DSTEM.import_file( filepath )

test_arina.py:13:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../py4DSTEM/io/importfile.py:93: in import_file
    data = read_arina(filepath, mem=mem, binfactor=binfactor, **kwargs)
../../py4DSTEM/io/filereaders/read_arina.py:40: in read_arina
    nimages = nimages + f["entry"]["data"][dset].shape[0]
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
/Users/Ben/opt/anaconda3/envs/py4dstem_dev/lib/python3.10/site-packages/h5py/_hl/group.py:357: in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   KeyError: "Unable to open object (unable to open external file, external link file name = 'STO_STEM_bench_20us_data_000001.h5')"

h5py/h5o.pyx:190: KeyError

Seems like it found and read the master file but then had trouble reading the first data file!

[smribet]

Thanks Ben! I'm having trouble recreating that error you posted. This just worked for me:

test = py4DSTEM.import_file('STO_STEM_bench_20us_master.h5', scan_width = 1000)
test

DataCube( A 4-dimensional array of shape (1000, 1000, 96, 96) called 'datacube',
          with dimensions:

          Rx = [0,1,...] pixels
          Ry = [0,1,...] pixels
          Qx = [0,1,...] pixels
          Qy = [0,1,...] pixels

[smribet]

Added the correction option, so I think this should be ready now