Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for parallelism & memmaping #227

Open
3 tasks
keflavich opened this issue Aug 3, 2015 · 3 comments
Open
3 tasks

Better support for parallelism & memmaping #227

keflavich opened this issue Aug 3, 2015 · 3 comments
Assignees

Comments

@keflavich
Copy link
Contributor

There are a few problems with how SpectralCube is set up for use with, e.g., joblib:

  • SpectralCube.base should return the base of its _data
  • __getitem__ should not use filled_data
  • but, in order for __getitem__ to return meaningful data, we should have OneDSpectrum and Projection and Slice implement masking

@astrofrog - at this point, am I pretty much asking for nddata here? Can nddata do any of this stuff? Are we working in parallel with orthogonal to any other groups within the astropy collaboration?

If we get all this working, it will become easy to make some of the slicewise iterated things operate in parallel!

@keflavich keflavich self-assigned this Aug 3, 2015
@keflavich
Copy link
Contributor Author

On the base issue, it is needed to support this function

@astrofrog
Copy link
Member

@keflavich - since we originally started SpectralCube, NDData has changed a lot (much more lightweight now) so it might be worth investigating whether we can start to merge back in that direction and solve some of these issues in combination with efforts there.

@keflavich
Copy link
Contributor Author

@astrofrog finally looking at this again. So, should SpectralCube inherit from NDData and NDSlicingMixin? Presumably LowerDimensionalObject should, since that will save us writing new masking routines, but it looks like SpectralCube already implements nearly everything the base NDData and mixins provide, and SpectralCube does it differently for good reasons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants