Gb/bc debug #232
Merged
Gb/bc debug #232
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…thout pre loading of data
…dataset() which will default to a dask array manager if chunks is specified and load into memory as numpy arrays in chunks is None.
bnb32
approved these changes
Oct 21, 2024
| @@ -232,7 +232,7 @@ def pre_load(self): | |||
| recommended to improve performance with the new sup3r data handler | |||
| access patterns""" | |||
| super().pre_load() | |||
| if hasattr(self.bias_fut_dh.data, 'compute'): | |||
| if hasattr(self.bias_fut_dh, 'compute'): | |||
There was a problem hiding this comment.
Needing to account for the rex handlers not having a data attribute is another reason to default to using the Loaders eventually. This will also allow us to remove the pre load methods that just check if theres a compute method or not (aka rex handler or sup3r handler)
|
|
||
|
|
||
| @pytest.mark.parametrize('compute', (False, True)) | ||
| def test_sup3rdataset_slicing(compute): |
| >>> xds = xr.Dataset(...) | ||
| >>> ds = Sup3rDataset(hourly=xds) | ||
| >>> ds.hourly # returns Sup3rX object | ||
| >>> ds[feature] # returns a single dataarray |
There was a problem hiding this comment.
Good additions. The Sup3rX vs Sup3rDataset stuff is the most complicated part of the refactor imo but looks like it didn't take too long to figure out.
Bnb/compute on none chunks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.