Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to swap dataset longitude axis orientation #145

Merged
merged 2 commits into from
Jan 18, 2022

Conversation

tomvothecoder
Copy link
Collaborator

@tomvothecoder tomvothecoder commented Nov 10, 2021

Description

This PR adds the ability to swap the axis orientation of longitude coordinates and bounds in the Dataset, either upon opening via xcdat.open_dataset/xcdat.open_mfdataset or as an independent function call. It also handles the prime meridian cell when converting from the 180 axis orientation to 360 by splitting it into east and west parts (reuses _align_lon_bounds_to_360() from the spatial averaging feature).

Summary of Changes

  • Add swap_lon_axis() to axis.py
    • Supports axis orientations [-180, 180] and [0, 360]
    • Extendable to support subsets and wrapping beyond orientation min/max
  • Add lon_orient kwarg to open_dataset() and open_mfdataset() for specifying orientation upon opening a Dataset
    • Default behavior is to sort values in ascending order
  • Add _align_lon_to_360() to handle prime meridian cell for the entire Dataset
  • Move _align_lon_bounds_to_360() from spatial_avg.py to axis.py
    • Extract helper function _get_prime_meridian_index() from _align_lon_bounds_to_360()
    • Update _get_longitude_weights() to use _get_prime_meridian_index()

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules

If applicable:

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass with my changes (locally and CI/CD build)
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have noted that this is a breaking change for a major release (fix or feature that would cause existing functionality to not work as expected)

@tomvothecoder tomvothecoder self-assigned this Nov 10, 2021
tests/test_bounds.py Outdated Show resolved Hide resolved
@tomvothecoder
Copy link
Collaborator Author

TODO @pochedls: add comments from meeting about covering some axis orientation edge cases and tag Jiwoo.

@tomvothecoder tomvothecoder marked this pull request as ready for review November 10, 2021 23:05
@codecov-commenter
Copy link

codecov-commenter commented Nov 10, 2021

Codecov Report

Merging #145 (f97f4b1) into main (201a575) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##              main      #145    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files            8         8            
  Lines          381       481   +100     
==========================================
+ Hits           381       481   +100     
Impacted Files Coverage Δ
xcdat/__init__.py 100.00% <100.00%> (ø)
xcdat/axis.py 100.00% <100.00%> (ø)
xcdat/bounds.py 100.00% <100.00%> (ø)
xcdat/dataset.py 100.00% <100.00%> (ø)
xcdat/spatial_avg.py 100.00% <100.00%> (ø)
xcdat/xcdat.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77ff299...f97f4b1. Read the comment docs.

@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from 4a9c54a to 6508ac2 Compare November 11, 2021 19:12
@pochedls
Copy link
Collaborator

TODO @pochedls: add comments from meeting about covering some axis orientation edge cases and tag Jiwoo.

@tomvothecoder and @lee1043 - I've noted the CDAT behavior here. This has some advantages. If you purely convert the bounds to 0-360 units, then a bound that spans the prime meridian has a range that may be unexpected (e.g., [-1, 1] becomes [359, 1]). This has created headaches for spatial averaging (since the weight in each dimension scales as the difference between the bounds; 1 - -1 = 2, but 1 - 359 = -358).

@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from 6508ac2 to 614171c Compare November 15, 2021 22:20
@tomvothecoder
Copy link
Collaborator Author

If you purely convert the bounds to 0-360 units, then a bound that spans the prime meridian has a range that may be unexpected (e.g., [-1, 1] becomes [359, 1]). This has created headaches for spatial averaging (since the weight in each dimension scales as the difference between the bounds; 1 - -1 = 2, but 1 - 359 = -358).

We can pull some similar code from #152 to handle this edge case at the Dataset level

@chengzhuzhang
Copy link
Collaborator

chengzhuzhang commented Nov 15, 2021

We can pull some similar code from #152 to handle this edge case at the Dataset level

I was thinking the same, they are useful as general purpose functionality. :)

Copy link
Collaborator

@pochedls pochedls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is functionality that I have not used, but I suspect that users would want one change: a reordering of the longitude axis after the longitude units are converted. I suspect that a lot of what you would want to do might be complicated if you go from [-180, -179, ..., -1, 0, 1, ..., 179] to [180, 181, ..., 359, 0, 1, ..., 179] (e.g., plotting). There would ideally be a third step that also reorders the longitude axis (and the corresponding dataarrays and bounds) to [0, 1, ..., 179, 180, 181, ..., 359].

Bounds are a special issue. The example above would have bounds of (notice the weird first bound):

lon_bnds = [[359.5, 0.5],
            [0.5, 1.5],
            ...,
            [178.5, 179.5],
            [179.5, 180.5],
            [180.5, 190.5], 
            ...,
            [358.5, 359.5]]

CDAT breaks this first bound up making the longitude axis go from length 360 to 361 (see added/modified values marked with asterisks): [0, 1, ..., 179, 180, 181, ..., 359, *360*] and

lon_bnds = [*[0, 0.5]*,
            [0.5, 1.5],
            ...,
            [178.5, 179.5],
            [179.5, 180.5],
            [180.5, 190.5], 
            ...,
            [358.5, 359.5], 
            *[359.5, 360.5]*]

This is what is done in #152.

The corresponding data array (e.g., tas) would simply repeat the values for the first and last longitude. I think CDAT might actually modify the first and last longitude coordinate value, too (e.g., [*0.5*, 1, ..., 179, 180, 181, ..., 359, *359.5*]).

@lee1043 - could you comment here on this PR (and let me know if you agree with this comment).

@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from 614171c to d214a74 Compare November 17, 2021 23:48
@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch 3 times, most recently from d8e1094 to 23d0d3a Compare December 22, 2021 19:35
@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from c491e90 to da94939 Compare January 4, 2022 17:55
tests/test_axis.py Outdated Show resolved Hide resolved
xcdat/dataset.py Outdated Show resolved Hide resolved
xcdat/dataset.py Outdated Show resolved Hide resolved
@tomvothecoder
Copy link
Collaborator Author

tomvothecoder commented Jan 4, 2022

Hey @pochedls this PR is ready for review. @lee1043 and @chengzhuzhang, if you want to test out this feature that'd be nice too.

After checking out the branch, there are two ways to test it:

  1. Using xcdat.open_dataset()/xcdat.open_mfdataset() with kwarg lon_orient set to 180 or 360
    import xcdat
    
    ds = xcdat.open_dataset("path/to/file.nc", lon_orient=360)
    
  2. Open dataset (can use xcdat or xarray) and pass object to xcdat.swap_lon_axis()
    import xcdat
    
    ds = xcdat.open_dataset("path/to/file.nc")
    # `sort_ascending` is optional
    ds = xcdat.swap_lon_axis(ds, to=360, sort_ascending=True)
    

@lee1043
Copy link
Collaborator

lee1043 commented Jan 4, 2022

Hey @pochedls this PR is ready for review. @lee1043 and @chengzhuzhang, if you want to test out this feature that'd be nice too.

After checking out the branch, there are two ways to test it:

1. Using `xcdat.open_dataset()`/`xcdat.open_mfdataset()` with kwarg `lon_orient` set to 180 or 360
   ```
   import xcdat
   
   ds = xcdat.open_dataset("path/to/file.nc", lon_orient=360)
   ```

2. Open dataset (can use xcdat or xarray) and pass object to `xcdat.swap_lon_axis()`
   ```
   import xcdat
   
   ds = xcdat.open_dataset("path/to/file.nc")
   # `sort_ascending` is optional
   ds = xcdat.swap_lon_axis(ds, to=360, sort_ascending=True)
   ```

@tomvothecoder it is great that this capability has been added, thanks! I like that there are 2 ways of using it. I have mostly used above way 1 in PMP, but have noticed that in some case I needed above way 2 to consider different areas with one open file (e.g., one area 60 to 120, another -30 to 30).

Is there any chance that parameters for lon_orient or to of xcdat.swap_lon_axis() to be more articulated? I am thinking of [0, 360] or [-180, 180], instead of 360 or 180, but open to any suggestion.

@pochedls
Copy link
Collaborator

pochedls commented Jan 4, 2022

I haven't reviewed this code yet, but the syntax looks fine for my use cases.

@lee1043 is suggesting we should do a bit more to orient and subset in the same function call. An even more advanced version of this was suggested here - it was noted we should even be able to "tile" the grid (e.g., -180 to 360 which would repeat part of the grid).

@tomvothecoder - should this all be lumped together or should the subsetting be a future feature?

@lee1043
Copy link
Collaborator

lee1043 commented Jan 4, 2022

@lee1043 is suggesting we should do a bit more to orient and subset in the same function call. An even more advanced version of this was suggested here - it was noted we should even be able to "tile" the grid (e.g., -180 to 360 which would repeat part of the grid).

@pochedls thanks for your comment. I was not thinking that to parameter should be more flexible like [-180, 360] or any number e.g., [20, 270] or something. I was simply thinking 180 to [-180, 180] and 360 to [0, 360] so it is more straightforward. Although I like the idea of the flexibility, I think I am okay with having it for future feature.

@tomvothecoder tomvothecoder requested review from jasonb5 and removed request for chengzhuzhang and lee1043 January 11, 2022 20:48
Copy link
Collaborator

@jasonb5 jasonb5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from b671abb to 43338ed Compare January 18, 2022 18:29
- Add option to swap longitude axis orientation in `open_dataset()` and `open_mfdataset()`
- Add `swap_lon_axes()` to `__init__.py`

Update ref to `_get_coords()`

Move `swap_lon_axis()` to `spatial_avg.py` to `axis.py`
- Add tests for `swap_lon_axis()` and prime meridian cells
- Add `swap_lon_axis_to` kwarg to `open_dataset()` and `open_mfdataset()`

Move `_align_longitude_to_360_axis` to `axis.py`
- Move related tests to `test_axis.py`
- Rename kwarg `swap_lon_axis_to` to `lon_orient`

Add `_align_lon_to_360()` to handle prime meridian
- Add option for sorting ascending in `swap_lon_axis()`
- Extract helper function `_get_prime_meridian_index()` from `_align_lon_bounds_to_360()`
- Update `_get_longitude_weights()` to use `_get_prime_meridian_index()`

Update docstrings on axis orientations

Update `lon_orient` and `to` kwargs to tuple
- This allows for extension of the swapping feature for subsets
@tomvothecoder tomvothecoder force-pushed the feature/135-axis-orientation branch from 43338ed to 51f6cc4 Compare January 18, 2022 18:32
@tomvothecoder tomvothecoder merged commit 194f84d into main Jan 18, 2022
@tomvothecoder tomvothecoder deleted the feature/135-axis-orientation branch January 18, 2022 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New enhancement request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option to specify longitude axis system when opening up a dataset
6 participants