Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 1.0: North Atlantic Update - Tides, Curved Boundaries, Config, and More!😀🤯🥳 #193

Open
wants to merge 203 commits into
base: main
Choose a base branch
from

Conversation

manishvenu
Copy link
Collaborator

@manishvenu manishvenu commented Oct 11, 2024

Authors: @ashjbarnes, @manishvenu, @helenmacdonald

Description:

Ashley and I worked together to add a few features to RM6. While we get the changes ready to merge with the main COSIMA branch, please check out this draft with a list of the changes below.

Major Features:

  1. Accepting curved boundary supergrids and bathymetry (Add angled grids to RMOM6 CROCODILE-CESM/regional-mom6#13)
  2. Implement Boundary Tides from OSU's TPXO using GFDL NWA25 Code (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  3. Import experiment configuration from a light config JSON (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  4. Major bathymetry fixes for land-sea mask (Various bathymetry improvements / bugfixes CROCODILE-CESM/regional-mom6#17)
  5. Moving experiment specific changes to MOM_override (All OBC, Tides, and other minor changes) and out of MOM_input
  6. New GLORYS downloader (that can account for curved boundaries) (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  7. Tests: Additional tests (tides. rotation, and regridding), Conftest.py, Add to Docker Image (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34)
  8. Changing function names to verb beginnings (initial_condition -> setup_initial_condition) (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  9. Add properties (i.e. more functional object variables) and dataset plotting (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  10. Add Rotation: Add MOM6 angle calculation at t points (MOM6 doesn't use an input hgrid's angle_dx field. It calculate the angle internally at t-points) & two angle approximation methods for the boundary q/u/v points (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34)
  11. Add Land Mask (Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33)

Minor Features:

  1. Ability to code without greek letters
  2. Removed unintentional machine-specific code
  3. Increased print statements

Breaking Changes (for backwards compatibility):

  1. Functions changed names, so it's not compatible with older code. All that needs to be updated is the function name itself in previous code (like the demos). (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)

Documentation Changes:

  1. Update README and Docs (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34, Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33)

Co-authored-by: @ashjbarnes, @helenmacdonald

this PR closes #184 , #168 , and also #55 in a roundabout way since user now specifies which boundaries they want, and the MOM inputs are adjusted accordingly.

Before merging, need to resolve the following:

Outstanding Questions:

  1. Logging was implemented in the two new modules of RM6 (regridding.py and rotation.py). Do we want to switch over RM6 to use python logging instead of print statements? Currently, we would prefer that but open it up to opinions.
  2. A while back, we all briefly discussed that this should be version 0.8 instead of 1. Any opinions or insight?
  3. (Solved!) The docker image is currently CROCODILE's image, if we want to update RM6's image, it's a three step process by someone with write permissions & docker downloaded

ashjbarnes and others added 30 commits August 28, 2024 14:00
Changes to make compatible with CESM
having mask table and layout no longer required
Replace minimum layers with min depth
…nt num

After talking with Ashley, the find_MOM6_orientation will likely be removed
The function names for rectangular and sinmple boundaries were changed because the tides are a kind of boundary function, the old names now give a warning and call the correct function. GFDL had rough horizontal subsetting for the tpxo dataset (probably for efficiency?), implemented in setup_tides_rectangular_boundaries.
Collapsed the *_tidal_dims functions into the encode_tides function in segment, and add citing documentation to the docstrings for now.
rotational_method: RotationMethod, hgrid: xr.Dataset, orientation=None
):
"""
Returns the rotation angle - THIS IS ALWAYS BASED ON THE ASSUMPTION OF DEGREES - based on the rotational method and provided hgrid, if orientation & coords are provided, it will assume the boundary is requested.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the capitals here note to self? should we remove it? Or at least make it lowercase?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making it lower case is probably best!

Comment on lines +12 to +14
```bash
docker pull ghcr.io/cosima/regional-test-env:updated
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manishvenu I install docker and run this and got an error!

(base) navid:croco-regional-mom6/ (main) $ docker pull ghcr.io/cosima/regional-test-env:updated                     [17:26:51]

Cannot connect to the Docker daemon at unix:///Users/navid/.docker/run/docker.sock. Is the docker daemon running?

The instructions need to be expanded to be self-sufficient... I'm sorry I don't have any idea how to work with docker images to help here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you don't have docker engine running. You may need to start the service first before running docker commands if you want to test this out.

I think taking your recommendation of a link to the Docker documentation would be best (http://docs.docker.com/get-started/get-docker/). This error is a great example, it's a pretty common error, and any future devs who are trying to learn Docker and add to testing can figure this out with docs & online sources. Docker is a pretty big software with a lot of documentation . It's hard enough learning Docker without providing our own attempt at running Docker. I can add the link to the docs when I get back to this.

Lmk how you feel about that, happy to discuss further.

Copy link
Contributor

@navidcy navidcy Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right! We don't need to include all the instructions. Just point the devs to somewhere they can get info? Perhaps we can say something along the lines of "As soon as you have a docker set up and running and X, Y, Z is OK then do this..." and we continue with how the docs are atm?

What I'm suggesting is 1-2 sentences before of what is currently in the docs atm, not much more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sounds like we're in agreement. I'm not sure what X, Y, & Z would be but I can come up with some checks probably!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our docker setup is fairly standard. There are plenty of resources out there for people to learn docker basics like the one Manish mentioned which is all you need to get going with this so probably no need to go overboard on what we include in our docs!

Copy link
Contributor

@navidcy navidcy Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps just mention the website and tell people have a docker setup or something. Don’t worry for the X, Y, Z. It was just an example. If nothing comes to mind straight away then it’s not the way forward.

Like what would you advise me to do before starting to follow the instructions in the docs already. I still don’t know what to do.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@navidcy I would read the documentation to get docker installed and make sure you can run docker commands on the command line (hah! There's a check!)

I'll add this to the docs!

@navidcy
Copy link
Contributor

navidcy commented Jan 29, 2025

seems like we skip a lot of cells from the tests via NBVAL_SKIP. It's OK to skip the plotting cells, but why are we skipping the cells related to setting up tides? We'd like to test the tide setup, right?

@manishvenu
Copy link
Collaborator Author

seems like we skip a lot of cells from the tests via NBVAL_SKIP. It's OK to skip the plotting cells, but why are we skipping the cells related to setting up tides? We'd like to test the tide setup, right?

Tidal data isn't directly public: https://www.tpxo.net/. Putting it on our docker image is probably not a good idea. (Check the comment in that cell)

We do have test functions for tides in the actual test files. I don't think the point of the demos is to test our functions, and it's probably not worth the overhead to either add fake data to a docker image or make the image not public - we're losing pretty much no coverage.

@navidcy
Copy link
Contributor

navidcy commented Jan 29, 2025

seems like we skip a lot of cells from the tests via NBVAL_SKIP. It's OK to skip the plotting cells, but why are we skipping the cells related to setting up tides? We'd like to test the tide setup, right?

Tidal data isn't directly public: https://www.tpxo.net/. Putting it on our docker image is probably not a good idea. (Check the comment in that cell)

We do have test functions for tides in the actual test files. I don't think the point of the demos is to test our functions, and it's probably not worth the overhead to either add fake data to a docker image or make the image not public - we're losing pretty much no coverage.

Fair! Thanks for elaborating on this!

@ashjbarnes
Copy link
Collaborator

Authors: @ashjbarnes, @manishvenu, @helenmacdonald

Description:

Ashley and I worked together to add a few features to RM6. While we get the changes ready to merge with the main COSIMA branch, please check out this draft with a list of the changes below.

Major Features:

1. Accepting curved boundary supergrids and bathymetry ([Add angled grids to RMOM6 CROCODILE-CESM/regional-mom6#13](https://github.com/CROCODILE-CESM/regional-mom6/pull/13))

2. Implement Boundary Tides from OSU's TPXO using GFDL NWA25 Code ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

3. Import experiment configuration from a light config JSON ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

4. Major bathymetry fixes for land-sea mask ([Various bathymetry improvements / bugfixes CROCODILE-CESM/regional-mom6#17](https://github.com/CROCODILE-CESM/regional-mom6/pull/17))

5. Moving experiment specific changes to MOM_override (All OBC, Tides, and other minor changes) and out of MOM_input

6. New GLORYS downloader (that can account for curved boundaries) ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

7. Tests: Additional tests (tides. rotation, and regridding), Conftest.py, Add to Docker Image ([MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34](https://github.com/CROCODILE-CESM/regional-mom6/pull/34))

8. Changing function names to verb beginnings (initial_condition -> setup_initial_condition) ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

9. Add properties (i.e. more functional object variables) and dataset plotting ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

10. Add Rotation: Add MOM6 angle calculation at t points (MOM6 doesn't use an input hgrid's angle_dx field. It calculate the angle internally at t-points) & two angle approximation methods for the boundary q/u/v points ([MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34](https://github.com/CROCODILE-CESM/regional-mom6/pull/34))

11. Add Land Mask ([Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33](https://github.com/CROCODILE-CESM/regional-mom6/pull/33))

Minor Features:

1. Ability to code without greek letters

2. Removed unintentional machine-specific code

3. Increased print statements

Breaking Changes (for backwards compatibility):

1. Functions changed names, so it's not compatible with older code. All that needs to be updated is the function name itself in previous code (like the demos). ([Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12](https://github.com/CROCODILE-CESM/regional-mom6/pull/12))

Documentation Changes:

1. Update README and Docs ([MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34](https://github.com/CROCODILE-CESM/regional-mom6/pull/34), [Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33](https://github.com/CROCODILE-CESM/regional-mom6/pull/33))

Co-authored-by: @ashjbarnes, @helenmacdonald

this PR closes #184 , #168 , and also #55 in a roundabout way since user now specifies which boundaries they want, and the MOM inputs are adjusted accordingly.

Before merging, need to resolve the following:

* [x]  Include a curved boundary / tide example alongside the existing Tassie example

* [x]  Resolve the tidal regridding issue here ([Tidal regridding doesn't appear to take into account curved lat / lon CROCODILE-CESM/regional-mom6#22](https://github.com/CROCODILE-CESM/regional-mom6/issues/22) and ([Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33](https://github.com/CROCODILE-CESM/regional-mom6/pull/33))

Outstanding Questions:

1. Logging was implemented in the two new modules of RM6 (`regridding.py` and `rotation.py`). Do we want to switch over RM6 to use python logging instead of print statements? Currently, we would prefer that but open it up to opinions.

2. A while back, we all briefly discussed that this should be version 0.8 instead of 1. Any opinions or insight?

3. The docker image is currently CROCODILE's image, if we want to update RM6's image, it's a three step process by someone with write permissions & docker downloaded
  1. If not too much hassle then yes going to proper logging would be good. But I'd suggest this as a separate PR
  2. I think 1.0 makes the most sense because of all the major breaking changes right? Like almost no code written with a previous version will work any more
  3. I tried to give Manish the right permissions for the docker image but maybe it didn't work. I can try do it myself though @manish maybe dm me the steps since you've figured it out? It's been a while!

@manishvenu
Copy link
Collaborator Author

Outstanding Questions:

1. Logging was implemented in the two new modules of RM6 (`regridding.py` and `rotation.py`). Do we want to switch over RM6 to use python logging instead of print statements? Currently, we would prefer that but open it up to opinions.

2. A while back, we all briefly discussed that this should be version 0.8 instead of 1. Any opinions or insight?

3. The docker image is currently CROCODILE's image, if we want to update RM6's image, it's a three step process by someone with write permissions & docker downloaded
  1. If not too much hassle then yes going to proper logging would be good. But I'd suggest this as a separate PR
  2. I think 1.0 makes the most sense because of all the major breaking changes right? Like almost no code written with a previous version will work any more
  3. I tried to give Manish the right permissions for the docker image but maybe it didn't work. I can try do it myself though @manish maybe dm me the steps since you've figured it out? It's been a while!

@ashjbarnes Thanks!

  1. Logging is something I've wanted to do for a while hah, so I might commit if I get the time here (Already added logging to the regridding & rotation modules so it shouldn't be super out of place). But if I don't have time, I'll open a new PR.
  2. Awesome! @navidcy, I remember you had suggested it to be v0.8, any opinions?
  3. Angus figured it out for me! It looks like people with permissions on this repo will also get permissions on the package.

* Change Copernicusmarine version & remove xarray limiter

* Put back Xarray just in case even though it's probably no longer relevant

* Remove Xarray Dependency
@navidcy
Copy link
Contributor

navidcy commented Feb 4, 2025

  1. Awesome! @navidcy, I remember you had suggested it to be v0.8, any opinions?

I think either is fine!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need to override MOM6 min depth based on value set in setup_bathymetry function
8 participants