Add utility for modifying pop model output to be compatible with xgcm #21

andersy005 · 2019-09-25T17:12:13Z

Closes #17, and towards #16

pop_tools/modify_pop_for_xgcm.py

andersy005 · 2019-09-26T22:35:01Z

@matt-long, this is ready for another look. If you deem it complete, we can go ahead and merge it. I will add tests for it in a separate PR when test data is available on the FTP server.

dcherian · 2019-09-26T22:50:43Z

pop_tools/modify_pop_for_xgcm.py

+    y_loc_key = int(grid_loc[2])
+    z_loc_key = int(grid_loc[3])
+
+    x_loc = {1: 'nlon_t', 2: 'nlon_u'}[x_loc_key]


You also need

x_loc = {1: "nlon_t", 2: "nlon_u", 3: "nlon_u"}[x_loc_key] y_loc = {1: "nlat_t", 2: "nlat_u", 3: "nlat_t"}[y_loc_key]

to make DZU and DZT work

Sounds good. Thank you, @dcherian

Can you point me to a small test dataset that I can use for testing purposes?

Ask Anna on that email thread. I just left work

klindsay28 · 2019-09-26T23:21:06Z

When partial_bottom_cells=.true. in POP, the bottom layer doesn't always have the nominal thickness. We probably want this utility to handle that usecase, assuming that xgcm can handle it (which I think is a reasonable assumption). I think this can be detected in model output by comparing HT to something like z_w[KMT] (I might have an off-by-1 error here). If they differ in some places, then partial_bottom_cells=.true..

matt-long

Do we have tests?

What is the primary user-facing API that has been enabled?

pop_tools/modify_pop_for_xgcm.py

andersy005 · 2019-09-30T22:48:41Z

Do we have tests?

Not yet. Can someone point me to some sample data to use?

What is the primary user-facing API that has been enabled?

To use this functionality the user would need to do the following:

import xarray as xr
from pop_tools.xgcm_util import relabel_pop_dims
ds = xr.open_mfdataset('pop_netcdf_files.*.nc')
ds = relabel_pop_dims(ds)

# now you can use xgcm
import xgcm
grid = xgcm.Grid(ds)

What additional user-facing functionality should we support?

matt-long · 2019-09-30T22:54:01Z

Let's wrap that in a method poptools.get_xgcm_grid(ds).

matt-long · 2019-09-30T22:55:57Z

I think tests/data/cesm_pop_monthly.T62_g17.nc
should be appropriate for tests. If it's missing key fields, we could augment.

bradyrx · 2019-10-16T21:35:37Z

Here's some examples of computations to try out in the testing: https://xgcm.readthedocs.io/en/latest/grid_metrics.html

andersy005 · 2019-11-24T05:08:58Z

This seems to be working:

>>> from pop_tools import get_xgcm_grid, DATASETS
>>> import xarray as xr
>>> fname = DATASETS.fetch("iron_tracer.nc")
>>> ds = xr.open_dataset(fname)
>>> ds
<xarray.Dataset>
Dimensions:    (nlat: 384, nlon: 320, time: 24, z_t: 60, z_w_bot: 60, z_w_top: 60)
Coordinates:
    * time       (time) object 0249-02-01 00:00:00 ... 0251-01-01 00:00:00
    * z_t        (z_t) float32 500.0 1500.0 2500.0 ... 487508.34 512502.8 537500.0
    TLAT       (nlat, nlon) float64 ...
    ULONG      (nlat, nlon) float64 ...
    TLONG      (nlat, nlon) float64 ...
    ULAT       (nlat, nlon) float64 ...
    * z_w_top    (z_w_top) float32 0.0 1000.0 2000.0 ... 500004.7 525000.94
    * z_w_bot    (z_w_bot) float32 1000.0 2000.0 3000.0 ... 525000.94 549999.06
Dimensions without coordinates: nlat, nlon
Data variables:
     UE         (time, z_t, nlat, nlon) float32 ...
     VN         (time, z_t, nlat, nlon) float32 ...
     WT         (time, z_w_top, nlat, nlon) float32 ...
     HDIFE      (time, z_t, nlat, nlon) float32 ...
     HDIFN      (time, z_t, nlat, nlon) float32 ...
     HDIFB      (time, z_w_bot, nlat, nlon) float32 ...
     DIA_IMPVF  (time, z_w_bot, nlat, nlon) float32 ...
     KPP_SRC    (time, z_t, nlat, nlon) float32 ...
     STF        (time, nlat, nlon) float32 ...
     SMS        (time, nlat, nlon) float32 ...
>>> grid = get_xgcm_grid(ds)
>>> grid
<xgcm.Grid>
Z Axis (periodic):
        * center   z_t --> left
        * right    z_w_bot --> center
        * left     z_w_top --> center
Y Axis (periodic):
        * center   nlat_t --> right
        * right    nlat_u --> center
X Axis (periodic):
        * center   nlon_t --> right
        * right    nlon_u --> center

I still haven't addressed @klindsay28's comment though.

When partial_bottom_cells=.true. in POP, the bottom layer doesn't always have the nominal thickness. We probably want this utility to handle that usecase, assuming that xgcm can handle it (which I think is a reasonable assumption). I think this can be detected in model output by comparing HT to something like z_w[KMT] (I might have an off-by-1 error here). If they differ in some places, then partial_bottom_cells=.true..

Since I am not that familiar with POP model output, it would be nice for someone to take a look at the current code in this PR, and let me know what changes need to be made to address this.

andersy005 · 2019-12-05T17:24:36Z

@dcherian, @matt-long, the gentlest of bumps on this. Have two minute to take a look at this? 😄

dcherian

Don't know anything about bottom cells.

dcherian · 2019-12-05T17:47:33Z

pop_tools/xgcm_util.py

+    """
+    import xgcm
+
+    ds = relabel_pop_dims(ds)


Fairly certain this doesn't work. ds isn't returned but a copy is made deep down in the call stack and all the xgcm stuff nrequires the renamed dimensions.

How about

grid, new_ds = pop_tools.to_xgcm_grid_dataset(ds)

How about

grid, new_ds = pop_tools.to_xgcm_grid_dataset(ds)

👍 Good catch. I've addressed this and renamed the utility function

tests/test_xgcm_util.py

dcherian · 2019-12-05T17:53:47Z

pop_tools/xgcm_util.py

+
+
+def _label_coord_grid_locs(ds):
+    grid_locs = {


we need a dataset that tests all of these. I've emailed Anna on our previous email thread. She was looking at one that had DZU & DZT

dcherian

Nice! I left one minor comment but LGTM otherwise.

tests/test_xgcm_util.py

andersy005 · 2019-12-09T19:59:31Z

@dcherian, Thank you for reviewing this! I think this is ready to go.

bradyrx · 2019-12-09T21:16:59Z

This looks great @andersy005!

andersy005 added 3 commits September 25, 2019 11:10

Add xgcm as a dependency

27ca63b

Add module for modifying pop for xgcm

1e42aaf

Add notebook to depencies

5ffa051

andersy005 commented Sep 25, 2019

View reviewed changes

pop_tools/modify_pop_for_xgcm.py Outdated Show resolved Hide resolved

andersy005 marked this pull request as ready for review September 25, 2019 22:52

andersy005 added 2 commits September 25, 2019 21:21

Replace try block with if statement

d66987a

Add DZU, DZT to grid_locs

b1391bf

dcherian reviewed Sep 26, 2019

View reviewed changes

Update x_loc and y_loc

d5a5bdd

matt-long reviewed Sep 30, 2019

View reviewed changes

pop_tools/modify_pop_for_xgcm.py Outdated Show resolved Hide resolved

andersy005 added 2 commits September 30, 2019 16:56

Use a more appropriate module name

edb64d4

Add top-level get_xgcm_grid function

52562cd

This comment has been minimized.

Sign in to view

bradyrx mentioned this pull request Oct 16, 2019

add budget module and regional tracer budget computation #12

Closed

andersy005 added 5 commits November 23, 2019 15:28

Merge branch 'master' of github.com:NCAR/pop-tools into pop2xgcm_grid

3f3f8ea

Add test

6be0b8c

Use xgcm master

58cce46

Update environment

ac04527

Add dask and xarray

5279f34

andersy005 requested review from dcherian and matt-long November 24, 2019 05:09

dcherian suggested changes Dec 5, 2019

View reviewed changes

andersy005 added 4 commits December 9, 2019 11:14

Fix bug

356ac54

Add test for UEU, VNU variables

21cca80

Return dataset with renamed dimensions

f5624a1

Use coarsen-ed dataset

ea3e344

dcherian approved these changes Dec 9, 2019

View reviewed changes

tests/test_xgcm_util.py Show resolved Hide resolved

Check for non-existance of new coords in the original ds

53c396c

Fix indentation

6bc5445

andersy005 mentioned this pull request Dec 10, 2019

Handle partial_bottom_cells=.true. in xgcm_util.py module #27

Open

andersy005 merged commit 2cc1b90 into NCAR:master Dec 10, 2019

andersy005 deleted the pop2xgcm_grid branch December 10, 2019 17:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add utility for modifying pop model output to be compatible with xgcm #21

Add utility for modifying pop model output to be compatible with xgcm #21

andersy005 commented Sep 25, 2019 •

edited

Loading

andersy005 commented Sep 26, 2019

dcherian Sep 26, 2019

andersy005 Sep 26, 2019

dcherian Sep 26, 2019

klindsay28 commented Sep 26, 2019

matt-long left a comment

andersy005 commented Sep 30, 2019

matt-long commented Sep 30, 2019

matt-long commented Sep 30, 2019

This comment has been minimized.

bradyrx commented Oct 16, 2019

andersy005 commented Nov 24, 2019

andersy005 commented Dec 5, 2019

dcherian left a comment

dcherian Dec 5, 2019

andersy005 Dec 9, 2019

dcherian Dec 5, 2019

dcherian left a comment

andersy005 commented Dec 9, 2019

bradyrx commented Dec 9, 2019

Add utility for modifying pop model output to be compatible with xgcm #21

Add utility for modifying pop model output to be compatible with xgcm #21

Conversation

andersy005 commented Sep 25, 2019 • edited Loading

andersy005 commented Sep 26, 2019

dcherian Sep 26, 2019

Choose a reason for hiding this comment

andersy005 Sep 26, 2019

Choose a reason for hiding this comment

dcherian Sep 26, 2019

Choose a reason for hiding this comment

klindsay28 commented Sep 26, 2019

matt-long left a comment

Choose a reason for hiding this comment

andersy005 commented Sep 30, 2019

matt-long commented Sep 30, 2019

matt-long commented Sep 30, 2019

This comment has been minimized.

bradyrx commented Oct 16, 2019

andersy005 commented Nov 24, 2019

andersy005 commented Dec 5, 2019

dcherian left a comment

Choose a reason for hiding this comment

dcherian Dec 5, 2019

Choose a reason for hiding this comment

andersy005 Dec 9, 2019

Choose a reason for hiding this comment

dcherian Dec 5, 2019

Choose a reason for hiding this comment

dcherian left a comment

Choose a reason for hiding this comment

andersy005 commented Dec 9, 2019

bradyrx commented Dec 9, 2019

andersy005 commented Sep 25, 2019 •

edited

Loading