Error processing CMIP6 data and writing to netcdf file.

[richardarsenault]

Description

Trying to extract CMIP6 data from a notebook on the pavics.ouranos.ca server.

A first error stops me from being able to subset the climate data by time. So I download the entire timeseries to get past this one.

A second error appears for some climate models, wherein I can extract the data from CMIP6 sources, subset lat/lon no problem, and merge multiple variables into a dataset. This works for all climate models. Then, when trying to write to netcdf: For some climate models it works, for others, it fails.

Attributes look identical, except for the calendar, so it's probably related. Looking for advice or guidance on how to approach this. Maybe XClim has some tools to address this?

Environment

The current server instance on pavics.ouranos.ca

Steps to Reproduce

Unzip and upload the attached .ipynb file to the pavics.ouranos.ca server, in a writable-workspace;
Run the cells. Markdown cells indicate where the errors occur.

Additional Information

CMIP6_to_netcdf_problem.zip

[tlvu]

@richardarsenault Not sure if it would help but I am currently deploying a newer version of the Jupyter env as "beta" image to pavics.ouranos.ca. It has a newer Xclim and a bunch of newer updates. See PR Ouranosinc/PAVICS-e2e-workflow-tests#101 for all details.

If you feel like it, you could retry in the "beta" image see whether there is a difference. This beta image will be in production probably by tomorrow.

[richardarsenault]

@tlvu Thanks for the tip! I just tried within the beta server image, with the same results unfortunately.

[huard]

The first problem in the notebook can be resolved using clisops subsetting functions.

future_start = "2099"
future_stop = "2101-01-01"
...
future_tasmin=subset.subset_bbox(xr.open_zarr(mapper, consolidated=True), 
                                 lon_bnds=[-75, -70], lat_bnds=[47, 53], 
                                 start_date=future_start, end_date=future_stop)

I can't reproduce the second problem on the current Jupyter instance.

[richardarsenault]

Thanks, the subsetting using clisops still fails for me when using, for example:

future_start = "2015"
future_stop = "2101-01-01"
future_tasmin=subset.subset_bbox(xr.open_zarr(mapper, consolidated=True), 
                                 lon_bnds=[-75, -70], lat_bnds=[47, 53], 
                                 start_date=future_start, end_date=future_stop)

But it works with xarray subsetting in combination with the string format of dates:

future_start = "2015"
future_stop = "2101-01-01"

future_pr=subset.subset_bbox(xr.open_zarr(mapper, consolidated=True).sel(time=slice(future_start ,future_stop )),
lon_bnds=[-75, -70], lat_bnds=[47, 53])

The second error I can also no longer reproduce, so probably something has been updated.

I will be able to integrate this fix into the notebooks. Thanks!