encode_cf_datetime: reference_date can not be "2000-02-30"

[mathause]

What happened:

I try to save the result of ds.resample("time": "Q-FEB").mean() with a 360_day calendar as netcdf. Thus, my first date is cftime.Datetime360Day(2000-02-30) (note the day). xarray then tries to use units='days since 2000-02-30' which fails with ValueError: day is out of range for month.

What you expected to happen:

The dataset can be saved.

Minimal Complete Verifiable Example:

import cftime
import xarray as xr

time = xr.cftime_range("2000-02-30", "2001-01-01", freq="3M", calendar="360_day")

dates = np.asarray(time)
reference_date = xr.coding.times.infer_datetime_units(dates)
# 'days since 2000-02-30 00:00:00.000000'

xr.coding.times.encode_cf_datetime(time)
# ValueError

Traceback:

---------------------------------------------------------------------------
OutOfBoundsDatetime                       Traceback (most recent call last)
~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/coding/times.py in encode_cf_datetime(dates, units, calendar)
    367             # parse with cftime instead
--> 368             raise OutOfBoundsDatetime
    369         assert dates.dtype == "datetime64[ns]"

OutOfBoundsDatetime: 

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-157-727cb22f83b1> in <module>
      4 time = xr.cftime_range("2000-02-30", "2001-01-01", freq="3M", calendar="360_day")
      5 
----> 6 xr.coding.times.encode_cf_datetime(time)

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/coding/times.py in encode_cf_datetime(dates, units, calendar)
    385 
    386     except (OutOfBoundsDatetime, OverflowError):
--> 387         num = _encode_datetime_with_cftime(dates, units, calendar)
    388 
    389     num = cast_to_int_if_safe(num)

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/coding/times.py in _encode_datetime_with_cftime(dates, units, calendar)
    332         return np.nan if d is None else cftime.date2num(d, units, calendar)
    333 
--> 334     return np.vectorize(encode_datetime)(dates)
    335 
    336 

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/numpy/lib/function_base.py in __call__(self, *args, **kwargs)
   2089             vargs.extend([kwargs[_n] for _n in names])
   2090 
-> 2091         return self._vectorize_call(func=func, args=vargs)
   2092 
   2093     def _get_ufunc_and_otypes(self, func, args):

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/numpy/lib/function_base.py in _vectorize_call(self, func, args)
   2159             res = func()
   2160         else:
-> 2161             ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)
   2162 
   2163             # Convert args to object arrays first

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/numpy/lib/function_base.py in _get_ufunc_and_otypes(self, func, args)
   2119 
   2120             inputs = [arg.flat[0] for arg in args]
-> 2121             outputs = func(*inputs)
   2122 
   2123             # Performance note: profiling indicates that -- for simple

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/coding/times.py in encode_datetime(d)
    330 
    331     def encode_datetime(d):
--> 332         return np.nan if d is None else cftime.date2num(d, units, calendar)
    333 
    334     return np.vectorize(encode_datetime)(dates)

cftime/_cftime.pyx in cftime._cftime.date2num()

cftime/_cftime.pyx in cftime._cftime._dateparse()

ValueError: day is out of range for month

Anything else we need to know?:

This goes wrong here:

xarray/xarray/coding/times.py

Lines 249 to 250 in 1553749

	reference_date = dates[0] if len(dates) > 0 else "1970-01-01"
	reference_date = format_cftime_datetime(reference_date)

A possible fix is to add the following lines:

try:
    cftime._cftime._dateparse(reference_date)
except ValueError:
    reference_date = type(dates[0])(dates[0].year, dates[0].month, 1)
    reference_date = xr.coding.times.format_cftime_datetime(reference_date)

To workaround set the encoding manually:

encoding = {}
encoding['time'] = {'units': 'days since 1850-01-01'}
ds.to_netcdf(filename, encoding=encoding)

[dcherian]

The fix looks nice, except we should not rely on ._cftime._dateparse...

[spencerkclark]

Thanks @mathause -- interestingly I happened to try this example out with a branch that's soon to be merged in cftime and it appears to work:

In [1]: import xarray as xr

In [2]: import cftime

In [3]: import numpy as np

In [4]: time = xr.cftime_range("2000-02-30", "2001-01-01", freq="3M", calendar="360_day")

In [5]: dates = np.asarray(time)

In [6]: reference_date = xr.coding.times.infer_datetime_units(dates)

In [7]: xr.coding.times.encode_cf_datetime(time)
Out[7]:
(array([  0,  90, 180, 270]),
 'days since 2000-02-30 00:00:00.000000',
 '360_day')

As a general point I think we should strive to be able to handle reference dates that might not be valid in all calendars (they might occur in the wild). Would we be ok labeling this as an upstream issue?

[mathause]

I can confirm that this works now. So I close this.