Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode times adds micro-second noise to standard calendar #1917

Closed
mankoff opened this issue Feb 16, 2018 · 5 comments
Closed

Decode times adds micro-second noise to standard calendar #1917

mankoff opened this issue Feb 16, 2018 · 5 comments

Comments

@mankoff
Copy link
Contributor

mankoff commented Feb 16, 2018

Code Sample, a copy-pastable example if possible

I have a simplified NetCDF file with the following header:

netcdf foo {
dimensions:
	time = UNLIMITED ; // (366 currently)
	x = 2 ;
	y = 2 ;
variables:
	float time(time) ;
		time:standard_name = "time" ;
		time:long_name = "time" ;
		time:units = "DAYS since 2000-01-01 00:00:00" ;
		time:calendar = "standard" ;
		time:axis = "T" ;
...
}

I would expect xarray to be able to decode these times. It does, but appears to do so incorrectly and without reporting any issues. Note the fractional time added to each date.

In [4]: xr.open_dataset('foo.nc').time
Out[4]:
<xarray.DataArray 'time' (time: 366)>
array(['2000-01-01T00:00:00.000000000', '2000-01-02T00:00:00.003211264',
       '2000-01-03T00:00:00.006422528', ..., '2000-12-29T00:00:01.962606592',
       '2000-12-30T00:00:01.672216576', '2000-12-31T00:00:01.381826560'], dtype='datetime64[ns]')
Coordinates:
  * time     (time) datetime64[ns] 2000-01-01 2000-01-02T00:00:00.003211264 ...
Attributes:
    standard_name:  time
    long_name:      time
    axis:           T

Problem description

Days since a valid date on a standard calendar should not add microseconds.

I know that xarray has time issues, for example #118 #521 numpy:#6207 #531 #789 and #848. But all of those appear to address non-standard times. This bug (if it is a bug) seems to occur with a very simple and straight forward calendar, and is silent, so it took me 2 days to figure out what was going on.

Output of xr.show_versions()

In [5]: xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

xarray: 0.10.0
pandas: 0.22.0
numpy: 1.12.1
scipy: 0.19.1
netCDF4: 1.3.1
h5netcdf: None
Nio: None
bottleneck: 1.2.1
cyordereddict: 1.0.0
dask: 0.16.0
matplotlib: 2.1.1
cartopy: 0.15.1
seaborn: 0.8.1
setuptools: 38.4.0
pip: 9.0.1
conda: None
pytest: None
IPython: 6.2.1
sphinx: None

@shoyer
Copy link
Member

shoyer commented Feb 16, 2018

Can you share a file that reproduces the issue?

@mankoff
Copy link
Contributor Author

mankoff commented Feb 16, 2018

foo.nc.zip

@shoyer
Copy link
Member

shoyer commented Feb 17, 2018

I just tested this out, and it appears that the issue is fixed on master -- see #1859.

@jhamman
Copy link
Member

jhamman commented Feb 26, 2018

@mankoff - can you try with xarray v0.10.1 (released today)?

@mankoff
Copy link
Contributor Author

mankoff commented Feb 26, 2018

Appears fixed. Thank you!

@mankoff mankoff closed this as completed Feb 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants