Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't set encoding attributes on bounds variables. #2965

Merged
merged 18 commits into from
Jun 25, 2019
Merged

Don't set encoding attributes on bounds variables. #2965

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ff2fd49
Don't set attributes on bounds variables.
dcherian May 15, 2019
302ab63
remove whitespace stuff.
dcherian May 15, 2019
931f973
Make sure variable exists in first file before assigning encoding
dcherian May 15, 2019
5526fe4
Make sure we iterate over coords too.
dcherian May 15, 2019
3889ba6
lint fix.
dcherian May 15, 2019
6f2bc05
docs/comment fixes.
dcherian May 15, 2019
b903e89
mfdataset encoding test.
dcherian May 17, 2019
70c8c5c
time_bounds attrs test + allow for slight CF non-compliance.
dcherian May 17, 2019
d637e9e
I need to deal with encoding!
dcherian May 19, 2019
2f1dd25
minor fixes.
dcherian May 19, 2019
12f3e55
another minor fix.
dcherian May 19, 2019
f8789e7
review fixes.
dcherian May 19, 2019
e0c49a4
lint fixes.
dcherian May 19, 2019
674e5a5
Merge branch 'master' into fix/bounds_encode_2
dcherian Jun 12, 2019
34d0e60
Remove encoding changes and xfail test.
dcherian Jun 24, 2019
f187ca1
Merge branch 'master' into fix/bounds_encode_2
dcherian Jun 24, 2019
b1dcf1d
Merge branch 'fix/bounds_encode_2' of github.com:dcherian/xarray into…
dcherian Jun 24, 2019
c63cf33
Update whats-new.rst
dcherian Jun 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions xarray/backends/api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -724,6 +724,9 @@ def open_mfdataset(paths, chunks=None, concat_dim=_CONCAT_DIM_DEFAULT,

combined._file_obj = _MultiFileCloser(file_objs)
combined.attrs = datasets[0].attrs
for v in combined.variables:
if v in datasets[0]:
combined[v].encoding = datasets[0][v].encoding
dcherian marked this conversation as resolved.
Show resolved Hide resolved
return combined


Expand Down
15 changes: 14 additions & 1 deletion xarray/conventions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -599,7 +599,7 @@ def cf_encoder(variables, attributes):
as possible. This includes masking, scaling, character
array handling, and CF-time encoding.

Decode a set of CF encoded variables and attributes.
Encode a set of CF encoded variables and attributes.

See Also, decode_cf_variable

Expand All @@ -621,4 +621,17 @@ def cf_encoder(variables, attributes):
"""
new_vars = OrderedDict((k, encode_cf_variable(v, name=k))
for k, v in variables.items())

# Remove attrs from bounds variables before encoding
# See issue #2921
for var in new_vars.values():
bounds = var.attrs['bounds'] if 'bounds' in var.attrs else None
if bounds and bounds in new_vars:
# see http://cfconventions.org/cf-conventions/cf-conventions.html#cell-boundaries # noqa
for attr in ['units', 'standard_name', 'axis', 'positive',
'calendar', 'long_name', 'leap_month', 'leap_year',
'month_lengths']:
if attr in new_vars[bounds].attrs:
dcherian marked this conversation as resolved.
Show resolved Hide resolved
new_vars[bounds].attrs.pop(attr)

Copy link
Collaborator

@mathause mathause May 17, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we issue a warning here?

else:
    warning.warn("The attribute 'units' is not the same in the variable "
"'time' and it's associated bounds 'time_bnds',"
" which is not cf-compliant.")

or some such

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to? xarray allows writing CF-non-compliant files anyway...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the warning you added is the perfect approach. It will still be issued in the case of @mathause's example, but will still allow a user to write a non-CF-compliant file without a warning if the encoding attributes do not need to be computed on the fly.

return new_vars, attributes