CRS: CF grid_mapping attribute to encoding

[snowman2]

Related to:

• Understand NetCDF CF convention for projection info in .geobox property #837
• Read grid mapping and bounds as coords pydata/xarray#2844
• REF: Store grid_mapping in encoding instead of attrs corteva/rioxarray#284
• Add full support for xarray decode_coords corteva/rioxarray#282

Apparently it is not CF-compliant to store the CRS grid_mapping as a coordinate. However, it is desirable to keep it in the xarray coords as it persists better. The workaround is to store the grid_mapping in the encoding instead of attrs and to_netcdf will take care of the rest when writing to a netCDF file.

[snowman2]

Note: This requires Python 3.7+ and xarray 0.17+

[Kirill888]

@snowman2 these are all pretty long threads. Can I ask you to summarize what would you like to propose?

I'm open to a suggestion to

• supporting xarray style of geospatial info in .geobox code
• providing method for converting datacube xarrays to a form compatible with .to_netcdf

I'm not too keen to drop coords-based .geobox though, as this is the most robust way we found so far for preserving geo-registration information through data transformations.

[snowman2]

The only change is to store grid_mapping in encoding instead of attrs.

rds.encoding["grid_mapping"] = "spatial_ref"

[snowman2]

This PR is how rioxarray handled it: corteva/rioxarray#284

[snowman2]

I'm not too keen to drop coords-based .geobox though, as this is the most robust way we found so far for preserving geo-registration information through data transformations.

I agree with you 💯

[Kirill888]

Is this about data var attributes only or x,y coords too?

[Kirill888]

datacube-core/datacube/api/core.py

Lines 465 to 473 in 80d466a

	def mk_data_var(m, data_func):
	data = data_func(m)
	attrs = dict(**m.dataarray_attrs(),
	**crs_attrs)
	return xarray.DataArray(data,
	name=m.name,
	coords=coords,
	dims=dims,
	attrs=attrs)

datacube-core/datacube/utils/geometry/_base.py

Lines 1458 to 1462 in 80d466a

	xx.attrs.update(grid_mapping=crs_coord_name)
	if isinstance(xx, xr.Dataset):
	for band in xx.data_vars.values():
	band.attrs.update(grid_mapping=crs_coord_name)

[snowman2]

Should just be for the data vars.

[snowman2]

Remembered this today with something else I was working on. Is this a change you would like to see?

[Kirill888]

@snowman2 I forgot about this one.

Ideally I would like to see this code taken out into a separate library under opendatacube org, as you were suggesting previously, it would make it easier to iterate on.

Looks like this work might be done under https://github.com/opendatacube/odc-tools/projects/2, as there is interest in decoupling data loading code from collection management. Moving geometry handling code out into a standalone library would be a first step toward that goal.

I'm yet to write an "Enhancement Proposal" for that task though. I'm thinking of doing some serious git history surgery on datacube repo to get geometry code and tests into a separate lib while keeping original authorship and history. Followed up by changing datacube to depend on that lib instead of internal implementation.

I believe this a more promising approach, that still delivers tangible benefits, than currently stalled datacube 2.0 effort.

cc: @gadomski

[snowman2]

Sounds great 👍.

[gadomski]

I'm thinking of doing some serious git history surgery on datacube repo to get geometry code and tests into a separate lib while keeping original authorship and history.

For background @Kirill888 we used git-filter-repo when we broke out the stactools subpackages (stac-utils/stactools#111), and it worked well for us. Relevant line here: https://gist.github.com/gadomski/a0ab8d18f56ff34454d2ed890d234534#file-extract_stactools_subpackage-sh-L21.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[snowman2]

Ping to leave open.

[Kirill888]

@snowman2 I pinned this so it won't expire. I'm starting the work of extracting odc.geo library from datacube-core. The plan is to

1. Move geometry and related tests to a separate package and repository under opendatacube org (with history)
2. Setup all the necessary infra: testing, packaging, docs in the new repository
3. Do cleanup: formatting, docs, types, test coverage
4. Replace datacube dependency in odc-stac with the odc-geo, this is likely to highlight various missing bits, so will require some iteration. It's likely that some parts from odc.algo will migrate to odc.geo as part of this work too (reprojection logic in particular).
5. Finally refactor datacube to depend on odc.geo rather than having duplicate code

So it will take a while before datacube itself is updated, but work is starting now.

[snowman2]

I believe that this is addressed by #1424