Datasets with multidimensional coordinates

[forman]

Coordinates for latitudes and longitudes (e.g. latand lon coordinate variables) in climate and meteorological datasets are often provided as 2D arrays with dimensions y and x. Here x and y are the horizontal and vertical resolutions of the spatial grid in use. This is often done although the grid uses a equi-rectangular "projection" and therefore all lat columns and all lon rows are the same.

Here is an example 7-day subset from the QA4ECV project:

<xarray.Dataset>
Dimensions:                     (time: 7, x: 720, y: 360)
Coordinates:
    lat                         (y, x) float32 89.75 89.75 89.75 89.75 89.75 ...
    lon                         (y, x) float32 -179.75 -179.25 -178.75 ...
  * time                        (time) int64 2001001 2001002 2001003 2001004 ...
Dimensions without coordinates: x, y
Data variables:
    BHR_VIS                     (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_NIR                     (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_SW                      (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_VIS                     (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_NIR                     (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_SW                      (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_alpha_VIS_NIR           (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_alpha_VIS_SW            (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_alpha_NIR_SW            (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_alpha_VIS_NIR           (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_alpha_VIS_SW            (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_alpha_NIR_SW            (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_sigma_VIS               (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_sigma_NIR               (time, y, x) float64 nan nan nan nan nan nan ...
    BHR_sigma_SW                (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_sigma_VIS               (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_sigma_NIR               (time, y, x) float64 nan nan nan nan nan nan ...
    DHR_sigma_SW                (time, y, x) float64 nan nan nan nan nan nan ...
    Relative_Entropy            (time, y, x) float64 nan nan nan nan nan nan ...
    Weighted_Number_of_Samples  (time, y, x) float64 nan nan nan nan nan nan ...
    Goodness_of_Fit             (time, y, x) float64 nan nan nan nan nan nan ...
    Data_Mask                   (time, y, x) float64 nan nan nan nan nan nan ...
    Solar_Zenith_Angle          (time, y, x) float64 nan nan nan nan nan nan ...
Attributes:
    title:                      QA4ECV Albedo Product
    institution:                Mullard Space Science Laboratory, Department ...
    source:                     Satellite observations, BRDF/Albedo Inversion...
    history:                    QA4ECV Processing, 2014-2017
    references:                 www.qa4ecv.eu
    tracking_id:                49b8d6eb-1503-4400-91a4-1f1a8f87f12b
    Conventions:                CF-1.6
    product_version:            1.0
    summary:                    This dataset contains Level-3 daily surface b...
    keywords:                   Albedo, TIP, FAPAR
    id:                         Qa4ecv.albedo.avh_geo.2001001_upscaled.nc
    naming_authority:           ucl.ac.uk
    keywords_vocabulary:        
    cdm_data_type:              Alb
    comment:                    These data were produced in the frame of the ...
    date_created:               20170613T081550Z
    creator_name:               Mullard Space Science Laboratory, Department ...
    creator_url:                www.ucl.ac.uk/mssl
    creator_email:              [email protected]
    project:                    QA4ECV - European Union Framework Program 7
    geospatial_lat_min:         -90.0
    geospatial_lat_max:         90.0
    geospatial_lon_min:         -180.0
    geospatial_lon_max:         180.0
    geospatial_vertical_min:    0.0
    geospatial_vertical_max:    0.0
    time_coverage_start:        20010101T000000Z
    time_coverage_end:          20010101T235959Z
    time_coverage_duration:     P1D
    time_coverage_resolution:   P1D
    standard_name_vocabulary:   NetCDF Climate and Forecast (CF) Metadata Con...
    license:                    QA4ECV Data Policy: free and open access
    platform:                   NOAA-xx, Meteosat MFG, Meteosat MSG, GOES, GMS
    sensor:                     AVHRR, MVIRI, SEVIRI, Imager, VISSR
    spatial_resolution:         1km at Equator
    geospatial_lat_units:       degrees_north
    geospatial_lon_units:       degrees_east
    geospatial_lat_resolution:  0.5
    geospatial_lon_resolution:  0.5

Expected behavior

Cate should be able to open such datasets as they are CF-conform, i.e. such datasets pass the CF check at http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl

Also xarray has no problem such data in general.

Actual behavior

Currently, Cate cannot deal with such datasets at all. The code raises exceptions at various locations as it is presumed that lonand lat are 1D coordinate variables in numerous operations and data access functions:

• open_dataset
• harmonize
• spatial_subset
• coregister
• ...

Steps to reproduce the problem

1. Download some data from http://gws-access.cems.rl.ac.uk/public/qa4ecv/olafd/albedo/L3_Mosaic_NoSnow/v0.9/2001/
2. Create a local data source
3. Open a dataset

Specifications

Cate 0.9.0dev3

Workaround

Let harmonize reformat such datasets, so we can deal with it.