-
Notifications
You must be signed in to change notification settings - Fork 3
Deployment Configuration
Wiki ▸ Deployment Configuration
Aside from the source glider data files, gncutils needs to know some information about the following:
- Deployment metadata
- Glider payload
- NetCDF global attributes
- Sensor to NetCDF variable name mappings
This information is contained in 4 json configuration files:
Here's what you need to know about each one and how to configure them. Templates for each one of these file can be found in /gncutils/resources/config-templates.
The structure of these files should not be changed however, the contents should be modified to reflect the glider and deployment specific characteristics.
The deployment.json file stores information specific to the deployment that is being processed. Here's an example of a deployment.json file:
{
"glider": "ru26d",
"trajectory_datetime": " ",
"trajectory_name": "2017-12-11 09:20Z",
"global_attributes": {
"wmo_id": "7801502",
"deployment_number" : 528
},
"platform": {
"type": "platform",
"id": "ru26d",
"wmo_id": "7801502",
"long_name": "Slocum G2 Glider ru26d",
"comment": "Slocum Glider ru26d",
"type": "platform"
}
}
The instruments.json file contains an array of objects in which each object contains information about a specific payload sensor on the glider.
Here's an example instrument.json file:
[
{
"nc_var_name": "instrument_ctd",
"type": "i4",
"attrs": {
"serial_number": "156",
"make_model": "Sea-Bird GPCTD",
"comment": "Slocum Glider ru26d CTD",
"long_name": "Pumped Conductivity, Temperature, Depth (CTD) Sensor",
"calibration_date": " ",
"user_calibrated": "",
"factory_calibrated": " ",
"calibration_report": "",
"type": "instrument",
"platform": "platform",
"deployment" : 528
}
},
{
"nc_var_name": "instrument_flbbcd",
"type": "i4",
"attrs": {
"serial_number": "3326",
"make_model": "Sea-Bird Scientific ECO puck",
"comment": "The ECO FLbb–FD/bb-2K fluorometer",
"long_name": "ECO FLbb-FD/bb-2K",
"calibration_date": " ",
"user_calibrated": "",
"factory_calibrated": " ",
"calibration_report": "",
"type": "instrument",
"platform": "platform",
"deployment" : 528
}
}
]
The nc_var_name field contains the name of the variable to be written to the NetCDF files. All instrument variables are scalar container variables that store metadata about the instrument. The metadata is stored in the attrs dictionary that maps variable attribute names to their values. Each instrument must have a separate entry in the array.
This file contains a dictionary of global attribute keys mapped to their values. The attributes listed below are from both the CF specification and Unidata's ACDD specfication:
{
"Conventions": "CF-1.6, COARDS, ACDD-1.3",
"acknowledgment": "Funding provided by the National Science Foundation",
"cdm_data_type": "trajectory",
"comment": "Glider deployed and piloted by Rutgers University Coastal Ocean Observation Lab",
"contributor_name": "Oscar Schofield, David Aragon, Chip Haldeman, Laura Palamara, John Kerfoot",
"contributor_role": "Principal Investigator, Glider Pilot, Glider Pilot, Data Manager, Data Manager",
"creator_email": "[email protected]",
"creator_institution": "Rutgers University Coastal Ocean Observing Laboratory",
"creator_name": "John Kerfoot",
"creator_type": "person",
"creator_url": "https://www.whoi.edu/ooi_cgsn/",
"date_created": " ",
"date_issued": " ",
"date_metadata_modified": " ",
"date_modified": " ",
"featureType": "Trajectory",
"geospatial_bounds": " ",
"geospatial_bounds_crs": "EPSG:4326",
"geospatial_bounds_vertical_crs": "EPSG:5831",
"geospatial_lat_max": " ",
"geospatial_lat_min": " ",
"geospatial_lat_resolution": "0.00001 degree",
"geospatial_lat_units": "degree_north",
"geospatial_lon_max": " ",
"geospatial_lon_min": " ",
"geospatial_lon_resolution": "0.00001 degree",
"geospatial_lon_units": "degree_east",
"geospatial_vertical_max": " ",
"geospatial_vertical_min": " ",
"geospatial_vertical_positive": "down",
"geospatial_vertical_resolution": " ",
"geospatial_vertical_units": "EPSG:5831",
"history": " ",
"id": " ",
"institution": "Rutgers University Center for Ocean Observing Leadership",
"instrument": "In Situ/Laboratory Instruments > Profilers/Sounders > CTD",
"instrument_vocabulary": "NASA/GCMD Instrument Keywords Version 8.5",
"keywords": "Oceans > Ocean Pressure > Water Pressure, Oceans > Ocean Temperature > Water Temperature, Oceans > Salinity/Density > Conductivity, Oceans > Salinity/Density > Density, Oceans > Salinity/Density > Salinity",
"keywords_vocabulary": "NASA/GCMD Earth Sciences Keywords Version 8.5",
"license": "This data may be redistributed and used without restriction. Data provided as is with no expressed or implied assurance of quality assurance or quality control",
"metadata_link": " ",
"naming_authority": "edu.rutgers.marine.rucool",
"ncei_template_version": "NCEI_NetCDF_Trajectory_Template_v2.0",
"platform": "In Situ Ocean-based Platforms > AUVS > Autonomous Underwater Vehicles",
"platform_type": "Slocum Glider",
"platform_vocabulary": "NASA/GCMD Platforms Keywords Version 8.5",
"processing_level": "Slocum glider data from the native data file format. No quality control provided",
"product_version": " ",
"program": "LTER",
"project": "Long Term Ecological Research Program",
"publisher_email": "[email protected]",
"publisher_institution": "Rutgers University",
"publisher_name": "John Kerfoot",
"publisher_type": "person",
"publisher_url": "https://rucool.marine.rutgers.edu",
"references": " ",
"sea_name": "Southern Ocean",
"source": "Observational data from a profiling glider",
"standard_name_vocabulary": "CF Standard Name Table v27",
"summary": "The purpose of this glider deployment is to survey the waters off the Palmer Peninsula.",
"time_coverage_duration": " ",
"time_coverage_end": " ",
"time_coverage_resolution": " ",
"time_coverage_start": " ",
"title": " "
}
All of the entries in this dictionary are added to each NetCDF file as a global attribute. If you would like to add an additional global attribute all you need to do is add it to the dictionary. The following global attributes (contained in the above example file) are automatically created/populated during the NetCDF writing process:
- cdm_data_type
- history
- featureType
- geospatial_bounds
- geospatial_lat_max
- geospatial_lat_min
- geospatial_lon_max
- geospatial_lon_min
- geospatial_vertical_max
- geospatial_vertical_min
- id
- time_coverage_duration
- time_coverage_end
- time_coverage_start
- title
The sensor definitions file defines NetCDF variable definitions and the mapping of the native glider data file sensor/variable names to the corresponding NetCDF variable name. The JSON file contains a dictionary of dictionaries in which each key of the dictionary maps a native glider data file sensor name to it's NetCDF equivalent (nc_var_name). The general format is as follows:
{
"llat_time": {
"attrs": {
"axis": "T",
"calendar": "gregorian",
"standard_name": "time",
"units": "seconds since 1970-01-01T00:00:00Z",
"valid_max": 2147483647,
"valid_min": 0,
"_FillValue": NaN
},
"dimension": "time",
"is_dimension": true,
"dimension_length": null,
"nc_var_name": "time",
"type": "f8"
},
...
}
Where the key name (llat_time) is the raw glider data file sensor name. All additional keys and their functions are defined below:
-
attrs: dictionary that contains the NetCDF variable attributes. You may place any additional attribute here and it will be added to the variable's attributes (with the exception of _FillValue and missing_value, see below)
- A _FillValue or missing_value attribute is used to create the variable's fill value.
- dimension: The variable's dimension name.
- is_dimension: boolean designating whether the variable is also a dimension.
- dimension_length: don't modify.
- nc_var_name: name of the variable in the NetCDF file
- dtype: numpy data type
The following raw glider sensor names are required to be present in the Raw Glider File Parser Output and are used by TrajectoryNetCDFWriter and ProfileNetCDFWriter classes to populate specific NetCDF variables (profile_time, profile_lat, profile_lon) as well as global attributes beginning with geospatial_ and time_coverage_. See documentation on raw glider file parser output for more information:
- llat_time
- llat_depth
- llat_pressure
- llat_latitude
- llat_longitude