CosmoNetcdfConversion

Conversion to the netcdf observation format for COSMO model

Note: this section is specific for the COSMO model, see also this presentation at 2011 COSMO General Meeting (may require authentication) for more information on the motivations and scope of bufr2netcdf.

Performing the conversion

If the input BUFR file comes from the GTS, a preliminary "lightweight" conversion has to be performed:

dbamsg convert --bufr2netcdf-categories --template=wmo \
 wmo.bufr > wmo_cosmo.bufr

This is the same command used for template conversion: when the input is already coded with WMO template its effect is mainly to modifiy data category, subcategory and local category in order to match the categories expected by the COSMO model. This step is not necessary if the message comes from the DWD observation processing system or it has been converted from another template as indicated in the Template conversion page (using the --bufr2netcdf-categories switch).

Afterwards, the conversion of a BUFR file wmo_cosmo.bufr is simply performed with the command

bufr2netcdf -o obs wmo_cosmo.bufr

the output will be a set of files named according to obs-<categ>-<subcateg>-<localsubcateg>.nc; if required, additional files for the same category/subcategory will be created, with the name obs-<categ>-<subcateg>-<localsubcateg>.i.nc i=1,...n. Notice that the bufr2netcdf program is able to convert almost any BUFR message to Netcdf format, but the COSMO model is able to import only Netcdf files created from a BUFR which was coded according to the proper template for each kind of observation (e.g. WMO for SYNOP and TEMP).

If the bufr2netcdf package has been installed in a nonstandard location, you have to preliminarly set the following environment variable:

export B2NC_TABLES=$PREFIX/share/bufr2netcdf

where $PREFIX is the installation prefix used when building and/or installing bufr2netcdf (default /usr).

Renaming the files for COSMO model

COSMO model expects the netcdf files with a very peculiar naming scheme, so the following script may help:

#!/bin/sh

makelink() {
# link first file
    test -s "$1" && ln -sf $1 $2.nc
# link following files if they exist
    base=${1%.nc}
    for file in $base.*.nc; do
        if [ -s "$file" ]; then
            ext=${file#$base}
            ln -sf $file $2$ext
        else
            break
        fi
    done
}

OBSPREFIX=obs
# Cleanup
rm -f $OBSPREFIX*.nc
# Perform conversion
bufr2netcdf -o $OBSPREFIX wmo_cosmo.bufr
# Link the files
makelink $OBSPREFIX-0-0-13.nc 'cdfin_synop'
makelink $OBSPREFIX-0-0-14.nc 'cdfin_synop_mob'
makelink $OBSPREFIX-1-0-255.nc 'cdfin_ship'
makelink $OBSPREFIX-2-4-255.nc 'cdfin_temp'
makelink $OBSPREFIX-2-5-255.nc 'cdfin_tempship'
# non-p pilot should be disabled if converting from ECMWF template
# until a bug in template conversion is fixed
# makelink $OBSPREFIX-2-1-4.nc 'cdfin_pilot'
makelink $OBSPREFIX-2-1-5.nc 'cdfin_pilot_p'
makelink $OBSPREFIX-4-0-8.nc 'cdfin_amdar'
makelink $OBSPREFIX-4-0-9.nc 'cdfin_acars'

If there are different messages in the same category having a different data descriptor record, they cannot fit in a single NetCDF file and they get split into as many output files as there are different DDR's, each with an additional number n in the name: $OBSPREFIX-<categ>-<subcateg>-<localsubcateg>.n.nc. This script takes into account this fact and makes the links to the additional NetCDF files if necessary, however the additional files for each observation category/subcategory, can be read only by COSMO version 4.22 or newer.