gribtoqd is a program that converts files from grib-format (version 1 and 2) to querydata-format. It is used when data is imported from external sources to SmartMet-system.
Before using gribtoqd user should know something about grib file structure. It is a bit different between grib1 and grib2. Grib files have data grids for each timestep parameter and level. Each parameter has number (different between grib1 and grib2), each level has level id and level value f.eg. 103 means fixed height level and value 2 means meter above mean sea level.
gribtoqd [options] inputgribfile|inputfilemask
The Most Common options:
- -o outputfile
By default data is writen to stdout. - -c paramChangeTableFile
Mapping between grib parameter id and smartmet id are done in this configuration file. It is also possible to do some scaling f.eg. unit conversions. File should contain semicolon separated values GribID;SmartMetID;Name;Base;Scale;LevelType;LevelValue F.eg. to take 2m temperature in kelvins from grib2 file and to change it to celcius add: 0;4;T;-273.15;1;103;2 If level type and value are not specified, conversion is done to every level that is found in the data. Note that in grib2 files parameter number is category + parameter. F.eg. Relative Humidity Cat 1 and Param 1 = 1001. - -p producerId,producerName
Set querydata producer id and name f.eg. 240,ECMWF Surface - -P <projectionString>
Define projection that the data will be projected to. Use SmartMet projection format. Give also the grid size or default 50x50 will be used. E.g. FMI Scandinavia area is stereographic,20,90,60:6,51.3,49,70.2:82,91 (where 82,91 is used grid size). You can also set pressurelevel and hybridlevel output data grid sizes separately by adding them after first grid size e.g. after 82,91 you add ,50,40,35,25 and pressure data has 50x40 and hybrid datat has 35x25 grid sizes). You can also give three different projections for surface-, pressure- and hybrid-data. Give two or three projections separated by semicolons ';'. E.g. proj1:gridSize1[;proj2:gridSize2][;proj3:gridSize3] - -n
Names output files by level type. E.g. output.sqd_levelType_100
Other options:
- -m <max-data-sizeMB>
Max size of generated data, default = 1024 MB - -l t105v3,t109v255,...*
Ignore following individual levels (e.g. 1. type 105, value 3, 2. type 109, value 255, etc.) You can use -wildcard with level value to ignore certain level type all together (e.g. t109v) - -L 105,109,...
Accept only following level types. - -D <definitions path>
Where is grib_api definitions files located, default uses path from settings (GRIB_DEFINITION_PATH=...). - -a
Try creating an Atlantic view of the data. If the data is in latlon projection and covers the globe using coordinates in range 0...360, the coordinates will be changed to range -180...180. - -A
Try creating a Pacific view of the data. If the data is in latlon projection and covers the globe using coordinates in range -180...180, the coordinates will be changed to range 0...360. - -S
Swap left and right hand halves of the data. - -y
Flip the data along the y-axis. - -C
Try to combine tiled GRIB files into larger combined grids. - -z
Read data lines in zig-zag fashion from left to right. - -i
Ignore reduced_ll data, try using grib_api instead of wgrib for the conversion - -t
Reports run-time to the stderr at the end of execution - -v
verbose mode - -d
Crop all params except those mensioned in paramChangeTable (and their mentioned levels) - -g printed-grid-info-count
If you want to print out (to cerr) info about different grids and projection information, give number of first grids here. If number is -1, every grids info is printed. - -G <x1,y1,x2,y2>
Define the minimal subgrid to be cropped by the bottom left and top right longitude and latitude. - -r <specHumId,hybridRHId=13,hybridRHName=RH>
Calculate and add relative humidity parameter to hybrid data. relative humidity parameter to hybrid data. - -H <sfcPresId,hybridPreId=1,hybridPreName=P>
Calculate and add pressure parameter to hybrid data. Give surfacePressure-id, generated pressure id and name are optional. pressure parameter to hybrid data. Give surfacePressure-id, generated pressure idand name are optional.
The most important level types are in the following table.
| ID | Name | Level Value |
|---|---|---|
| 100 | Isobaric level | Pressure in hPa |
| 103 | Altitude | Altitude above mean sea level |
| 105 | Height | Height above ground |
| 109 | Hybrid | NWP model level |
| 112 | Layer between two depths below land surface | Depth of upper surface (cm) / Depth of lower surface (cm) |
| 160 | Depth | Depth below mean sea level in meters |
| 200 | Flattened | Entire atmosphere considered as a single layer |
Full list of levels
The parameter change configuration file defines how the universal paramId as defined in grib_api documentation is to be converted into a Smartmer parameter.
This configuration file contains semicolon separated table of smartmet and grib id:s. Also basic scaling can be done with base and scale columns. Finally specific level can be selected with LevelType and LevelValue columns. If you omit level information, changes will apply for every level.
#GribID;SmartMetID;Name;Base;Scale;LevelType;LevelValue
# Pressure MSL
151;1;P;0;0.01
# Station Pressure SP
134;472;SP;0;0.01
# Geopotential Height GH
156;2;Z
# 2m Temperature 2T
167;4;T;-273.15;1
To get grib id, you can use wgrib2, wgrib and grib_dump command line programs from NOAA and ECMWF. To know SmartMet id:s use qdinfo -l command.
Note that if you are converting grib2 files, parameter id in this file is actually category number + parameter number f.eg. Relative Humdity Cat 1 and Parameter 1 = 1001
Information about parameter numbers in grib1 and grib2 files, can be found in following links. But in the end you should use above mentioned tools to see what is actually in the data.
- GRIB API Parameters
Note: gribtoqd uses wgrib internally to interpret reduced_ll GRIB1 files. In such cases the parameter conversion table is not expected to contain the universal grib_api paramId numbers, but the original GRIB1 parameter numbers.
To define projection use information from SmartMet projection page. Additionally grid size is needed, to maintain specific resolution or to make resolution more coarse, use qdgridcalc to calculate grid size for certain resolution.
qdgridcalc stereographic,10,90,60:-13.8,31.8,73.2,55.6 25 25
Grid size would be: 232 x 172
Grid size would be at center of bottom edge: 21.9274 x 21.8816 km
Grid size would be at center: 24.9983 x 24.9402 km
Grid size would be at center of top edge: 27.0825 x 26.9947 km
Here we calculate what would be the grid size for this projection and area, if we want 15x15km resolution.
grib2toqd -H 472 -n -t -r 1000 -c $CNF/ecmwf.cnf -p "240,ECMWF Surface,ECMWF Pressure,ECMWF Hybrid" -P "stereographic,10,90,60:-13.8 ,31.8,73.2,55.6:232,172;stereographic,10,90,60:-13.8,31.8,73.2,55.6:232,172;stereographic,10,90,60:5.5,51.1,45.0,60.1:86,64" -o /tmp/outname.sqd "/smartmet/data/incoming/L1D12140000*"
- -H gives gribId for surface pressure in gribfiles, this calculates pressure for each hybrid level
- -n names output files by level type, in this case outname.sqd_levelType_1, outname.sqd_levelType_100 and outname.sqd_levelType_109
- -t prints processing time for diagnostics purpose
- -r 1000 is gribId for specific humidity in order to calculate relative humidity for each hybrid level
- -c is the location of file that contains parameter id changes table
- -p "240,ECMWF Surface,ECMWF Pressure,ECMWF Hybrid" gives producer id and name pairs for each level
- -o /tmp/outputname.sqd is output filename
- "/smartmet/data/incoming/L1D12140000*" is input file mask, please note mandatory quotes