Re: Packing and Grib to netCDF conversion

• To: NetCDF mailing list <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Re: Packing and Grib to netCDF conversion
• From: Timothy Hume <tim.hume@xxxxxxxxxxxxxx>
• Date: Thu, 3 Oct 2002 21:07:47 +0000 (UTC)

Hi everyone,

I have thought quite a bit about a general GRIB to NetCDF converter. There
are a number of issues to consider:

GRIB data are stored in two dimensional records (one record per
horizontal level). To make the data more useful, it is nice to be able to
slot the two dimensional records into a three or four dimensional NetCDF
array (where the third and fourth dimensions are vertical level and time).

At first sight, it would seem trivial to simply stack data on pressure
levels into arrays with a pressure dimension, data on sigma levels into
arrays with a sigma dimension and so on. However, even data on supposedly
the same type of vertical level does not necessarily want to be stored on
the same dimension. For example, relative humidity data often only go as
high as 200 or 300 hPa, but other data go to 30 or 10 hPa. If compactness
is an issue, you might want to store relative humidity on a separate
pressure dimension from temperature data.

Then there is the issue of data on "slab" levels (eg average temperature
between 1000 and 850 hPa etc). This can be dealt with using more than one
co-ordinate variable for the vertical dimension, or a two dimensional
coordinate variable.

There are numerous other difficulties. The only general solution that I
could think of (that didn't require configuration information to be
supplied by the user) was to store GRIB data in two dimensional arrays
that match the GRIB two dimensional records. However, this makes it
difficult to do vertical cross-sections etc. If the user was prepared to
supply extra configuration information, then it would be possible to have
a "control file" that specified how data with a specific GRIB PDS would
map into a NetCDF array. That way, it should be possible to have rules
that said:

GRIB PDS (translated into English)      NetCDF array
------------------------------------    -------------------------
Relative humidity data on prs levels -> NCvar(record, t, Prh, x, y)
Any other data on prs levels         -> NCvar(record, t, P, x, y)
Any data on sigma levels             -> NCvar(record, t, sigma, x, y)
Any other data                       -> NCvar(record, x, y)

The record dimension is necessary to ensure each GRIB record can be mapped
uniquely into the NetCDF file (otherwise we might get data overwriting
each other). For example, if there was no rule for mapping 10m winds into
a NetCDF array, then 10m winds at t=0h and t=12h would overwrite each
other unless a record dimension was used (in which case the data for t=0h
would go onto record=1, and the data for t=12h would go onto record=2). In
the most general case, where no user configuration data was supplied, then
the GRIB data would simply map into arrays:

GRIB(record_n) -> NCvar(n, x, y)

Tim Hume

>
> gribtonc uses NUWG, and has various limitations. We are interested in
> possibly upgrading gribtonc. If anyone can make use of more flexible
> GRIB to netCDF conversion, I'd like to hear about your "use-case".
>
> I think in order to correctly group the GRIB records into 3 or 4
> dimensional netCDF variables, you will need (for the general case) some
> sort of configuration info for the converter, although I suppose a
> "common case" could be assumed. Anyone have any thoughts on that?

• References:
  • Packing and Grib to netCDF conversion
    • From: John Caron