Re: grib2 decoder

• To: Jim Cowie <cowie@xxxxxxxx>
• Subject: Re: grib2 decoder
• From: Robb Kambic <rkambic@xxxxxxxxxxxxxxxx>
• Date: Mon, 3 Jan 2005 16:41:52 -0700 (MST)

On Mon, 3 Jan 2005, Jim Cowie wrote:

Thanks for the update Robb. I figured the files were going to become
very large but I was thinking that in netCDF4 when HDF is the underlying
data storage layer that compression would help out here. Is that likely
or am I dreaming?

no that will help a bunch. but some of the new grib2 eta files are almost
400 megs, current expansion would result in 8 gigs. wow  don't know how
netCDF4 will handle that large of a data set. so on demand decoding will
be the most reasonable solution.

robb...


Maybe it is best to just degrib on the fly after all.

-jim


Robb Kambic wrote:
> On Mon, 3 Jan 2005, Jim Cowie wrote:
>
>
>>Hi Robb,
>>
>>I saw in one of your emails last month a mention that the new
>>gribtonc decoder (for grib2) was going to be released pretty soon.
>>Can you tell me when? We are antsy to try it. Thanks,
>
>
> jim,
>
> good news and bad news. the good news is the decoder has been developed
> and tested, the bad news is the resulting netCDF files are huge. the large
> file size is the main reason for the delay. the expansion for the files have
> a range from 1:20 to 1:50  on average. for example:
>
> NDFD  12.4 megs -> 626.9 megs
> eta2  16.3 megs -> 668.5 megs
>
> a couple reasons, the netcdf file is not optimized and the more missing
> values in the data the larger the expansion. also the complex packing
> scheme permits more variables and levels. john and i have been working
> together to leave the files in the grib2 format and to extract the data in
> an on demand process.  to implement this, the grib files are scanned to
> create an index of products and grid informations. his programs reads
> the index and presents the information to the user.  it works well. here's
> an url to down load ToolUI program.
>
> 
http://my.unidata.ucar.edu/content/software/netcdf-java/v2.2/webstart/index.html
>
> in a nutshell,
>
> click viewer  then open a grib2 file
> click geogrid  then open the same grid file  geogrid boxes should be
>    checked
> click view  to access a sanity type view of the variables.
>
> ToolUI currently  doesn't do grids with multiple GDS or thin grids, we are
> working on those features now.
>
>
> i'll probably be only available by email tomorrow if you have
> questions then i'll be on PTO for a week.
>
> robb...
>
>
>
>>-jim
>>
>>--
>>Jim Cowie
>>NCAR/RAL
>>cowie@xxxxxxxx
>>303-497-2831
>>

==============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx                   WWW: http://www.unidata.ucar.edu/
==============================================================================

• References:
  • Re: grib2 decoder
    • From: Robb Kambic