Compressed NetCDF & HDF

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: Compressed NetCDF & HDF
• From: "William Weibel" <weibel@xxxxxxxxxxxxxxxxxx>
• Date: Thu, 9 Apr 1998 11:06:43 -0700

A major conceptual difference between these two approaches is that
znetcdf defines a new filetype, whereas HDF files containing compressed
objects are still defined as HDF.

A "compressed variable" approach could be applied to NetCDF, thus avoiding
a new *DF type.  Perhaps the relevant compression parameters could be stored
as attributes, such as 'compressed = "true";' and 'compression_factor = 5;'.
Global atts could hold default parameters, while variable atts would hold
settings for individual variables.

Just a thought,
Bill Weibel  (Not the original Bill)

P.S.  At least twice now, I've seen the NetCDF magic string referred to as
"CDF1"*, yet in the actual files, the value is "CDF\001".
Does anyone know how this inconsistency got going?


*"Graphics File Formats" by Brown and Sheperd, p 400.
On Apr 9, 12:17pm, Arlindo da Silva wrote:
> Subject: Re: Compressed netCDF library available
> Bill,
>
> >
> > Arlindo -- I haven't looked at their library.  Do you have a link
> > to it?  --Bill
> >
> > > If I am not mistaken, this is precisely what MFHDF (NCSA's NetCDF API
> > > for HDF) does thru an interface to zlib. Have you compared your
> > > approach with theirs? I makes me a little worried to have yet another
> > > *DF format around.
> >
>
> The HDF home page is:
>
>       http://hdf.ncsa.uiuc.edu/
>
> For information on data compression look at the FAQ:
>
>     http://hdf.ncsa.uiuc.edu/HDF-FAQ.html#18
>
>
> Quoting:
>
> 18) Does HDF support data compression?
>
> HDF 4.0 (and later releases) supports a low-level compression
> interface, which allows any data-object to be
> compressed using a variety of algorithms.
>
> Currently only three compression algorithms are supported: Run-Length
> Encoding (RLE), adaptive Huffman, and an
> LZ-77 dictionary coder (the gzip 'deflation' algorithm). Plans for
> future algorithms include an Lempel/Ziv-78 dictionary
> coding, an arithmetic coder and a faster Huffman algorithm.
>
> HDF 4.0 (and later releases) supports n-bit compression for SDSs.
>
> HDF 4.0 (and later releases) supports RLE (Run Length Encoding),
> IMCOMP, and JPEG compression for raster images.
>
> New with HDF 4.1 is support for "chunking" and "chunking with
> compression". Data chunking allows an n-dimensional
> SDS or GR image to be stored as a series of n-dimensional chunks. See
> the HDF User's Guide for more information.
>
>
> --
> Arlindo da Silva
> dasilva@xxxxxxxxxxxxx
>-- End of excerpt from Arlindo da Silva



--
 |||| | | | |  |  |  |   |   |    |    |    |   |   |  |  |  |  | | | | | ||||
   William Weibel                      weibel@xxxxxxxxxxxxxxxxxx
   Jet Propulsion Laboratory           Tel. (818)354-0728            \\\\/
   4800 Oak Grove Dr.  MS 183-501                  4-1812             O-O
   Pasadena, CA 91109-8099             Fax  (818)393-5103              |
   U.S.A.                                                              -

 ||||DISCLAIMER:  Neither JPL nor I speak for each other. |  |  | | | | | ||||


---End of forwarded mail from Majordomo-Owner@xxxxxxxxxxxxxxxx

-- 
 |||| | | | |  |  |  |   |   |    |    |    |   |   |  |  |  |  | | | | | ||||
   William Weibel                      weibel@xxxxxxxxxxxxxxxxxx
   Jet Propulsion Laboratory           Tel. (818)354-0728            \\\\/
   4800 Oak Grove Dr.  MS 183-501                  4-1812             O-O
   Pasadena, CA 91109-8099             Fax  (818)393-5103              |
   U.S.A.                                                              - 

 ||||DISCLAIMER:  Neither JPL nor I speak for each other. |  |  | | | | | ||||