Re: Bzip2 Compression

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: Re: Bzip2 Compression
• From: Phil Rasch <pjr@xxxxxxxx>
• Date: Wed, 9 Aug 2000 09:30:03 -0600

The compression gains must be extremely data dependent.

Here are the results for the compression of a dataset that consists of
32bit floats of a collection of many meteorological fields from the NCAR CCM3:

35577060 bytes ENSANNAVG-0105.nc
26739972 bytes ENSANNAVG-0105.nc.bz2   

The compression is very small.

Phil

On Wed, Aug 09, 2000 at 08:03:50AM -0600, Roy Mendelssohn wrote:
> There was some question as to how well Bzip2 would compress floating 
> point data.  We just did a test on a large NetCDF file, containing a 
> years worth of global, 1-degree six-hourly pressure fields, so that 
> the data is floating point.  here are the sizes from a Unix ls 
> command (i.e in 512 byte units).
> 
> raw.nc        380,551,616
> 
> raw.nc.Z      143,716,211
> 
> raw.nc.bz2     55,633,062
> 
> 
> 
> Bzip2 gave about the same 7:1 compression that Russ reported, and 
> roughly 3 times better that Unix compress.
> 

-- 
----------------------------------------
Phil Rasch, Climate Modeling Section, National Center for Atmospheric Research
P.O. Box 3000, Boulder CO 80307
internet: pjr@xxxxxxxx, http://www.cgd.ucar.edu/cms/pjr
Phone: 303-497-1368, FAX: 303-497-1324 

• Follow-Ups:
  • Re: Bzip2 Compression
    • From: Laurie Carson

• References:
  • Bzip2 Compression
    • From: Roy Mendelssohn