[netcdfgroup] taking szip write capability out of netCDF snapshot code...

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] taking szip write capability out of netCDF snapshot code...
• From: Ed Hartnett <ed@xxxxxxxxxxxxxxxx>
• Date: Tue, 27 Oct 2009 07:42:25 -0600

Howdy all!

Recently I announced the availability of two new C functions,
nc_def_var_szip and nc_inq_var_szip, (and equivalent code for F77 and
F90) to allow full netCDF-4 support for szip compression.

Since netCDF-4 can already read szip files, nc_inq_var_szip is
not controversial. It simply reports existing szip settings. The
controversial part is allowing the user to *write* szipped netCDF-4
data.

After I added this we had a discussion within the netCDF programming
team, and have decided to take out the capability to write szipped files
from netCDF-4. Our discussion is summarized below for those users who
are interested in szip compression.

Szip Pros:

* Faster than zlib.
* Requested by some users. (But one, NASA GMAO, has since renounced
  using szip because it provides too many data distribution problems.)
* Already in use in HDF5 world.
* No licensing restrictions for reading.
* Read-access already possible in netCDF-4 with szipped HDF5 files.
* Source code is freely available - no danger of szipped data being lost
  to science.

Szip Cons:

* No existing Java version.
* License restrictions for commercial writers of data.
* Some (or most) netCDF-4 installations will not be able to read szipped
  files without rebuilding netCDF.
* Will not (and should not) be used by CMIP5 effort and (probably) other
  important archives. 
* Due to licensing szip will not be available in stock Fedora
  distribution. Fedora is a very popular Linux distribution, and at least
  some other free software distributions will probably feel the same
  about szip licensing problems.

Conclusion:

I will take out the def_var_szip function in all APIs. NetCDF users will
be able to use existing HDF5 szipped data, but not create it with the
netCDF API.

Thanks,

Ed

-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx

• Follow-Ups:
  • Re: [netcdfgroup] taking szip write capability out of netCDF snapshot code...
    • From: Francesco P. Lovergine
  • Re: [netcdfgroup] taking szip write capability out of netCDF snapshot code...
    • From: Ted Mansell