NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Another recent possibility that supports parallel compression is Peter Lindstrom's zfp <http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6876024>. On Thu, Jan 8, 2015 at 4:16 PM, Jim Edwards <jedwards@xxxxxxxx> wrote: > pigz is only shared memory parallel and decompression is still serial so > it's not really a solution. > > On Thu, Jan 8, 2015 at 4:10 PM, Orion Poplawski <orion@xxxxxxxxxxxxx> > wrote: > >> On 01/07/2015 02:52 AM, Heiko Klein wrote: >> > Hi Russ, >> > >> > thanks for your answer and the explanation from the hdfgroup. >> > >> > For our purposes, it is not the parallel IO, which is important, but the >> > bottleneck is in many cases zlib. Are there other ways to parallelize >> the >> > 'zlib' portion of netcdf/hdf? >> > >> > Heiko >> >> Perhaps looking at the pigz code would be useful for this. >> >> >> -- >> Orion Poplawski >> Technical Manager 303-415-9701 x222 >> NWRA, Boulder/CoRA Office FAX: 303-415-9702 >> 3380 Mitchell Lane orion@xxxxxxxx >> Boulder, CO 80301 http://www.nwra.com >> >> _______________________________________________ >> netcdfgroup mailing list >> netcdfgroup@xxxxxxxxxxxxxxxx >> For list information or to unsubscribe, visit: >> http://www.unidata.ucar.edu/mailing_lists/ >> > > > > -- > Jim Edwards > > CESM Software Engineer > National Center for Atmospheric Research > Boulder, CO >
netcdfgroup archives: