Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used

• To: Orion Poplawski <orion@xxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
• From: Jim Edwards <jedwards@xxxxxxxx>
• Date: Thu, 8 Jan 2015 16:16:48 -0700

pigz is only shared memory parallel and decompression is still serial so
it's not really a solution.

On Thu, Jan 8, 2015 at 4:10 PM, Orion Poplawski <orion@xxxxxxxxxxxxx> wrote:

> On 01/07/2015 02:52 AM, Heiko Klein wrote:
> > Hi Russ,
> >
> > thanks for your answer and the explanation from the hdfgroup.
> >
> > For our purposes, it is not the parallel IO, which is important, but the
> > bottleneck is in many cases zlib. Are there other ways to parallelize the
> > 'zlib' portion of netcdf/hdf?
> >
> > Heiko
>
> Perhaps looking at the pigz code would be useful for this.
>
>
> --
> Orion Poplawski
> Technical Manager                     303-415-9701 x222
> NWRA, Boulder/CoRA Office             FAX: 303-415-9702
> 3380 Mitchell Lane                       orion@xxxxxxxx
> Boulder, CO 80301                   http://www.nwra.com
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
Jim Edwards

CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO

• Follow-Ups:
  • Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
    • From: Russ Rew

• References:
  • [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
    • From: Heiko Klein
  • Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
    • From: Russ Rew
  • Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
    • From: Heiko Klein
  • Re: [netcdfgroup] parallel netcdf4 fails on nc_enddef when deflate-filter is used
    • From: Orion Poplawski