Re: [netcdfgroup] Patch for netCDF4 file bit-for-bit reproducibility

• To: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] Patch for netCDF4 file bit-for-bit reproducibility
• From: "Latham, Robert J." <robl@xxxxxxxxxxx>
• Date: Fri, 21 Feb 2014 17:56:48 +0000

On Mon, 2014-02-10 at 14:55 -0700, Russ Rew wrote:
> Hi Rimvydas,
> 
> > Recently I started to work with netcdf in fortran, mainly changing f77
> > interface to more flexible f90 one.
> > And I love it! Fantastic API.
> > 
> > I am dealing with code that has huge testsuite for regression testing,
> > so I am trying to found compromise for size and speed.
> > Code was intended to output lots of diagnostics (~1Gb) for every test.
> > 
> > Lack of ncdiff tool made me to write my one, but while trying to
> > optimize it for time
> > I learned that half the time I am spending in my comparison loops,
> > other half in swap8b...
> > NETCDF4 features like compression and native endianess are very appealing
> > but lack of BFB (even with nccopy) just because of internal timestamping is 
> > s
> > ad.
> > 
> > http://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2008/msg00003.
> > html
> > Is this still valid?
> 
> It looks like the fix you've developed and tested (first suggested by
> Quincey Koziol in January 2008) would make bit-for-bit reproducibility
> possible for netCDF-4 files.
> 
> We're testing turning off tracking times for HDF5 objects to determine
> if there are any undesirable side effects.  If not, we'll incorporate it
> into the next release.

Sounds like the atime problem, so not only will this allow BFB, it will
probably help performance a bit.

==rob

• References:
  • [netcdfgroup] Patch for netCDF4 file bit-for-bit reproducibility
    • From: Rimvydas Jasinskas
  • Re: [netcdfgroup] Patch for netCDF4 file bit-for-bit reproducibility
    • From: Russ Rew