NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Yes, an HDF5 limitation, amplified by the delete and recreate cycle in netcdf-4 code, as described in GH #350. --Dave On Thu, Jan 26, 2017 at 4:02 PM, dmh@xxxxxxxx <dmh@xxxxxxxx> wrote: > Let me make sure I understand. That counter is an HDF5 limitation, > correct? > =Dennis Heimbigner > Unidata > > On 1/26/2017 3:46 PM, Dave Allured - NOAA Affiliate wrote: > >> All, >> >> We traced this problem to a known bug. In NetCDF-4, the number of times >> an attribute can be modified over the life of a file is currently >> limited by the per-variable HDF5 attribute creation index. This is a >> 16-bit counter with maximum value 65535. >> >> This becomes a problem for data sets with attributes that are updated >> frequently. At the NetCDF user level, the problem shows as return >> status -107, or "NetCDF: Can't open HDF5 attribute", as reported below >> by Cathy Smith. >> >> Please see this Github issue for more details. Thanks to Constantine >> Khroulev for a great analysis, as well as previous reporters Heiko Klein >> and Andrey Paramonov (HDF forum). >> >> https://github.com/Unidata/netcdf-c/issues/350 >> >> We look forward to some kind of fix at the NetCDF or HDF5 level. >> >> --Dave A. >> NOAA/ESRL/PSD/CIRES >> >> >> On Tue, Nov 15, 2016 at 8:45 AM, Cathy Smith (NOAA Affiliate) >> <cathy.smith@xxxxxxxx <mailto:cathy.smith@xxxxxxxx>> wrote: >> >> Hi all >> We are trying to figure out why we have been getting a particular >> HDF5 error when trying to change an attribute. I run update code for >> datasets via cron. At random times over the las year for at least 5 >> separate files in different datasets with different update code, we >> discovered we suddenly couldn't update the actual range attribute >> for a variable as the netCDF API did not see the attribute anymore. >> There was no obvious cause . Once the file was "broken" it stayed >> that way . Trying to update or fix the attribute gives the errors >> below. >> >> To fix the file we either restore the file from backup or regenerate >> it using nccopy on the broken file. >> >> Someone here has done some extensive testing. They found that >> Fortran, NCO, and NCL can't work on the attribute. Also, h5edit >> can't. But, some HDF5 applications that view the file do not see an >> error. And the file gives no error with ncdump though it doesn't >> show the actual_range attribute. >> >> Has anyone seen this error before? Any ideas on the cause come to >> mind? We are debating NFS issues or library as the cause but are >> leaning towards the former, at least partially. >> >> I have a broken and working file in ncftp ftp.cdc.noaa.gov >> <ftp://ftp.cdc.noaa.gov> >> cd Public/csmith/netcdf >> >> Thanks for any insight >> >> Cathy Smith >> ESRL/PSD >> >> Commands we ran to show error: >> ncdump -h precip.V1.0.2016.nc <http://precip.V1.0.2016.nc> >> <http://precip.V1.0.2016.nc> <http://precip.V1.0.2016.nc> |grep act >> lat:actual_range = 20.125f, 49.875f ; >> lon:actual_range = 230.125f, 304.875f ; >> time:actual_range = 1893408., 1900824. ; >> >> When I try to add it back, I get >> >> ncatted -h -O -a actual_range,precip,c,f,"100000.,-100000." >> precip.V1.0.2016.nc <http://precip.V1.0.2016.nc> >> <http://precip.V1.0.2016.nc> <http://precip.V1.0.2016.nc> >> nco_err_exit(): ERROR Short NCO-generated message (usually name of >> function that triggered error): nco_enddef() >> nco_err_exit(): ERROR Error code is -107. Translation into English >> with nc_strerror(-107) is "NetCDF: Can't open HDF5 attribute" >> nco_err_exit(): ERROR NCO will now exit with system call >> exit(EXIT_FAILURE) >> >> and >> ncatted -h -O -a actual_range,precip,o,f,"10000 >> >> 0.,-100000." precip.V1.0.2016.nc <http://precip.V1.0.2016.nc> >> <http://precip.V1.0.2016.nc> <http://precip.V1.0.2016.nc> >> nco_err_exit(): ERROR Short NCO-generated message (usually name of >> function that triggered error): nco_enddef() >> nco_err_exit(): ERROR Error code is -107. Translation into >> English with >> nc_strerror(-107) is "NetCDF: Can't open HDF5 attribute" >> nco_err_exit(): ERROR NCO will now exit with system call >> exit(EXIT_FAILURE) >> >> ---------------------------------------------- >> >
netcdfgroup archives: