NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Karen, Oops, I didn't wait long enough after testing the "workaround" provided in the last reply. It turns out the code I suggested still eventually got an HDF5 error when the variable size exceeded 2**31 bytes on a Sun platform, due to a known HDF5 bug on Suns when writing large files. We're waiting on a fix for that from the HDF5 Group. I don't think you said what platform you were testing on, but if it was a Solaris/Sun, then what you saw was just a symptom of that large file bug in HDF5. So then I tried your original program, without specifying chunk shapes, on a Linux x86 platform, and it ran to completion without getting the HDF5 error you are seeing: $ cc -g -Dnetcdf4 -I/shecky/russ/nc4/include -on4err n4err.c -L/shecky/russ/nc4/lib -lnetcdf -L/upc/share/ed/local/shecky/lib -lm -lhdf5_hl -lhdf5 -lz ./n4err -d . -r 10 Using netcdf4 Time to define: 1 number of chunks 16 Time step0 Time to write step: 270 Time to write data: 274 Time to close: 0 *** SUCCESS writing example file ./r10.nc! Then just to see if using smaller chunks would make it faster, I inserted the suggested modification specifying the chunk shapes as 1 x 100 x 655360, with the following result: ./n4err -d . -r 10 Using netcdf4 Time to define: 0 number of chunks 16 Time step0 Time to write step: 88 Time to write data: 91 Time to close: 0 *** SUCCESS writing example file ./r10.nc! So it works on the Linux system even with the single large chunk, but it's about 3 times faster to write if you explicitly set the chunks to match your access pattern. I used the 4.0.1-beta2 release of netCDF and the 1.8.2 release of HDF5, but I don't think the results would differ using the 4.0 version of netCDF or the 1.8.1 version of HDF5. --Russ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: KVY-259447 Department: Support netCDF Priority: Emergency Status: Closed