NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Ed Hartnett wrote:
Ken Knapp <Ken.Knapp@xxxxxxxx> writes:Hi-I'm kind of new to writing unlimited dimensions in netcdf and ran into some odd behavior when attempting to write unlimited dimension with netCDF4 and Fortran90. In short, I find two problems (for details of the problems see below examples and code): 1) writing unlimited dimensions in netCDF4 significantly increases file size over same data written with version 3.6. I realize I could turn on deflate ... but why would netcdf4-unlimited be 10x bigger than the ver.3.6 or ver.4??The file size is much greater for tiny files, but not that big for larger files. The netCDF-4 format is HDF5, which has much more header information than netCDF-3 classic format files. So a very small file (i.e. one without much data) will be bigger in HDF5 due to the increased header information. But once you start writing data to the file, the importance of the header is less, and when you are writing reasonably sized data files there should not be too much difference. For example, with a 600 x 1200 array of ints, I get the following sizes: -rw-r--r-- 1 ed ustaff 2880104 Aug 1 13:59 tst_unlims_3.nc -rw-r--r-- 1 ed ustaff 2917551 Aug 1 13:59 tst_unlims_4.nc2) writing large amounts of unlimited data seems impossible? Is there a performance issue? I realize my code is likely not optimized. Maybe I'm using the wrong netCDF flags? When attempting to write 1 million points with an unlimited dimension, ver3 file (not shown) completes quickly, ver.4 file never completed.Hmmm, I am having the same problem. I will look into this some more and get back to you. Thanks, Ed
Ed:For what it's worth, I'm also seeing very poor performance writing large chunks of data along the unlimited dimension in NETCDF4 and NETCDF4_CLASSIC (using the python interface, which is just a wrapper around the C interface). Works fine if the format is NETCDF3 though.
So, I don't think it's an issue with the fortran interface. -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@xxxxxxxx 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg ============================================================================== To unsubscribe netcdfgroup, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
netcdfgroup archives: