NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Tue, Apr 24, 2007 at 11:31:20AM -0600, John Caron wrote: > Could you give use case(s) where the limits are being hit? > I'd be interested in actual dimension sizes, number of variables, whether > you are using a record dimension, etc. I'm just the intermediary, but I've copied two users who ran into this limit. Here's what Greg had to say: As a quick answer to the question, we use netcdf underneath our exodusII file format for storing finite element results data. If the mesh contains #nodes nodes and #elements elements, then there will be a dataset of the size #elements*8*4 (assuming a hex element with 8 nodes, 4 bytes/int) to store the nodal connectivity of each hex element in a group of elements (element block). Assuming 4GiB, this limits us to ~134 Million elements per element block which is large, but not enough to give us more than a few months breathing room. In both cases, they want to stick with a single netcdf file as they scale to very large machines and problems. They also like the stability of the CDF-2 format: there's no metadata corruption if a process dies in the middle of a write. If we (parallel-netcdf) make changes to either the API or file format to overcome CDF-2 limitations, would the serial-netcdf folks be interested in that work? Thanks ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Lab, IL USA B29D F333 664A 4280 315B ============================================================================== To unsubscribe netcdfgroup, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
netcdfgroup archives: