NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
I don't think you can chunk an unlimited dimension by more than 1. What are the variable dimensions? Your formula makes it sound like they are 1-D and only sized by the unlimited dimension. If that is the case, compression won't help. You might be better off with a netcdf-3 file? -- Ted On Jan 9, 2012, at 8:15 AM, Ross Williamson wrote: > I'm trying to get my head around the filesize of my netcdf-4 file - > Some background. > > 1) I'm using the netcdf_c++4 API > 2) I have an unlimited dimensions which I write data to about every second > 3) There are a set of nested groups > 4) I'm using compression on each variable > 5) I'm using the default chunk size which I think is 1 for the > unlimited dimensions and sizeof(type) for other dimensions > 6) I take data for 900 samples - There are about 100 variables so I > would expect (given doubles) a file size of 900x100x4 = 360K. Now I > fully expect some level of overhead but my file sizes are 5MB which is > incredibly large. > > Now compression doesn't make much difference (5Mb vs 5.3Mb). I'm > assuming here the thing that is screwing me over is that I haven't got > my chuncking set right. The issue is that I'm rather confused. It > appears that you set the chunk size for each variable rather than the > whole file which doesn't make sense to me. Would I just say multiply > each chunk size by say 100 so have 100 for the unlimited dimension and > sizeof(type)*100 for other dimensions? > > I'd really like to fix this as netcdf-4 seems ideal for my project but > I can't deal with a size overhead of an order of magnitude. > > I can attach the header of the netcdf file if it helps. > > Ross > > -- > Ross Williamson > Associate Research Scientist > Columbia Astrophysics Laboratory > 212-851-9379 (office) > 212-854-4653 (Lab) > 312-504-3051 (Cell) > > _______________________________________________ > netcdfgroup mailing list > netcdfgroup@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/
netcdfgroup archives: