NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Thu, Jan 9, 2014 at 4:22 PM, Charlie Zender <zender@xxxxxxx> wrote: > We use record and unlimited interchangeably. > A record dimenison is an unlimited dimension. got it -- and that explains the lack of references to unlimited... > yes, the rd1 map is not good for access speeds on 1D vars. > use the --cnk_dmn=time,sz option to override this. or use lfp map... got it. > > Not so clear from the amount of time I've spent reading that, but what > > would be the default chunking for a 1-d unlimited variable? or a 2-d, > > with one dimension very small (Nx3, for instance)? > > for lfp map these are [N] and [N/3,3] where N is total chunksize, as > you can see with these commands: > ncks -O -4 --cnk_plc=all --cnk_map=lfp ~/nco/data/in.nc ~/foo.nc > ncks -m --cdl --hdn ~/foo.nc > > We may change the NCO default map from rd1 to lfp or something > similar in 4.4.1. > I think that's a good idea --I'm sure most users don't "get" chunking -- I sure didn't until recently. So good default are key. Note that in our case, the default (chunk size 1) writing a bunch of big 1-d arrays nto only was slow, but it made for huge files, and ended up crashing out our model. Changing that to 1024 chunk made a huge difference. So I think there should be a default minimum chunk size for any record variable -- maybe about 1024, but not sure about that. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@xxxxxxxx
netcdfgroup archives: