NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Chris and Russ, I read with interest your discussion on chunking. I added Chris's suggestion to NCO's supported chunking options. NCO 4.4.0 now implements six "chunking maps" http://nco.sf.net/nco.html#cnk including Chris's that we call the "lfp" map. When NCO introduced chunking support in 2009 I was under the impression that the record dimension _had to have_ a chunksize of one. Maybe I mistook this default (1) as mandatory for the record dimension. Anyway, now it can be anything and, with lfp, it automatically expands until the total chunksize for the array matches the user request (--cnk_byt) in bytes. We'll work on optimizing chunking maps in NCO 4.4.1. Would certainly like to include Russ's balanced 1-D/3-D access algorithm. Unfortunately we lack resources to benchmark access time for the results. So our goal is instead to give users a flexible menu of per-file, per-dimension, per-variable, and per-access type chunking options that they can benchmark. Hopefully something will be learned about optimal defaults in the process... Charlie -- Charlie Zender, Earth System Sci. & Computer Sci. University of California, Irvine 949-891-2429 )'(
netcdfgroup archives: