NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [netcdfgroup] Inquire about cache size and nslots ?


On Nov 17, 2009, at 12:25 PM, Ed Hartnett wrote:


Jennifer Adams <jma@xxxxxxxxxxxxx> writes:


Dear Experts,Â
I am working with the 4.1 snapshot of 20091102. I have been using the nc_inq_libvers() routine to discover the version of the library that is 
linked with my program (GrADS). Now I am testing various configure
options, specifically "--with-chunk-cache-size=n" and
"--with-chunk-cache-nelems=n".  Is there any way to discover these cache parameters from within my program -- something like nc_inq_cache()?  Proper settings for cacheing and chunking are critical for optimizing performance when working with compressed netcdf files. It would be good if the users 
could know these settings before setting chunk sizes for creating
compressed netcdf files.Â
--Jennifer
--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
[1]jma@xxxxxxxxxxxxx



Howdy Jennifer!
Thanks for asking a question for which there is an easy answer. Some of 
the recent support issues have been real stumpers!


I'm glad it wasn't too tough, and also that the answer was 'YES'.


There are functions to both set and learn about the chunk
cache. Changing the cache settings only applies to future opens/ creates, 
it does not affect already-open files.




These don't tell you what the file was opened/created with, just the
current settings. So if you change them, and want to know what they are 
for each file you open/create, then you must remember what they were
when the file was opened/created.
Let me see if I have understood properly ... The cache parameters are set on a per-file basis, but copied from global settings at the moment I open the file? And the original global settings are those used at compile time?
What if I open a file, query the chunk and the cache sizes, and discover that the cache isn't big enough to hold a single chunk and my I/O is doomed. Do I have to close the file, reset the cache size, then open the file again? 



/* Set the cache size, nelems, and preemption policy. */
EXTERNL int
nc_set_chunk_cache(size_t size, size_t nelems, float preemption);

/* Get the cache size, nelems, and preemption policy. */
EXTERNL int
nc_get_chunk_cache(size_t *sizep, size_t *nelemsp, float *preemptionp); 

Your question makes me note that these functions are not documented. I
am adding them now, and they will be in the snapshot documentation
tomorrow.

OK. That will be a big help. Thanks!
--Jennifer
  • 2009 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: