Re: [netcdfgroup] nc_open takes a long time to open a big file

• To: "'Jennifer Adams'" <jma@xxxxxxxxxxxxx>, "'netCDF Mail List'" <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] nc_open takes a long time to open a big file
• From: Denis Nadeau <denis.nadeau@xxxxxxxx>
• Date: Thu, 9 Sep 2010 20:06:27 -0400

Hi Jennifer,

 

Did you call nc_set_chunk_cache?  If not, do you know what the default cache
size was set up to?  (Look in your config.log created after you called
configure)

I must admit that 20 seconds is quite long.

 

Denis

 

 

From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jennifer Adams
Sent: Thursday, September 09, 2010 7:34 PM
To: netCDF Mail List
Subject: [netcdfgroup] nc_open takes a long time to open a big file

 

Dear Experts,

I'm using netcdf-4.1.1-rc1 and hdf5-1.8.4-patch1 on a 64-bit linux server
running CentOS-5.5.

I have a netcdf-4 file that is 18404502496 bytes large. 

 

My file's dimensions look like this:

        lon = 320 ;

        lat = 160 ;

        lev = 11 ;

        time = 1581 ;

 

It has 7 variables that look like this:

   float temp(time, lev, lat, lon) ;

                temp:_Storage = "chunked" ;

                temp:_ChunkSizes = 1, 1, 160, 320 ;

                temp:_DeflateLevel = 1 ;

                temp:_Shuffle = "true" ;

 

and 1 variable that looks like this: 

    float sfp(time, lat, lon) ;

                sfp:_Storage = "chunked" ;

                sfp:_ChunkSizes = 1, 160, 320 ;

                sfp:_DeflateLevel = 1 ;

                sfp:_Shuffle = "true" ;

 

Is it normal for nc_open to take 20 seconds to open this file before
returning control to my C program? 

 

--Jennifer

 

 

--

Jennifer M. Adams

IGES/COLA

4041 Powder Mill Road, Suite 302

Calverton, MD 20705

jma@xxxxxxxxxxxxx

 

 

 

• Follow-Ups:
  • Re: [netcdfgroup] nc_open takes a long time to open a big file
    • From: Jennifer Adams

• References:
  • [netcdfgroup] nc_open takes a long time to open a big file
    • From: Jennifer Adams