[netcdfgroup] Reading groups is very very slow. What am I doing wrong?

To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: [netcdfgroup] Reading groups is very very slow. What am I doing wrong?
From: Paul van Delst <paul.vandelst@xxxxxxxx>
Date: Wed, 30 Oct 2013 11:00:29 -0400

Hello,

I've just converted some of my netCDF writing code to write/readmultiple groups rather than use an unlimited dimension. I did this soeach group can have different "base" dimensions for the data arrays.

I have one data set where the unlimited dimension is 5000. Theread/write of this data in netCDF3 format is almost instantaneous. WhenI use the netCDF4 approach (reading and writing 5000 separate groups)the reads and write can take upwards of 10minutes (I started the programat 10:33am. It is now 10:51am and the read of the created file is stillgoing on).

I realise there's going to be additional overhead using the "groups"approach (defining dimensions and variables for each group) but Ipresume I'm doing something very wrong/stupid to cause the I/O to be asslow as it is. Before I start posting code snippets, does anyone haveany experience hints as to what could be causing this supa slow I/O?


Thanks for any info.

cheers,

paulv

p.s. It's now 11:00am and the dataset reading is still going on...

2013 messages navigation, sorted by:
1. Thread
2. Subject
3. Author
4. Date
5. ↑ Table Of Contents
Search the netcdfgroup archives: