[netcdfgroup] Fixed length array of VLA of compound types?

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] Fixed length array of VLA of compound types?
• From: Nick Johnson <Nick.Johnson@xxxxxxxx>
• Date: Mon, 09 Jul 2012 10:14:13 +0100

Hi,

I'm using netcdf4.1.3 (although 4.2 is also possible) in C. I currently have a 
code to which I've added basic
netcdf4/hdf5 parallel support as opposed to the initial serial input. This 
works well but I'd like to scale up and my
rather naive method may not work so well so I wondered if any experts could 
voice their opinions...

I currently store my 3D data as a fixed length array of a compound type (x,y,z 
co-ords and 2 items of data per point).
This is somewhat efficient as the data space is only about 4-5% populated and 
therefore I can get away with only storing
an entry in the dataset for the populated points.

I'd like to change this to use some of the data-space dimensions as dimensions 
in the dataset so that any code reading
the dataset doesn't have to do parsing and re-distribution of the data. Most of 
the processing is done in chunks of
z-plane, so I thought I would use the z-dimension of the data as the primary 
dimension of the dataset, define a variable
over this dimension and at each point have a variable length array (VLA) of a 
compound type.

This would appear to be possible from the documentation but I cannot get my 
head around how I would define this, I
couldn't find a suitable example to start from. Could anyone give me a pointer 
on how to do this (or point me to an
example if there is one I'm missing)?

Also, is this likely to be wise with regards compression? I currently compress 
the data using the deflate option which
gives me decent compression however I'm not sure if this would be sensible with 
the new scheme.


Cheers,
-Nick.

-- 
Nick Johnson, Applications Developer, EPCC
2407 JCMB, King's Buildings, Mayfield Road, Edinburgh, EH9 3JF
e: Nick.Johnson@xxxxxxxx t: 0131 651 3388

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.