[netcdfgroup] writing/reading a scalar variable in a NETCDF4 type file with parallel I/O

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] writing/reading a scalar variable in a NETCDF4 type file with parallel I/O
• From: "Hernan G. Arango" <arango@xxxxxxxxxxxxxxxxxx>
• Date: Wed, 28 Jan 2009 16:28:11 -0500

Dear Group,

Did any of you try reading/writing a scalar variable into
a NetCDF4 type file open/created for parallel I/O access?

I haven't been able to find documentation for this case.
What are the roules?

We either have to choices:

(1) Set independent access (NF90_INDEPENDENT) for such
    variable and just select a single parallel node
    (say, master node) to read/write. If reading, we need
    to broadcast such variable to all parallel nodes in
    the group.

(2) Set either independent(NF90_INDEPENDENT) or collective
    (NF90_COLLECTIVE) access for such variable and allow all
    the parallel nodes to participate during reading/writing. If
    writing, each node will write over and over the same value
    into the file provided that some barrier is set inside the
    HDF5 low level routines to avoid race conditions.
    Or you may convert such variable to an array with just
    1-element, say A(1), and set the count vector as
    (1, 0, 0, ..., 0), as suggeted coupled of days ago.

I tried both (1) and (2) and work in some applications but it
hangs during MPI communications in others during writing of
exactly the same variable.  I am confused about this and I
don't understand why.  I haven't been able to reproduce this
problem in a simpler program so others may take a look at it.

I am suspecting that there is a bug somewhere but I don't
if it is in the NetCDF-4 or HDF5 libraries.

I don't understand why the NETCDF-4 selected collective
operations as the default.  This may be dangerous and may be at
the root of my problem if collective/independent tests are not done
everywhere in the low level routines.  MPICH2 sometimes have
problems (misbehave) during collective I/O operations.

Any comments are appreciated...

Cheers, H

-----------------------------------------------------------------------

Hernan G. Arango           Institute of Marine and Coastal Sciences
arango@xxxxxxxxxxxxxxxxxx  Rutgers University
off:  (732) 932-6555 x266  71 Dudley Road
FAX:  (732) 932-6520       New Brunswick, NJ 08901-8521, USA

http://marine.rutgers.edu/po/arango
http://marine.rutgers.edu/po/arango/rocco
http://marine.rutgers.edu/roms
http://www.myroms.org
http://www.ocean-modeling.org