[netcdfgroup] Fwd: Questions about parallel-netcdf-c: COLLECTIVE mode V.S. INDEPENDENT mode

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] Fwd: Questions about parallel-netcdf-c: COLLECTIVE mode V.S. INDEPENDENT mode
• From: "Zhiyu (Drew) Li" <zyli2004@xxxxxxxxx>
• Date: Wed, 22 Feb 2017 09:54:14 -0700

(I forgot to 'join' the mailing list before posting my questions just now.
So I "joined" and am sending the same questions here again. Sorry for
duplication if any)

Hi there,

I am playing with the parallel-netcdf-c examples to learn if I could apply
this technology to improve netcdf i/o in my project. I got some questions
about this example tst_parallel4.c found at https://github.com/Unidata/net
cdf-c/blob/master/nc_test4/tst_parallel4.c.

I saw the statements "nc_var_par_access(ncid, varid, NC_COLLECTIVE)" and
"nc_var_par_access(ncid, varid, NC_INDEPENDENT)" are commented out on lines
133 and 134 (https://github.com/Unidata/netcdf-c/blob/master/nc_test4/ts
t_parallel4.c#L133).

Q1: Is this nc_var_par_access() statement optional?

Q2: I enabled each of the two lines one at a time to test NC_COLLECTIVE
mode and NC_INDEPENDENT mode separately. Each test was ran with 4 processes
(mpiexec -np 4 ./tst_parallel4). Then I used jumpshot to visualize the
clog2 files they produced. The snapshots are attached below. The green bars
represent "Write to netcdf file" events (I turn off other bars (other mpi
events) in visualization).

[image: Inline image 1]
                             NC_INDEPENDENT mode
In NC_INDEPENDENT mode, the Write events occurred at different time steps
in the 4 processes (the x-axis is time step). If I understood it correctly,
although we had 4 processes running in parallel, the Write events still
happened in sequence, not in parallel, because p0 wrote first, then p1
wrote, and then p2, and then p3 wrote last. Is it supposed to be like
this???


[image: Inline image 2]
                             NC_COLLECTIVE mode

In NC_COLLECTIVE mode, p0 started writing first but its Write event lasted
until the fourth process p3 finished writing. I thought all the four
process should start and stop writing at the same time in NC_COLLECTIVE
mode???


My testing environment:
a local ubuntu VM (16.04 x64) with 4 CPUs in Virtual Box
mpich 3.2
hdf5 1.10
netcdf4-c 4.4.1.1

Thank you.


Zhiyu (Drew) Li
Research Associate
Civil and Environment Department, Engineering College
Brigham Young University
Provo, UT
USA

• References:
  • [netcdfgroup] Questions about parallel-netcdf-c: COLLECTIVE mode V.S. INDEPENDENT mode
    • From: Zhiyu (Drew) Li