Re: [netcdfgroup] Netcdf-4 and parallel I/O

• To: "Hernan G. Arango" <arango@xxxxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] Netcdf-4 and parallel I/O
• From: Ed Hartnett <ed@xxxxxxxxxxxxxxxx>
• Date: Tue, 27 Jan 2009 09:44:56 -0700

"Hernan G. Arango" <arango@xxxxxxxxxxxxxxxxxx> writes:

> Hi All,
>
> I implemented the parallel I/O in ROMS (http://www.myroms.org) using
> the latest snapshot of the NetCDF-4/HDF5 libraries.
>
> I have two questions:
>
> (1) The documentation for "nf90_open_par" says that it is only
>      possible to open a NetCDF4 format file for parallel processing.
>      This makes sense if the file is open for writing.  However,
>      I think that it should be able to open a classic format file
>      (NetCDF 3.x) for just reading since the library is backward
>      compatible.  I tested this in ROMS and in fact it is reading
>      a classic format in parallel as far as I can see in the
>      debugger. I am not going inside the libraries.  So the
>      question is this actually happening?  Is the library reading
>      a classic format in parallel?  If so, perhaps the documentation
>      needs to be revised.
>
>
http://www.unidata.ucar.edu/software/netcdf/docs_snapshot/netcdf-f90.html#NF90_005fOPEN_005fPAR

This is a very good question.

Parallel reads with netCDF classic files work because of the fseek
parallelism that work in parallel situation. Since the classic file is
laid out in a simple and strict way, each process ends up calculating
their offsets separately, and when the netCDF library reads the file,
it is able to do so in parallel.

I have added the following to the documentation:

   Since the API for netCDF-4/HDF5 files is the same as the API for
   classic files, parallel I/O code written for netCDF-4/HDF5 will
   also work for classic files, when reading the file.

   It is not possible to create a classic or 64-bit offset file with
   parallel access, or write to classic or 64-bit files in parallel
   with the netCDF library.

This is in the NetCDF Users' Guide:
http://www.unidata.ucar.edu/software/netcdf/docs_snapshot/netcdf.html#Parallel-Access

> (2) Why doesn't the library create identical files with the
>      same executable? Are there any time makings or other execution
>      information hidden in the created NetCDF-4 file.  I tried to run
>      the same execute twice and a binary difference tells me
>      that the files are different:
>
>      > diff new.nc old.nc
>      Binary files new.nc and old.nc differ
>
>      However, if I dump the data into ASCII files, using "ncdump", the
>      files are identical.
>
>      I need to understand this issue because this is an important
>      parallel partition test.  Regardless of the parallel partition
>      or number of MPI nodes, we should get identical solutions.
>      Otherwise, we have a parallel bug.
>
>      If there is an internal marking, how should we deactivate it?

I do not believe you can deactivate this; it is a characteristic of
HDF5.

I have forwarded your question to the HDF5 programmers for
discussion.

Thanks,

Ed

--
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx

• References:
  • [netcdfgroup] Netcdf-4 and parallel I/O
    • From: Hernan G. Arango