NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
To clarify: the way I see it, you can do parallel I/O in three different ways. The first is to reserve a process which will only deal with I/O and other process will exchange data to read/write with it. -- In a sense, this is not a parallel IO. It is using the sequential IO to handle the parallel applications. The second is to have each process read/write independantly. -- You may talk about the independent IO here. The third is to aggregate the I/O for several processes to improve performances. -- You may mean collective IO here. So my question was: in practice, which approach does parallel netCDF use ? -- You can do both independent IO and collective IO with parallel HDF5. Definitely collective IO for parallel NetCDF-3(argonne's parallel NetCDF) and very also very possible independent IO. > in strict performance terms -- which in the end is not really the > be-all end all -- Argonne-Northwestern Parallel-NetCDF will be hard to > beat, unless you are working with record variables. Do you speak from personal experience ? I would be very interested in seeing some data or benchmark about it. -- There should be a paper that listed the flash benchmark comparison between parallel NetCDF from Northwest(or parallel netcdf-3) and parallel HDF5. However, it is an unfair comparison. It used collective IO for parallel NetCDF-3 but independent IO for parallel HDF5. You can find more detailed about the fair comparison with the collective IO for these two packages from http://www.spscicomp.org/ScicomP12/Presentations/User/Yang.pdf Be aware this was also a bit old. Don't know what's the current status between these two packages. Kent -- Alexis Praga _________________________________________________________ Ph.D Student Aviation et Environnement CERFACS alexis.praga@xxxxxxxxxx (33) 05 61 19 31 74
netcdfgroup archives: