NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On 11/07/2014 07:13 AM, Ed Hartnett wrote:
Howdy all! Before Ward's time, I was the one who split out the Fortran libraries in netCDF. Although this represents some extra inconvenience for Fortran programmers who want to build from source, it has a couple of benefits. 1 - Users who only want the C library don't get confused with the Fortran build. This was a significant problem when the releases were combined. Although the netCDF Fortran community is strong, there is an even larger set of tools and APIs that only need the C library. The Fortran build generated many more support questions than the C build. By providing a C-only library distribution we cut support questions pretty dramatically.
So, you orphaned us poor Fortran dinosaur programmers ... ... too bad because besides the (nice, high quality)tools that are based in NetCDF-C, virtually all ocean/atmosphere/climate models (and probably more software out there) is written in Fortran (of various generations).
How could the more expert programmers of those tools get more confused than we dinosaur Fortranists did after Fortran was dismembered from NetCDF? :)
2 - Splitting the libraries by language fits a lot better with the GNU/Linux worldview, and makes them easier and more attractive for dedicated volunteers to distribute via the Linux package management tools.
The argument isn't so strong, because the Fortran libraries are hierarchically dependent on the C ones (and in a dependence chain). Besides, the releases became totally independent, and who knows which Fortran library versions are consistent with such and such C library versions? The build became much more cumbersome.
On my stock Fedora machine I can type "yum install netcdf-fortran" and I will get the fortran build, and all the libraries it depends on.
Not everybody wears Fedora. RHEL/CentOS/SL, which are the stable distributions used in many servers and clusters are ages late w.r.t. NetCDF packages (4.1.1 I think, from Epel). Not sure about the situation with SLES, Debian/Ubuntu, and friends. But more important, packages are not always the best fit, actually the worst in many cases. Many programs depend on having libraries built with commercial compilers (eg. Intel, PGI, etc), won't link programs otherwise. So, one must have the ability to build from source with the compilers of choice, to have this flexibility. I (and many others) go through the trouble of building zlib, HDF5, the whole chain of NetCDF dependencies in order to provide a consistent and rich environment for compilation and linking.And on top of this build also some of the tools in the NetCDF constellation (NCO, CDO, etc).
That abiltiy is becoming hindered by the disconnected versions of the various NetCDF library interfaces distributed to the public.That can be easily remedied without hindering the developers preferences, by bundling them all into public releases with an
easy to use "configure/make/make-install" build system. Please, don't throw the baby with the bath water. Have you tried to compile NCAR, GFDL, MaxPlanckInstitute (ECHAM and friends), and other models with gfortran, and link them with NetCDF from packages? Please, give the a try ... F90 modules are not compatible across compilers, and if one uses NetCDF packages there is no game when the programs require commercial compilers.
This is by far the easiest way to install Netcdf, and it does not generate support questions.
Easy, sure. Effective, it is not, at least not for many applications.
NetCDF libraries are ported to most of the mainstream Linux distros I've encountered, and even on Windows with Cygwin. (But to use the Microsoft development tools you must use Ward's CMake-based windows build.) Best advice is to build from source only if you need the very latest release, and it is not yet offered by your package management system.
Not necessarily for the latest release. We can live with versions that are a tad old, but we needflexibility to build from source because not all programs will work otherwise.
Otherwise use the package management system and you will get all dependencies and everything properly installed in the right place, without having to read documentation or install multiple packages by hand. I note that the Fedora distro helpfully includes versions of netCDF built for parallel I/O as well. With all these helpful distributions, the average netCDF user should not have to build from source. Thanks, Ed
Thanks, Gus Correa
On Thu, Nov 6, 2014 at 8:55 PM, Gus Correa <gus@xxxxxxxxxxxxxxxxx <mailto:gus@xxxxxxxxxxxxxxxxx>> wrote: Thank you, Ward. On 11/06/2014 06:06 PM, Ward Fisher wrote: Thanks all for the input re: bundling the different interfaces; it’s clear this is more convenient. I would argue that the benefit is not strictly to us developers at the expense of the poor users (as somebody put it :) ); the split makes it much easier to provide support for individual interfaces, as well as faster bug fixes as previously mentioned. There are significant technical hurdles to recombining the interfaces into a single project, as it was for versions 4.1.3 and prior. However, the various interfaces have a hierarchical dependence, all of them - except for Java - depending on the C interface. This gives them some sort of unity, at least when one has to *use* them. Nobody can get the F90 interface to work without the Fortran (77) one, and the latter doesn't work without the C interface either. Hence, from the user/programmer standpoint, I think breaking this unit into smaller pieces is artificial and not helpful. Additional difficulties happen also when one has to build more tools in the NetCDF universe (NCO, NCL, CDO, etc) which depend on NetCDF (and sometimes on specific library versions). Those difficulties may also be reduced by having a clear and unified public distribution. However, I guess there may be a way to reconcile the two, keeping the developement in separate projects, and yet provide a bundled distribution for users/programmers as it was before. There may be avenues for making distribution more transparent and easier to keep track of from the end user point-of-view, however. I may start a new thread once I’ve explored a couple of ideas. That would really be great for NetCDF users/programmers, and hopefully won't be a hurdle for the developers. As others list subscribers certainly did, I built NetCDF C 4.3.0, along with Fortran 4.2 and C++4.4.2, which are available from Unidata as separate tarballs. I keep them under a single environment module in our clusters (actually one per compiler), which is very convenient from the user point of view, and seems to be all we need to compile and run correctly a large number of programs (from NCAR, GFDL, MPII, MIT, some inhouse codes, etc). But what took some time for me to setup, may not be so hard for the developers. So, maybe Unidata could bundle the three main interfaces into a single public/external/vanilla distribution, perhaps under a single release name (NetCDF.all.X.Y.Z perhaps?), which would build from a single configure/make/make_install action, as it was done until 4.3.1, and likewise for subsequent public releases. The existence of a single bundled release also reassures the end user that he is dealing with a consistent software package. (S)he doesn't need to go crazy to find out what set of specific interface releases are consistent with each other, or to spend too much time building/testing each one separately. Anyway, I'm glad that somehow my hijacking of this thread was not just an annoyance to the list, but had also a productive side. Thank you Gus Correa Regarding the question below about binary distributions for OSX and Windows. We provide Windows binaries because, frankly, building with Visual Studio can be a bit of a mess, and providing the libraries packaged with dependencies seemed like the easiest way to head off a lot of problems. These can be downloaded here: * http://www.unidata.ucar.edu/__software/netcdf/docs/winbin.__html <http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html> I wasn’t under the impression that there was much need for OSX binary distributions, since OSX is essentially BSD and works with autotools and/or CMake. I know that the popular package managers |homebrew| and |macports| have netcdf packages (which we do not maintain), and had always thought these must be sufficient, as nobody has said otherwise. I’d be really interested to know if these were insufficient! Thanks all, -Ward On Thu, Nov 6, 2014 at 3:34 PM, Chris Barker <chris.barker@xxxxxxxx <mailto:chris.barker@xxxxxxxx> <mailto:chris.barker@xxxxxxxx <mailto:chris.barker@xxxxxxxx>>__> wrote: On Thu, Nov 6, 2014 at 12:57 PM, Gus Correa <gus@xxxxxxxxxxxxxxxxx <mailto:gus@xxxxxxxxxxxxxxxxx> <mailto:gus@xxxxxxxxxxxxxxxxx <mailto:gus@xxxxxxxxxxxxxxxxx>>__> wrote: PS - Yes, I do understand the initial thread was about NetCDF3 vs NetCDF4, which is a separate discussion in itself. Bringing it back --- One of the main reasons users don't use netcdf4 is that it's substantially harder to build (and more heavy weight). As (at least for the c libs) the netcdf4 libs fully support netcdf3, there really isn't any reason for all client code to use the netcdf4 libraries, regardless of whether they are actually using netcdf4 files. So anything Unidata can do to make it easier for end users to use netcdf4 will really help: Easier to build Good binary distributions (for Window and probably OS-X anyway) ??? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 <tel:%28206%29%20526-6959> <tel:%28206%29%20526-6959> voice 7600 Sand Point Way NE (206) 526-6329 <tel:%28206%29%20526-6329> <tel:%28206%29%20526-6329> fax Seattle, WA 98115 (206) 526-6317 <tel:%28206%29%20526-6317> <tel:%28206%29%20526-6317> main reception Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx> <mailto:Chris.Barker@xxxxxxxx <mailto:Chris.Barker@xxxxxxxx>> _________________________________________________ netcdfgroup mailing list netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx> For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/__mailing_lists/ <http://www.unidata.ucar.edu/mailing_lists/>
netcdfgroup archives: