NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
The netCDF Operators NCO version 4.3.0 are ready. http://nco.sf.net (Homepage) http://dust.ess.uci.edu/nco (Homepage "mirror") This release brings some NCO floating point math capabilities to netCDF4 files. Specifically, ncbo (i.e., ncdiff) now works with on netCDF4 (and thus HDF5) files that make use of groups. This milestone in capability explains the jump from 4.2.6 to 4.3.0. We made this release relatively swiftly after 4.2.6 both because of the new floating point ability, and to correct a bug introduced in 4.2.6 in which ncks stopped copying attributes by default. This is extremely annoying and we want people to have an upgrade alternative rather than a downgrade alternative. Work on NCO 4.3.1 is underway and includes improved netCDF4 support for ncbo (including -G and --unn), other floating point operators, and on better support for Windows builds (enabling DAP, UDUnits). Enjoy, Charlie "New stuff" in 4.3.0 summary (full details always in ChangeLog): NEW FEATURES: A. ncbo works on netCDF4 files. It is the first NCO floating point operator to work with netCDF4 files that include groups, e.g., ncbo -O -p ~/nco/data in_grp.nc in_grp.nc ~/foo.nc B. By popular demand ncatted no longer NUL-terminates character attributes. Some would call this a bug fix, though I prefer NUL-terminated character attributes (i.e., strings). The behavior re-institutes behavior from NCO 4.0.2-4.2.2 (201007-201210) and is more consistent with ncgen. C. ncap2 now supports nearbyint(), rint(), round(), and trunc() functions. D. NCO now rounds floats to integers internally using the lrint() rather than the lround() family of functions. Halfway cases that were rounded away from zero now round to nearest even integer. e.g., 2.5 used to round to 3, now it rounds to 2, while 3.5 still rounds to 4. BUG FIXES: A. Fixed ncks bug introduced in version 4.2.6 where, by default, variable attributes were not copied to output file. One can work around this problem in 4.2.6 by specifying -m. Otherwise an upgrade is recommended. http://nco.sf.net#bug_ncks_mtd KNOWN BUGS NOT YET FIXED: This section of ANNOUNCE reports and reminds users of the existence and severity of known, not yet fixed, problems. These problems occur with NCO 4.3.0 built/tested with netCDF 4.3.0-rc3 released 20130311 on top of HDF5 hdf5-1.8.9 with these methods: cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or- cd ~/nco/bld;make dir;make all;make ncap2 # Old Makefile mechanism A. NOT YET FIXED Correctly read netCDF4 input over DAP, write netCDF4 output, then read resulting file. Replacing netCDF4 with netCDF3 in either location of preceding sentence leads to success. DAP non-transparency: Works locally, fails through DAP server. Unclear whether resulting file is "legal" because of dimension ID ordering assumptions. Demonstration: ncks -4 -O -v three_dmn_rec_var http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in_4.nc ~/foo.nc ncks ~/foo.nc # breaks with "NetCDF: Invalid dimension ID or name" 20120731: Unable to verify since in_4.nc no longer accessible on Unidata DAP server Bug report filed: netCDF #QUN-641037: dimension ID ordering assumptions B. NOT YET FIXED netCDF4 library fails when renaming dimension and variable using that dimension, in either order. Works fine with netCDF3. Problem with netCDF4 library implementation. Demonstration: ncks -O -4 -v lat_T42 ~/nco/data/in.nc ~/foo.nc ncrename -O -D 2 -d lat_T42,lat -v lat_T42,lat ~/foo.nc ~/foo2.nc # Breaks with "NetCDF: HDF error" ncks -m ~/foo.nc 20130319: Verified problem still exists Bug report filed: netCDF #YQN-334036: problem renaming dimension and coordinate in netCDF4 file C. NOT YET FIXED (requires change to DAP protocol) Unable to retrieve contents of variables including period '.' in name Periods are legal characters in netCDF variable names. Metadata is returned successfully, data is not. DAP non-transparency: Works locally, fails through DAP server. Demonstration: ncks -O -C -D 3 -v var_nm.dot -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc # Fails to find variable 20120731: Verified problem still exists. Stopped testing because inclusion of var_nm.dot broke all test scripts. NB: Hard to fix since DAP interprets '.' as structure delimiter in HTTP query string. Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47 D. NOT YET FIXED (requires change to DAP protocol) Correctly read scalar characters over DAP. DAP non-transparency: Works locally, fails through DAP server. Problem, IMHO, is with DAP definition/protocol Demonstration: ncks -O -D 1 -H -C -m --md5 -v md5_a -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc 20120801: Verified problem still exists Bug report not filed Cause: DAP translates scalar characters into 64-element, NUL-terminated strings so MD5 agreement fails E. NOT YET FIXED netCDF4 library can create dimensions with duplicate IDs if dimension with same name defined in a group and its ancestor group. Problem with HDF5 or with netCDF4 library implementation? Demonstration: ncks -O -v two_dmn_rec_var ~/nco/data/in_grp.nc ~/foo.nc ncks -m ~/foo.nc 20130321: Verified problem still exists (This appears to be fixed in the upcoming netCDF 4.3.0 release...will check again then) Bug report filed 20120312: netCDF #SHH-257980: Re: [netcdfgroup] Dimensions IDs F. NOT YET FIXED ncdump is unable to dump the NCO test file in_grp.nc This is not an NCO problem per se, but may indicate a deeper netCDF bug that affects NCO. Unclear whether problem with ncdump, HDF5, or with netCDF4 library implementation. Demonstration: ncdump ~/nco/data/in_grp.nc # breaks with "NetCDF: Invalid argument Location: file dumplib.c; line 970" 20130319: Verified Bug report not yet filed "Sticky" reminders: A. Pre-built, up-to-date Debian Sid & Ubuntu packages: http://nco.sf.net#debian B. Pre-built Fedora and CentOS RPMs: http://nco.sf.net#rpm C. Pre-built Windows (native) and Cygwin binaries: http://nco.sf.net#windows D. Pre-built AIX binaries: http://nco.sf.net#aix E. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)? SWAMP efficiently schedules/executes NCO scripts on remote servers: http://swamp.googlecode.com SWAMP can work command-line operator analysis scripts besides NCO. If you must transfer lots of data from a server to your client before you analyze it, then SWAMP will likely speed things up. F. NCO support for netCDF4 features is tracked at http://nco.sf.net/nco.html#nco4 NCO supports netCDF4 atomic data types, compression, chunking, and groups. G. Reminder that ncks and ncecat work on many common HDF5 datasets, e.g., NASA AURA HIRDLS HDF-EOS5 NASA ICESat GLAS HDF5 NASA SBUV HDF5... -- Charlie Zender, Earth System Sci. & Computer Sci. University of California, Irvine 949-891-2429 )'(
netcdfgroup archives: