NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
The netCDF operators NCO version 4.2.3 are ready. http://nco.sf.net (Homepage) http://dust.ess.uci.edu/nco (Homepage "mirror") This release occurs sooner than expected because it contains both an important bufgix and a new feature: We fixed a bug introduced in the last version, NCO 4.2.2, that could cause ncecat, under limited circumstances, to skip a single file. And we added a feature called Group Path Editing (GPE) that permits users to move data en masse amongst arbitrary netCDF4 groups. Again, let us know how you like or dislike the way we have implemented the netCDF4 group functionality in ncks and ncecat. Your feedback will help us finalize details like the printed output format for group information and the syntax of switches. This improved netCDF4 support will eventually come to all of NCO. Work on NCO 4.2.4 is underway: better netCDF4 support for ncrename and ncbo operators. Enjoy, Charlie "New stuff" in 4.2.3 summary (full details always in ChangeLog): NEW FEATURES: A. ncks and ncecat support Group Path Editing (GPE): The -G switch has been extended. Formerly it specified the root group in the output file where to store data. In other words, full output group paths were determined by appending the input group path to the -G argument, if any. This now is a special case of GPE, which handles appending, deleting, backspacing, and flattening group paths. GPE examples: ncks -G scenario1 in.nc out.nc # Put input file in group scenario1 ncks -G : in.nc out.nc # Flatten (remove groups from) input file ncks -G :2 in.nc out.nc # Remove first two group levels ncks -G g1:-2 in.nc out.nc # Replace last two groups levels by "g1" ncks -G g2:1 -g g1 in.nc out.nc # Move data from g1 to g2 http://nco.sf.net/nco.html#gpe BUG FIXES: A. ncecat: Fix 4.2.2 bug that can skip first input file in the default mode (RECORD_AGGREGATE) when -n NINTAP switch used to generate filenames. B. ncks correctly copies group attributes KNOWN BUGS NOT YET FIXED: This section of the ANNOUNCE file reports and reminds users of the existence and severity of known, not yet fixed, problems. These problems occur with NCO 4.2.3 built/tested with netCDF 4.2.1 on top of HDF5 hdf5-1.8.9 with these methods: cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or- cd ~/nco/bld;make dir;make all;make ncap2 # Old Makefile mechanism A. NOT YET FIXED Correctly read netCDF4 input over DAP, write netCDF4 output, then read resulting file. Replacing netCDF4 with netCDF3 in either location of preceding sentence leads to success. DAP non-transparency: Works locally, fails through DAP server. Unclear whether resulting file is "legal" because of dimension ID ordering assumptions Demonstration: ncks -4 -O -v three_dmn_rec_var http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in_4.nc ~/foo.nc ncks ~/foo.nc # breaks with "NetCDF: Invalid dimension ID or name" 20120731: Unable to verify since in_4.nc no longer accessible Bug report filed: netCDF #QUN-641037: dimension ID ordering assumptions B. NOT YET FIXED netCDF4 library fails when renaming dimension and variable using that dimension, in either order. Works fine with netCDF3. Problem with netCDF4 library implementation. Demonstration: ncks -O -4 -v lat_T42 ~/nco/data/in.nc ~/foo.nc ncrename -O -D 2 -d lat_T42,lat -v lat_T42,lat ~/foo.nc ~/foo2.nc # Breaks with "NetCDF: HDF error" ncks -m ~/foo.nc 20121025: Verified problem still exists Bug report filed: netCDF #YQN-334036: problem renaming dimension and coordinate in netCDF4 file C. NOT YET FIXED Unable to retrieve contents of variables including period '.' in name Metadata is returned successfully, data is not. DAP non-transparency: Works locally, fails through DAP server. Demonstration: ncks -O -C -D 3 -v var_nm.dot -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc # Fails to find variable 20120731: Verified problem still exists. Stopped testing because inclusion of var_nm.dot broke all test scripts. NB: Problem hard to fix since DAP interprets '.' as structure delimiter in query string of HTTP requests. Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47 D. NOT YET FIXED Correctly read scalar characters over. DAP non-transparency: Works locally, fails through DAP server. Problem, IMHO, is with DAP definition/protocol Demonstration: ncks -O -D 1 -H -C -m --md5 -v md5_a -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc 20120801: Verified problem still exists Bug report not filed Cause: DAP translates scalar characters into 64-element, NUL-terminated strings so MD5 agreement fails "Sticky" reminders: A. Pre-built, up-to-date Debian Sid & Ubuntu packages: http://nco.sf.net#debian B. Pre-built Fedora and CentOS RPMs: http://nco.sf.net#rpm C. Pre-built Windows (native) and Cygwin binaries: http://nco.sf.net#windows D. Pre-built AIX binaries: http://nco.sf.net#aix E. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)? SWAMP efficiently schedules/executes NCO scripts on remote servers: http://swamp.googlecode.com SWAMP can work command-line operator analysis scripts besides NCO. If you must transfer lots of data from a server to your client before you analyze it, then SWAMP will likely speed things up. F. NCO support for netCDF4 features is tracked at http://nco.sf.net/nco.html#nco4 NCO supports netCDF4 atomic data types, compression, and chunking. G. Have you seen the NCO logo candidates by Tony Freeman, Rich Signell, Rob Hetland, and Andrea Cimatoribus? http://nco.sf.net Tell us what you think... -- Charlie Zender, Earth System Sci. & Computer Sci. University of California, Irvine (949) 891-2429 :)
netcdfgroup archives: