NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
The netCDF operators NCO version 4.2.4 are ready. http://nco.sf.net (Homepage) http://dust.ess.uci.edu/nco (Homepage "mirror") This release continues, and largely concludes, the introduction of powerful new sub-setting features aimed to give users powerful and fine-grained control over subsetting hierarchical netCDF4 files. Last release saw the introduction of Group Path Editing (GPE) which controls the output location of extracted variables and groups. This release introduces three new features that control extraction: 1. Group regular expressions, 2. Anchoring and Recursion, and 3. Intersection and Unions. These latest three features are only in ncks and ncecat for the time-being. Again, let us know how you like or dislike the way we have implemented the netCDF4 group functionality in ncks and ncecat. There are many choices for handling (selecting, printing, moving, comparing) groups and we made what I would call tentative decisions. Your feedback will help us finalize details like the printed output format for group information and the syntax of switches. This improved netCDF4 support will eventually come to all of NCO. Work on NCO 4.2.5 is underway: better netCDF4 support for ncrename and ncbo operators. Enjoy, Charlie "New stuff" in 4.2.4 summary (full details always in ChangeLog): NEW FEATURES: A. ncks & ncecat support regular expressions in both variable and group lists. Formerly they worked only in variable lists. ncks -g 'g.' in_grp.nc # Groups g0-g9 ncks -g 'g.' -v 'v.' in_grp.nc # All variables 'v.' in groups 'g.' http://nco.sf.net/nco.html#rx B. ncks & ncecat support Anchoring and Recursion. Previously arguments to -g were extracted recursively by default. Meaning that descendents of specified groups were also extracted. And prepended slashes had no special significance. Now initial and trailing slashes control Anchoring and Recursion: ncks -g g1 # Extract, recursively, all groups with a g1 component ncks -g g1/ # Extract, non-recursively, all groups terminating in g1 ncks -g /g1 # Extract, recursively, root group g1 ncks -g /g1/ # Extract, non-recursively root group g1 http://nco.sf.net/nco.html#rcr C. ncks & ncecat allow control over intersection- and union-subsetting. Do users want intersections or unions of matches to the -v and -g subsetting instructions? NCO is no longer wishy-washy: Intersection mode is the default, and --unn selects union-mode: ncks -g g1 -v v1 # Yes: /g1/v1, /g2/g1/v1. No: /v1, /g2/v1 ncks -g g1 -v v1 --unn # All vars in g1 (or descendents), or named v1 http://nco.sf.net/nco.html#unn D. ncrename supports restricting actions to global/group attributes Formerly ncrename arguments like -a old_nm,new_nm had no way of restricting actions to global/group attributes and changing them required implicit assumptions about whether variables could have attributes of the same name. Now attribute renaming options can be restricted to global/group attributes either by leaving the var_nm portion empty (and retaining the @), or by using var_nm= global to indicated that att_nm refers to a global attribute. ncrename -a global@old_att_nm,new_att_nm in.nc ncrename -a @old_att_nm,new_att_nm in.nc ncrename -a global@.old_att_nm,new_att_nm in.nc http://nco.sf.net/nco.html#xmp_ncrename BUG FIXES: A. Group attributes are now copied into all groups which are ancestors of groups with extracted variables. Formerly, group attributes were only copied into groups which themselves had extracted variables. Now otherwise empty (i.e., no variables) intermediate groups will have their attributes copied. A. Fix bug introduced in 4.2.1 that causes ncea to break when handling files that do not contain a record dimension. Thanks to Martin Otte for reporting the bug and providing a working fix. http://nco.sf.net#bug_ncea_rec_var A. Fix bug introduced in 4.2.1 where ncra did not write results of final normalization if trailing file(s) was/were superfluous. In the most common case, all zeros appeared in the output file. http://nco.sf.net#bug_ncra_flg_bfr_nrm A. Fix 4.2.2 and 4.2.3 bug that kept NCO from building from source when netCDF version 3 library was not available. A. Handle variables specified in CF "bounds" and "coordinates" attributes as coordinates in all ways. Specifically, dimension- reducing operations (e.g., min/max/ttl) no longer operate on CF conventions (since performing these operations on grids usually makes no sense). Instead, the same dimension-reducing operation applied to the coordinate (averaging) is now applied to any CF "bounds" and "coordinates" variables. KNOWN BUGS NOT YET FIXED: This section of the ANNOUNCE file reports and reminds users of the existence and severity of known, not yet fixed, problems. These problems occur with NCO 4.2.4 built/tested with netCDF 4.2.1 on top of HDF5 hdf5-1.8.9 with these methods: cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or- cd ~/nco/bld;make dir;make all;make ncap2 # Old Makefile mechanism A. NOT YET FIXED Correctly read netCDF4 input over DAP, write netCDF4 output, then read resulting file. Replacing netCDF4 with netCDF3 in either location of preceding sentence leads to success. DAP non-transparency: Works locally, fails through DAP server. Unclear whether resulting file is "legal" because of dimension ID ordering assumptions Demonstration: ncks -4 -O -v three_dmn_rec_var http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in_4.nc ~/foo.nc ncks ~/foo.nc # breaks with "NetCDF: Invalid dimension ID or name" 20120731: Unable to verify since in_4.nc no longer accessible Bug report filed: netCDF #QUN-641037: dimension ID ordering assumptions B. NOT YET FIXED netCDF4 library fails when renaming dimension and variable using that dimension, in either order. Works fine with netCDF3. Problem with netCDF4 library implementation. Demonstration: ncks -O -4 -v lat_T42 ~/nco/data/in.nc ~/foo.nc ncrename -O -D 2 -d lat_T42,lat -v lat_T42,lat ~/foo.nc ~/foo2.nc # Breaks with "NetCDF: HDF error" ncks -m ~/foo.nc 20121025: Verified problem still exists Bug report filed: netCDF #YQN-334036: problem renaming dimension and coordinate in netCDF4 file C. NOT YET FIXED Unable to retrieve contents of variables including period '.' in name Metadata is returned successfully, data is not. DAP non-transparency: Works locally, fails through DAP server. Demonstration: ncks -O -C -D 3 -v var_nm.dot -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc # Fails to find variable 20120731: Verified problem still exists. Stopped testing because inclusion of var_nm.dot broke all test scripts. NB: Problem hard to fix since DAP interprets '.' as structure delimiter in query string of HTTP requests. Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47 D. NOT YET FIXED Correctly read scalar characters over. DAP non-transparency: Works locally, fails through DAP server. Problem, IMHO, is with DAP definition/protocol Demonstration: ncks -O -D 1 -H -C -m --md5 -v md5_a -p http://motherlode.ucar.edu:8080/thredds/dodsC/testdods in.nc 20120801: Verified problem still exists Bug report not filed Cause: DAP translates scalar characters into 64-element, NUL-terminated strings so MD5 agreement fails "Sticky" reminders: A. Pre-built, up-to-date Debian Sid & Ubuntu packages: http://nco.sf.net#debian B. Pre-built Fedora and CentOS RPMs: http://nco.sf.net#rpm C. Pre-built Windows (native) and Cygwin binaries: http://nco.sf.net#windows D. Pre-built AIX binaries: http://nco.sf.net#aix -- Charlie Zender, Earth System Sci. & Computer Sci. University of California, Irvine 949-891-2429 )'(
netcdfgroup archives: