NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
The netCDF Operators NCO version 4.3.8 are ready. http://nco.sf.net (Homepage) http://dust.ess.uci.edu/nco (Homepage "mirror") ncatted now supports groups. That's the main new feature. This completes NCO's migration from flat- to group-oriented files. All core operators are now "group-aware" and support netCDF4 groups. Most relevant options may support full or partial group paths. Remaining options that do not will be migrated as time permits. The exception to this is ncap2. Porting ncap2 to support groups will be undertaken as resources permit. The main ncap2 developer is a volunteer for whom we are trying to secure funding. Work on NCO 4.3.9 is underway and includes improved netCDF4 support for various options (all -X, ncwa -w and -m), other netCDF4 mop-up, and improved support for HDF4 files. And the operator nces, which we will debut/describe at AGU (Friday 12/13 11:35 AM Moscone West 2010). Enjoy, Charlie "New stuff" in 4.3.8 summary (full details always in ChangeLog): NEW FEATURES: A. ncatted accepts full group paths to variables and groups. Previously ncatted worked only on the root group of netCDF4 files. Now you can edit all attributes in group files, in all permutations, i.e., for variables, for groups and the root group. ncatted -a units,/g1/lon,o,c,"degrees_west" in_grp.nc ncatted -a new_grp_att,group,c,sng,"new group attribute" in_grp.nc http://nco.sf.net/nco.html#xmp_ncatted B. Add -7 switch to write netCDF4-classic files. This is simply a synonym for --fl_fmt=netcdf4_classic. The reason (and mnemonic) for -7 is that NETCDF4_CLASSIC files include great features of both netCDF3 (compatibility) and netCDF4 (compression, chunking) and, well, 3+4=7 :) Anyway, -7 is much easier to remember/type than the alternative. http://nco.sf.net/nco.html#fl_fmt BUG FIXES: A. Reduced namespace conflicts with other shared libraries B. Added missing --op_typ/--operation switches to ncra KNOWN ISSUES NOT YET FIXED: This section of ANNOUNCE reports and reminds users of the existence and severity of known, not yet fixed, problems. These problems occur with NCO 4.3.7 built/tested with netCDF 4.3.1-rc4 snapshot 20131106 on top of HDF5 hdf5-1.8.9 with these methods: cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or- cd ~/nco/bld;make dir;make all;make ncap2 # Old Makefile mechanism A. NOT YET FIXED netCDF4 library fails when renaming dimension and variable using that dimension, in either order. Works fine with netCDF3. Hence coordinate renaming does not work with netCDF4 files. Problem with netCDF4 library implementation. Demonstration: ncks -O -4 -v lat_T42 ~/nco/data/in.nc ~/foo.nc ncrename -O -D 2 -d lat_T42,lat -v lat_T42,lat ~/foo.nc ~/foo2.nc # Breaks with "NetCDF: HDF error" ncks -m ~/foo.nc 20130724: Verified problem still exists Bug report filed: netCDF #YQN-334036: problem renaming dimension and coordinate in netCDF4 file Workaround: Use ncrename twice; first rename the variable, then rename the dimension. B. NOT YET FIXED (would require DAP protocol change?) Unable to retrieve contents of variables including period '.' in name Periods are legal characters in netCDF variable names. Metadata are returned successfully, data are not. DAP non-transparency: Works locally, fails through DAP server. Demonstration: ncks -O -C -D 3 -v var_nm.dot -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc # Fails to find variable 20130724: Verified problem still exists. Stopped testing because inclusion of var_nm.dot broke all test scripts. NB: Hard to fix since DAP interprets '.' as structure delimiter in HTTP query string. Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47 C. NOT YET FIXED (would require DAP protocol change) Correctly read scalar characters over DAP. DAP non-transparency: Works locally, fails through DAP server. Problem, IMHO, is with DAP definition/protocol Demonstration: ncks -O -D 1 -H -C -m --md5_dgs -v md5_a -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc 20120801: Verified problem still exists Bug report not filed Cause: DAP translates scalar characters into 64-element (this dimension is user-configurable, but still...), NUL-terminated strings so MD5 agreement fails D. NOT YET FIXED (NCO problem) Correctly read arrays of NC_STRING with embedded delimiters in ncatted arguments Demonstration: ncatted -D 5 -O -a new_string_att,att_var,c,sng,"list","of","str,ings" ~/nco/data/in_4.nc ~/foo.nc 20130724: Verified problem still exists TODO nco1102 Cause: NCO parsing of ncatted arguments is not yet sophisticated enough to handle arrays of NC_STRINGS with embedded delimiters. E. NOT YET FIXED Report correct chunking and compression information for HDF4 files Demonstration: ncdump -h -s MOP01-20121231-L1V3.34.10.hdf ncks -m MOP01-20121231-L1V3.34.10.hdf 20131007: Verified problem still exists Cause: some libnetCDF library functions fail on HDF4 file inquiries. Bug report filed: netCDF #HZY-708311 ncdump/netCDF4 segfaults probing HDF4 file Tracking tickets NCF-272, NCF-273 "Sticky" reminders: A. Pre-built, up-to-date Debian Sid & Ubuntu packages: http://nco.sf.net#debian B. Pre-built Fedora and CentOS RPMs: http://nco.sf.net#rpm C. Pre-built Windows (native) and Cygwin binaries: http://nco.sf.net#windows D. Pre-built AIX binaries: http://nco.sf.net#aix E. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)? SWAMP efficiently schedules/executes NCO scripts on remote servers: http://swamp.googlecode.com SWAMP can work command-line operator analysis scripts besides NCO. If you must transfer lots of data from a server to your client before you analyze it, then SWAMP will likely speed things up. F. NCO support for netCDF4 features is tracked at http://nco.sf.net/nco.html#nco4 NCO supports netCDF4 atomic data types, compression, chunking, and groups. G. Reminder that NCO works on most HDF4 and HDF5 datasets, e.g., NASA AURA HIRDLS HDF-EOS5 NASA ICESat GLAS HDF5 NASA MERRA HDF4 NASA MODIS HDF4 NASA SBUV HDF5... -- Charlie Zender, Earth System Sci. & Computer Sci. University of California, Irvine 949-891-2429 )'(
netcdfgroup archives: