NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hello all, The Unidata netCDF team is happy to announce the availability of netCDF 4.8.0, now with Zarr support (nczarr)! This release extends the netcdf-c library to provide access to cloud storage (e.g. Amazon S3 by providing a mapping from a subset of the full netCDF Enhanced (aka netCDF–4) data model to a variant of the Zarr data model. The NetCDF variant is called NCZarr. It is possible for other implementations, such as Python Zarr, to read this variant. This implementation can optionally read and write using the pure version 2 Zarr storage format. It also can optionally support the XArray _ARRAY_DIMENSIONS extension to Zarr. In addition to this new functionality, the release notes appended to the end of this announcement detail some of the other changes, improvements and bug fixes included in this release. Please see the following links for more information: - Github release page: https://github.com/Unidata/netcdf-c/releases/tag/v4.8.0 - v4.8.0 Documentation: https://www.unidata.ucar.edu/software/netcdf/docs - NCZarr Documentation: https://www.unidata.ucar.edu/software/netcdf/documentation/NUG/nczarr_head.html Kudos to Dennis Heimbigner for the technical implementation of nczarr, and our thanks to the members of the Zarr working group and our community who have collaborated with us so patiently on this! Our next priority is to get caught up on the backlog of PRs and issues blocking the next releases of netCDF-Fortran and netCDF-CXX4. Thank you all very much, have a great day! -Ward Ward Fisher wfisher@xxxxxxxx UCAR/Unidata - Software Engineer NetCDF Team Lead Full Release Notes4.8.0 - March 30, 2021 - [Enhancement] Bump the NC_DISPATCH_VERSION from 2 to 3, and as a side effect, unify the definition of NC_DISPATCH_VERSION so it only needs to be defined in CMakeLists.txt and configure.ac. See Github #1945 <https://github.com/Unidata/netcdf-c/pull/1945> for more information. - [Enhancement] Provide better cross platform path name management. This converts paths for various platforms (e.g. Windows, MSYS, etc.) so that they are in the proper format for the executing platform. See Github #1958 <https://github.com/Unidata/netcdf-c/pull/1958> for more information. - [Bug Fixes] The nccopy program was treating -d0 as turning deflation on rather than interpreting it as “turn off deflation”. See Github #1944 <https://github.com/Unidata/netcdf-c/pull/1944> for more information. - [Enhancement] Add support for storing NCZarr data in zip files. See Github #1942 <https://github.com/Unidata/netcdf-c/pull/1942> for more information. - [Bug Fixes] Make fillmismatch the default for DAP2 and DAP4; too many servers ignore this requirement. - [Bug Fixes] Fix some memory leaks in NCZarr, fix a bug with long strides in NCZarr. See Github #1913 <https://github.com/Unidata/netcdf-c/pull/1913> for more information. - [Enhancement] Add some optimizations to NCZarr, dosome cleanup of code cruft, add some NCZarr test cases, add a performance test to NCZarr. See Github #1908 <https://github.com/Unidata/netcdf-c/pull/1908> for more information. - [Bug Fix] Implement a better chunk cache system for NCZarr. The cache now uses extendible hashing plus a linked list for provide a combination of expandibility, fast access, and LRU behavior. See Github #1887 <https://github.com/Unidata/netcdf-c/pull/1887> for more information. - [Enhancement] Provide .rc fields for S3 authentication: HTTP.S3.ACCESSID and HTTP.S3.SECRETKEY. - [Enhancement] Give the client control over what parts of a DAP2 URL are URL encoded (i.e. %xx). This is to support the different decoding rules that servers apply to incoming URLS. See Github #1884 <https://github.com/Unidata/netcdf-c/pull/1844> for more information. - [Bug Fix] Fix incorrect time offsets from ncdump -t, in some cases when the time units attribute contains both a *non-zero* time-of-day, and a time zone suffix containing the letter “T”, such as “UTC”. See Github #1866 <https://github.com/Unidata/netcdf-c/pull/1866> for more information. - [Bug Fix] Cleanup the NCZarr S3 build options. See Github #1869 <https://github.com/Unidata/netcdf-c/pull/1869> for more information. - [Bug Fix] Support aligned access for selected ARM processors. See Github #1871 <https://github.com/Unidata/netcdf-c/pull/1871> for more information. - [Documentation] Migrated the documents in the NUG/ directory to the dedicated NUG repository found at https://github.com/Unidata/netcdf - [Bug Fix] Revert the internal filter code to simplify it. From the user’s point of view, the only visible change should be that (1) the functions that convert text to filter specs have had their signature reverted and renamed and have been moved to netcdf_aux.h, and (2) Some filter API functions now return NC_ENOFILTER when inquiry is made about some filter. Internally, the dispatch table has been modified to get rid of the complex structures. - [Bug Fix] If the HDF5 byte-range Virtual File Driver is available )HDf5 1.10.6 or later) then use it because it has better performance than the one currently built into the netcdf library. - [Bug Fix] Fixed byte-range support with cURL > 7.69. See [ https://github.com/Unidata/netcdf-c/pull/1798]. - [Enhancement] Added new test for using compression with parallel I/O: nc_test4/tst_h_par_compress.c. See [ https://github.com/Unidata/netcdf-c/pull/1784]. - [Bug Fix] Don’t return error for extra calls to nc_redef() for netCDF/HDF5 files, unless classic model is in use. See [ https://github.com/Unidata/netcdf-c/issues/1779]. - [Enhancement] Added new parallel I/O benchmark program to mimic NOAA UFS data writes, built when –enable-benchmarks is in configure. See [ https://github.com/Unidata/netcdf-c/pull/1777]. - [Bug Fix] Now allow szip to be used on variables with unlimited dimension [https://github.com/Unidata/netcdf-c/issues/1774]. - [Enhancement] Add support for cloud storage using a variant of the Zarr storage format. Warning: this feature is highly experimental and is subject to rapid evolution [ https://www.unidata.ucar.edu/blogs/developer/en/entry/overview-of-zarr-support-in ]. - [Bug Fix] Fix nccopy to properly set default chunking parameters when not otherwise specified. This can significantly improve performance in selected cases. Note that if seeing slow performance with nccopy, then, as a work-around, specifically set the chunking parameters. [ https://github.com/Unidata/netcdf-c/issues/1763]. - [Bug Fix] Fix some protocol bugs/differences between the netcdf-c library and the OPeNDAP Hyrax server. Also cleanup checksum handling [ https://github.com/Unidata/netcdf-c/issues/1712].* [Bug Fix] IMPORTANT: Ncgen was not properly handling large data sections. The problem manifests as incorrect ordering of data in the created file. Aside from examining the file with ncdump, the error can be detected by running ncgen with the -lc flag (to produce a C file). Examine the file to see if any variable is written in pieces as opposed to a single call to nc_put_vara. If multiple calls to nc_put_vara are used to write a variable, then it is probable that the data order is incorrect. Such multiple writes can occur for large variables and especially when one of the dimensions is unlimited. - [Bug Fix] Add necessary _ *declspec declarations to allow compilation of netcdf library without causing errors or (*declspec related) warnings [https://github.com/Unidata/netcdf-c/issues/1725]. - [Enhancement] When a filter is applied twice with different parameters, then the second set is used for writing the dataset [https://github.com/Unidata/netcdf-c/issues/1713]. - [Bug Fix] Now larger cache settings are used for sequential HDF5 file creates/opens on parallel I/O capable builds; see Github #1716 <https://github.com/Unidata/netcdf-c/issues/1716> for more information. - [Bug Fix] Add functions to libdispatch/dnotnc4.c to support dispatch table operations that should work for any dispatch table, even if they do not do anything; functions such as nc_inq_var_filter [https://github.com/Unidata/netcdf-c/issues/1693]. - [Bug Fix] Fixed a scalar annotation error when scalar == 0; see Github #1707 <https://github.com/Unidata/netcdf-c/issues/1707> for more information. - [Bug Fix] Use proper CURLOPT values for VERIFYHOST and VERIFYPEER; the semantics for VERIFYHOST in particular changed. Documented in NUG/DAP2.md. See [https://github.com/Unidata/netcdf-c/issues/1684]. - [Bug Fix][cmake] Correct an issue with parallel filter test logic in CMake-based builds. - [Bug Fix] Now allow nc_inq_var_deflate()/nc_inq_var_szip() to be called for all formats, not just HDF5. Non-HDF5 files return NC_NOERR and report no compression in use. This reverts behavior that was changed in the 4.7.4 release. See [https://github.com/Unidata/netcdf-c/issues/1691]. - [Bug Fix] Compiling on a big-endian machine exposes some missing forward delcarations in dfilter.c.
netcdfgroup archives: