NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Nilesh Lahoti a écrit :
...The netCDF library works great for us. However, I came across with one particular issue of netCDF and would like to discuss if there are any solution to this problem or something that can do to make its performance better. When we process emissions for our three dimensional grid of size (172 x 172 x 22) for 24 hours time period having hourly data, the file size is around 1 gigabyte(GB). There are several cells that have zero values and therefore the floating point value for pollutants in netCDF file has zero values. When we use gzip utility on unix to compress this files, the file size become almost 10 MB which saves us 99% of disk space. Now the question arise that if the netCDF is most compress scientific format, than is it possible to suppress this zero values of the floating point variable or is there any switch that can be used to handle zero values and reduce file size by any chance....
Hello,You can also have a look at this page: http://snow.cit.cornell.edu/noon/z_netcdf.html This is a modification of the lowlevel file layer of netcdf library which allow compressing and uncompressing data on the fly (in fact the whole netCDF structure).
The compressed files produced are not readable by the standard library and it may be difficult to share those file. However there is a command line utility
(nczip) which allow converting the compressed file to a 'normal' netCDF file and any classical netCDF file to a compressed one. The "magic number" is not the same so there is no risk the library being confused about the file type (CDF for standard netCDF files CDZ for compressed ones). The advantage is that there is no interface modification of the API (except one attribute, NC_COMPRESS, on opening if you want to create compressed files directly in your program) and this allow you to wait until netCDF-4. The annoyance is that all your programs must be relinked with the modified library. I have used this patch with success years ago but I am not sure that it will work with recent versions of the netCDF library (specially the 64 bits extension known as file format version 2). -- Philippe Poilbarbe CLS - Space Oceanography Division ============================================================================== To unsubscribe netcdfgroup, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
netcdfgroup archives: