NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

[netcdfgroup] ncgen should remove special attributes like _Shuffle

Hi,
newer versions of netcdf (tested with 4.2.1 and 4.1.1) allow the generation of netcdf4 files with shuffling, chunking and compression by using the attributes:
`_Format', `_Fletcher32, `_ChunkSizes', `_Endianness',_DeflateLevel', `_Shuffle', and `_Storage'.
Unfortunately, these attributes are not only used for creating the file, but are also written as normal attributes to the file. 

A simple example:

netcdf test {
dimensions:
    time = UNLIMITED ; // (2 currently)
variables:
    short time(time) ;
        time:_Storage = "chunked" ;
        time:_ChunkSizes = 1 ;
        time:_Shuffle = "true" ;
        time:_DeflateLevel = 3 ;
// global attributes:
        :_Format = "netCDF-4 classic model" ;
data:
 time = 12, 24 ;
}


looks after a 'ncgen -b', and 'ncdump -hs test.nc' like:

netcdf test {
dimensions:
        time = UNLIMITED ; // (2 currently)
variables:
        short time(time) ;
                time:_ChunkSizes = 1 ;
                time:_DeflateLevel = 3 ;
                time:_Shuffle = "true" ;
                time:_Storage = "chunked" ;
                time:_ChunkSizes = 1 ;
                time:_DeflateLevel = 3 ;
                time:_Shuffle = "true" ;
                time:_Endianness = "little" ;

// global attributes:
                :_Format = "netCDF-4 classic model" ;
}
When using some further processing, e.g. 'nccopy -k 3 -d 9 test.nc test9.nc', test9.nc will have ambiguos _DeflateLevel settings: 

$ ncdump -hs test9.nc
netcdf test9 {
dimensions:
        time = UNLIMITED ; // (2 currently)
variables:
        short time(time) ;
                time:_ChunkSizes = 1 ;
                time:_DeflateLevel = 3 ;
                time:_Shuffle = "true" ;
                time:_Storage = "chunked" ;
                time:_ChunkSizes = 1 ;
                time:_DeflateLevel = 9 ;
                time:_Endianness = "little" ;

// global attributes:
                :_Format = "netCDF-4" ;
}
I guess, all those _Attributes should never be written to the file by ncgen. (Tested with netcdf-4.2.1.1 on linux (ubuntu 12.04 64bit)). 

Regards,

Heiko
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: