NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [netcdfgroup] Proposed changes to netcdf-c library: Detecting netCDF versus HDF5

  • To: "dmh@xxxxxxxx" <dmh@xxxxxxxx>
  • Subject: Re: [netcdfgroup] Proposed changes to netcdf-c library: Detecting netCDF versus HDF5
  • From: Stephan Hoyer <shoyer@xxxxxxxxx>
  • Date: Tue, 29 Aug 2017 17:24:41 -0700
Bringing this back up again, now that I see that _NCProperties is included
in the documentation under "Attribute Conventions" [1].

Is there are a good way to handle cases where there is no valid netcdflib
version to include here, because the file wasn't written using netcdf-c?

What I've been doing is including h5netcdfversion=X.Y.Z in the
_NCProperties attribute instead of netcdflib=X.Y.Z. I'm thinking now that
I'd also like to include versions of other libraries used to generate the
file (e.g., h5py). Does that seem reasonable? Or should I rather drop
unexpected names altogether?

At the very least, I don't think I can include any sensible netcdflib
version.

[1]
http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html

On Sat, Apr 30, 2016 at 12:45 PM, dmh@xxxxxxxx <dmh@xxxxxxxx> wrote:

> This will be documented in the existing documentation.
> At some point, it needs to be added to the netcdf NASA spec
> (https://earthdata.nasa.gov/standards/netcdf-4hdf5-file-format).
> =Dennis Heimbigner
>   Unidata
>
> On 4/29/2016 10:39 PM, Stephan Hoyer wrote:
>
>> What is the specification you're planning to use for the value of the
>> _NCProperties attribute? We'd like to able to generate compliant netCDF4
>> files with h5netcdf, which uses HDF5 directly to write netCDF4.
>>
>> I guess my main request is to not assume that only netCDF-C will write
>> _NCProperties attributes, and to ensure that netCDF-C and the specification
>> are flexible enough to allow for different implementations, maybe even
>> hypothetical implementations not using the HDF5 libraries at all.
>>
>> Thanks,
>> Stpehan
>>
>> On Fri, Apr 29, 2016 at 8:32 PM, dmh@xxxxxxxx <mailto:dmh@xxxxxxxx> <
>> dmh@xxxxxxxx <mailto:dmh@xxxxxxxx>> wrote:
>>
>>     I am about to commit a pull request for the netcdf-c library having to
>>     do with identifying the provenance and format of netcdf-4 files,
>>     and specifically targeted at detecting netcdf-4 files from HDF5 files.
>>
>>     This provenance consists of the following information. First, there is
>>     a hidden, persistent, attribute names _NCProperties.  It specifies the
>>     library versions of the netcdf library and the hdf5 library used to
>>     create the file. This attribute never changes during the lifetime
>>     of the
>>     file (unless modified deliberately thru the hdf5 API).
>>
>>     Second, there are two special, non-persistent, attributes that are
>>     computed from information already in the file.
>>     1. _SuperblockVersion
>>     2. _IsNetcdf4
>>     Non-persistence means these attributes do not actually appear in the
>>     file.  and are computed from other info already in the file.
>>
>>     The _SuperblockVersion attribute is a single integer giving the
>>     version
>>     number (currently 0-3) of the superblock in the hdf5/netcdf-4 file.
>>
>>     The _IsNetcdf4 attribute is a single integer 0/1 indicating if the
>>     file
>>     has various tags indicating it was produced thru the netcdf-4 API.
>>     This
>>     is computed by using the HDF5 API to walk the file to look for
>>     attributes specific to netcdf-4.  False negatives are possible for a
>>     small subset of netcdf-4 files, especially those not containing
>>     dimensions. False positives are (I think) only possible by deliberate
>>     modifications to an existing HDF5 file thru the HDF5 API. For
>>     files with
>>     the _NCProperties attribute, this attribute is redundant. For files
>>     created prior to the introduction of the _NCProperties attribute, this
>>     may be a useful indicator of the provenance of the file.
>>
>>     These three attributes are hidden in the sense that they can only be
>>     accessed thru the netcdf-C api calls via the name. They have no
>>     attribute number and will not be counted in the number of global
>>     attributes in the root group.
>>
>>     The simplest way to view these attributes is to use the -s flag to the
>>     ncdump command.
>>
>>     Comments are welcome.
>>
>>     =Dennis Heimbigner
>>     Unidata
>>
>>     _______________________________________________
>>     netcdfgroup mailing list
>>     netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
>>     For list information or to unsubscribe,  visit:
>>     http://www.unidata.ucar.edu/mailing_lists/
>>
>>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
  • 2017 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: