Adding netCDF support to HDF

• To: chouck@xxxxxxxxxxxxx
• Subject: Adding netCDF support to HDF
• From: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Date: Mon, 30 Mar 1992 10:08:44 -0700

Hi,

Chris Houck writes:

>       We here at NCSA (the home of HDF) are actively investigating
> adding a netCDF interface to HDF.   Our implementation would be a
> call-compatatible superset of the existing netCDF interface.  
> [We are asking for funding for this initiative from 
> NSF as well as NASA's EOSDIS (Earth Observing System Data 
> Information System) project.  EOSDIS is interested in 10-bit data, 
> among other things, so extensions to the netCDF interface will be 
> required].

We're pleased to hear about this, and will cooperate as much as practical
with your efforts, which should benefit both the netCDF and HDF communities.

With regard to your proposal for extending the netCDF interface to handle
packed data, we have designed (but not yet implemented) a way to provide
packed data in netCDF files that does not require extensions to the
interface, and does not require that a program accessing the data know
whether it is packed or not.  By "packed", I mean that an array of 10-bit
values, for example, could be stored without wasting space.  Currently, such
an array must be stored as 16-bit NC_SHORTs.  Glenn Davis suggested this
design to help make netCDF more useful for archival storage.  The proposal
uses three new reserved attributes that would be handled in the library
layer: "_Nbits", "_Scale", and "_Offset".  Here's a brief sketch of the
idea, without discussing the complications (which we think we have worked
out satisfactorily) with fill-values and changing attributes.


>From the view of user level code, a packed variable

usual way, with a netCDF type such as NC_LONG of NC_FLOAT, but the reserved
acked-variable attributes would need to be created:
   "_Nbits" specifying the number of bits per value,
   "_Scale" specifying the scaling (default 1.0), and
   "_Offset" specifying the offset (default 0.0).

To write packed data, ncvarput is called in the usual way.  For example, if
the type of the variable was NC_FLOAT, the user passes an array of floats.
For each value, the library subtracts _Offset and divides by _Scale and
converts the result to type long.  The least significant _Nbits of this
value are extracted and placed at the proper place in the netCDF file.

To read packed data, ncvarget is called in the usual way.  For example, if
the type was NC_DOUBLE, the user passes a pointer to storage appropriate for
an array of doubles.  For each value, the library gets the _Nbits from
storage and places it in a container of type long.  The value is then
muliplied by _Scale and has _Offset added to it.  The result has the type
converted to the type of the netCDF value.

While implementing this proposal is on a list of netCDF enhancements, we
cannot yet commit to it.  Right now implementing the initial netCDF
operators and providing a C++ interface are higher priority.

--Russ

• References:
  • Adding netCDF support to HDF
    • From: chouck