RE: ncdigest V1 #660 (Writing NetCDF Files: Best Practices)

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: RE: ncdigest V1 #660 (Writing NetCDF Files: Best Practices)
• From: Harvey.Davies@xxxxxxxx
• Date: Wed, 4 Sep 2002 14:58:39 +1000

> ncdigest           Thursday, August 29 2002           Volume 
> 01 : Number 660
> 
> Today's Topics:
> Writing NetCDF Files: Best Practices
> 
> Some guidelines for netCDF data design are included in a new web
> document "Writing NetCDF Files: Best Practices":
> 
>   http://www.unidata.ucar.edu/packages/netcdf/BestPractices.html
> 
> This is an evolving document, so if you have additional suggestions
> for best practices, please send them to us.

I think this document is very good. Most of my following comments
are suggestions for further detail.

First, I did not understand the whole 2nd dot point in the section
headed 'Unlimited Dimension'. This commences with:
- Don't use the simplest form of writing ... 
I think this needs rewriting with more explanation.

The use of NaN for missing values is recommended.  I agree with this,
but I thought the netCDF User Guide warned against it for portability
reasons.  (Perhaps this warning has now been removed, since nearly all
modern machines have a NaN, even those that do not use IEEE f.p.)

Perhaps there should also be something about infinity.  I suggest 
recommending against using it, since some packages will not handle it.

Some compilers reject expressions with division by 0.0.  The way round 
this is to define a function divide(x,y) which simply divides x by y.
Then you can define
double nan = divide(0.0 / 0.0);
double infinity = divide(1.0 / 0.0);

The mention of vector quantities (e.g. wind) is an important topic which I
feel needs further discussion.  There should be some standard netCDF
conventions for vector values. Most wind files I have encountered have
used separate variables for the two components (north and east), but
it would seem more logical to use a single variable with a dimension
(size 2 in this case) for this component of vector quantities. How about
having a coordinate variable of type character with value "NE" indicating
the directions?  Maybe there should be a special attribute for such
a coordinate variable.

SCALING:
There is a need to note the different convention from that used in HDF
(despite the same attribute names which misled me at 1st).
Another potential problem (which has also caused me problems) is having
a missing value just one different from valid_min/max.  Rounding error
can easily give an error of 1, so I now use a valid_min of say -32700
for data scaled to short (with misssing data set to default
fill value of -32767).  I like to allow room for several different
missing values, just in case I ever need to distinguish between different
kinds such as
- not applicable
- instrument breakdown
- not yet received

Harvey Davies, CSIRO Atmospheric Research,
Private Bag No. 1, Aspendale 3195
E-mail: harvey.davies@xxxxxxxx
Phone: +61 3 9239 4556
  Fax: +61 3 9239 4444