netCDF Utilities

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: netCDF Utilities
• From: hld@xxxxxxxxxxxxxxxxx (DAVIES Harvey)
• Date: Tue, 18 Feb 1992 11:27:28 +1100

In <9202101050.AA16358@xxxxxxxxxxxxxxxxx>, I commented on draft document
"NetCDF Operators and Utilities" by Russ Rew & Steve Emmerson

In <9202101902.AA06315@xxxxxxxxxxxxxxxxxxxxxx>, 
Steve Emmerson <steve@xxxxxxxxxxxxxxxx> replies
>Your ideas are not too 'way out' and we are, indeed, interested.

Thanks Steve for your encouragement. I would like to expand my comments a bit.

>As far as working with heterogeneous data-types is concerned, we're
>working on a prototype C++ implementation that will support polymorphic
>arithmetic (using a technique call "double polymorphism").

I convert from any type to type double using my procedure 'from_nc_type' & do
reverse using 'to_nc_type'. These are in nc.h, which together with other source
code mentioned below, is accessible by anonymous ftp to atmos.dar.csiro.au
followed by "cd netcdf/hld"

Now some more comments on above draft. I do not know the thinking behind the
current proposals & I may be missing some vital concept, but my list of
suggested initial utilities would be along following lines:

- ncgen - extended to allow insertion & modification of existing file
- ncdump - extended to allow specification of variables, etc
- ncrename - much as proposed
- ncdelete - delete attributes (& variables if possible)
- nccopy - copy data from one file:variable to another

I assume that ncgen could be used to copy data from standard input to
file:variable & that ncdump could be used to copy data from file:variable to
standard output. Alternatively additional utilities (say ncput & ncget) could
provide these functions. My program nc_put_var provides the essence of ncput,
including type conversion, unit conversion & packing (using  add_offset &
scale_factor).

In my previous posting I proposed a syntax for specifying an 'array' defined
by netcdf filename, variable name & a vector of subscripts for each dimension.
I want to argue that given such a flexible way of specifying source &
destination, only a small number (such as the 5 or 7 above) would be needed.
For example, one copy utility (say nccopy) would do the essence of the tasks
of the eleven proposed programs ncextr, nccut, ncthin, ncappend, nccat,
ncmerge, ncpaste, ncjoin, ncorder, ncunpack, ncpack

If the basic utilities have the power & orthogonality I am advocating, then
programs such as these eleven could be written as shell scripts, looping over
variables, etc. (In fact the proposed program ncmkgrd could be written as
a script calling the current ncgen, much along the lines of my script
nc500new.sh)

I do not claim to fully understand the current proposals or that my proposals
have been fully thought through. I just want to stimulate some lateral
thinking before things are set in concrete.

Harvey Davies                   CSIRO Division of Atmospheric Research,
Internet: hld@xxxxxxxxxxxx      Private Bag No. 1, Mordialloc,
Phone: +61 3 586 7574           Victoria 3195,  Australia