Re: 19990928: NetCDF conversion problems on Crays

• To: Steve Emmerson <steve@xxxxxxxxxxxxxxxx>
• Subject: Re: 19990928: NetCDF conversion problems on Crays
• From: jps@xxxxxxxx (John Sheldon)
• Date: Wed, 29 Sep 1999 16:15:06 -0400

On Sep 28,  8:47, Steve Emmerson wrote:
> Subject: Re: (Fwd) Re: 19990928: NetCDF conversion problems on Crays
> John,
> 
> >Date: Mon, 27 Sep 1999 10:30:06 -0400
> >From: jps@xxxxxxxx (John Sheldon)
> >Organization: GFDL
> >To: steve@xxxxxxxxxxxxxxxx
> >Subject: (Fwd) Re: 19990723: NetCDF conversion problems on Crays
> >Keywords: 199907231930.NAA00955
> 
> In the above message, you wrote:
> 
> > Sorry to pester you over this, but I've got several users here who
> > are quite anxious about not being able to control their internal-to-
> > external conversion adequately....Basically, they want to be able to
> > test the KIND (as in the FORTRAN "KIND") of a variable and call the
> > appropriate output function. Then, they can turn on "-r8"-type compiler
> > switches, yet retain the option to use REAL*4 variables, too.
> > 
> > Anything you can suggest?  Anything new in the pipeline?
> 
> There nothing new in the pipeline -- but there's nothing in the current
> release that prevents what you want.  You can currently write Fortran-90
> code that conditionally uses either doubleprecision or real netCDF I/O
> routines -- it's just that the netCDF library has to have been compiled
> with the "-dp" option and the netCDF library doesn't understand 128-bit
> floating-point values.
> 
> I guess I don't understand what the problem is.

I think you're right :-)>...
But this may be my fault :-( 

Here's the gist of it...Suppose a FORTRAN application has a mix of 
floating point declarations:

       REAL   ARR
       REAL*4 ARR_4
       REAL*8 ARR_8

The default compiler option on most platforms is to assume "REAL"'s
are 4-byte quantities (8-bytes on Crays, but we'll deal with that
separately).

When the netCDF library is built with default compiler options, it 
knows how to store 4-byte REAL and REAL*4 values via NF_PUT_VAR_REAL,
and how to store 8-byte REAL*8 values via NF_PUT_VAR_DOUBLE.

Now, suppose someone wants to compile their application so that REAL
quantities default to 8-bytes (eg, "-r8" option). Obviously, the
NF_PUT_VAR_REAL call will work incorrectly by grabbing only 4-bytes
from memory.  NF_PUT_VAR_DOUBLE should continue to work fine.

"Aha", you say, so just create a version of the netCDF library that is
also built with the same (eg, "-r8") compiler option(s), right?  This
should work fine storing REAL's via NF_PUT_VAR_REAL and REAL*8's via 
NF_PUT_VAR_DOUBLE.  BUT, there will now be no way to store REAL*4's!

Now, having presented the long-winded example, let's go back to Crays.
Under FORTRAN-77, there was no such thing as a 4-byte quantity.
You could declare something REAL*4, but you got an 8-byte quantity
anyhow.  However, under FORTRAN-90, one can now actually get 4-byte
quantities by declaring a variable as REAL*4.  Unfortunately, you can't
store such a quantity to a netCDF file, because NF_PUT_VAR_REAL assumes
8-byte internal quantities!

It would seem that what is needed is a variety of NF_PUT_VAR_REAL*
functions that will handle each declaration type (analogous to the
NF_PUT_VAR_INT* differentiation):
  
    NF_PUT_VAR_REAL   to handle "REAL ARR" declarations
    NF_PUT_VAR_REAL4  to handle "REAL*4 ARR" declarations
    NF_PUT_VAR_REAL8  to handle "REAL*8 ARR" declarations

Now, the user can use the KIND function to determine the internal 
representation, then call either NF_PUT_VAR_REAL4 or NF_PUT_VAR_REAL8.
Alternatively, a different version of the netCDF library can be linked
in which was built to expect REAL's to be 8-byte quantities. The first
option is more work for the user, but avoids the (likely) error of
compiling one's own code with "-r8" and then linking to the default
libnetcdf.a.  (Option 2 also assumes that the user is diligent about
using the NF_PUT_VAR_REAL* call that matches his declaration.)

What we are doing at the moment (at least for the application which
has an urgency level sufficient to pop this issue to the top
of my stack :-) is replacing all NF_PUT_VAR_REAL calls with
NF_PUT_VAR_DOUBLE calls, then mandating the use of the "-r8" compiler
option and disallowing storage of 4-byte quantities. Obviously, not
a very robust or desirable long-tern solution.

Is this any clearer?  Am *I* missing something?

Thanks-
John
----------------------------------------------+---------------------------
 John P. Sheldon                              | Technical Services Manager
 NOAA/Geophysical Fluid Dynamics Laboratory   | Email: jps@xxxxxxxx
 Princeton University/Forrestal Campus/Rte. 1 | Tel:   (609) 987-5053
 P.O. Box 308                                 | Fax:   (609) 987-5063
 Princeton, NJ, USA  08542                    | WWW:   http://www.gfdl.gov
----------------------------------------------+---------------------------

• Follow-Ups:
  • Re: 19990928: NetCDF conversion problems on Crays
    • From: Brian Eaton