Re: netCDF Java API question

• To: "John C Cartwright" <John.C.Cartwright@xxxxxxxx>
• Subject: Re: netCDF Java API question
• From: "John Caron" <caron@xxxxxxxxxxxxxxxx>
• Date: Wed, 17 Apr 2002 17:17:41 -0600

----- Original Message -----
From: "John C Cartwright" <John.C.Cartwright@xxxxxxxx>
To: <caron@xxxxxxxxxxxxxxxx>
Sent: Wednesday, April 17, 2002 9:19 AM
Subject: netCDF Java API question


> Hello John,
>
> I have a question about the Java API that I was hoping that you could
> help me with.
>
> I'm trying to create a simple "stack" of 2D grids using the code listed
> below.  The problem is that I'm running out of memory and I assume that
> there must be something I'm doing wrong since it seems unlikely that the
> size of the output file would be constrained by the available RAM.  The
> code seems to work OK if I reduce the number of rows and columns.
>
> Can you show me what I'm doing wrong or direct me to a better
> forum/source of information on this?
>
> Thanks!
>
> -- john

Hi John:

Your problem is in allocating the Variable Array all at once, which is an
in-memory array. You really want to allocate only a 2D slice, and write the
slices out sequentially. below is one way to do that.

BTW, in testing this, I discovered an embarrrassing performance bug that
makes things run (blush) 40 time slower in some cases, for example yours! I
just fixed that, and a new release 2.06 is available at

http://www.unidata.ucar.edu/packages/netcdf-java/

you will definitely want to use it!

Also, posting questions to netcdf-java@xxxxxxxxxxxxxxxx lets everyone on
that list see your questions, and possibly learn, if you dont mind exposing
you mistakes. With your permission, I would like to repost this answer
there.


     /* old way - uses too much memory
     int val=0;
     ArrayInt gviA = new ArrayInt.D3(daysSince1900.length,
latDim.getLength(), lonDim.getLength());
     Index ima = gviA.getIndex();
     // write
     for (i=0; i<daysSince1900.length; i++) {
       for (j=0; j<latDim.getLength(); j++) {
         for (k=0; k<lonDim.getLength(); k++) {
            // gviA.setInt(ima.set(i,j,k), gviData[i][j][k]);
            gviA.setInt(ima.set(i,j,k), val);  // write dummy data for
testing
         }
       }
       val++;
     }
     // write gviData out to disk
     try {
       ncfile.write("gvi", gviA);
     } catch (IOException e) {
       System.err.println("ERROR writing file");
     }

     */

     // new way, write one 2D slice at a time
     int[] origin = new int[3];
     int val=0;
     ArrayInt gviA = new ArrayInt.D3(1, latDim.getLength(),
lonDim.getLength());
     Index ima = gviA.getIndex();

     for (i=0; i<daysSince1900.length; i++) {
      origin[0] = i;  // which slice am i writing ?

       for (j=0; j<latDim.getLength(); j++) {
         for (k=0; k<lonDim.getLength(); k++) {
            gviA.setInt(ima.set(0,j,k), val);  // write dummy data for
testing
         }
       }

        // write gviData out to disk
       try {
         ncfile.write("gvi", origin, gviA);
       } catch (IOException e) {
         System.err.println("ERROR writing file");
       }

       val++;
     }