[netCdfJava #WVX-463093]: How to write to NetCDF from ASCII file without using temporary arrays in java code?

• To: serg@xxxxxxxx
• Subject: [netCdfJava #WVX-463093]: How to write to NetCDF from ASCII file without using temporary arrays in java code?
• From: "Unidata netCDF Java Support" <support-netcdf-java@xxxxxxxxxxxxxxxx>
• Date: Wed, 22 Feb 2006 21:52:02 -0700

Hello Sukhonosov Sergey,

By using an unlimited dimension, you can grow the file as you go rather than 
requiring that all the data be in memory before you write the file. In your 
case, the along the column direction would correspond to the unlimited 
dimension and the number of items in each row would correspond to the 
parameters that have that unlimited dimension.

Here's some code writing along the unlimited dimension:


    String fileName = "testFile.nc";
    NetcdfFileWriteable ncfile = NetcdfFileWriteable.createNew(fileName, true);
    Dimension timeDim
        new Dimension( "time", Dimension.UNLIMITED.getLength(),
                       true, true, false );
    ncfile.addDimension( null, timeDim );
    Dimension [] dims = { timeDim};
    ncfile.addVariable( "time", DataType.INT, dims );
    ncfile.addVariable( "temp", DataType.FLOAT, dims );
    ncfile.addVariable( "pres", DataType.FLOAT, dims );

    try
    {
      ncfile.create();
    }
    catch ( IOException e )
    {
      System.err.println( "Failed to create file <" +
                          fileName + ">: " + e.getMessage());
      System.exit();
    }

    ArrayInt.D1 timeArray = new ArrayInt.D1(1);
    ArrayFloat.D1 tempArray = new ArrayFloat.D1(1);
    ArrayFloat.D1 presArray = new ArrayFloat.D1(1);
    for ( int i = 0; i < 20; i++ )
    {
      int [] origin = { i };
      // Making up some data here.
      timeArray.set( 0 , i );
      tempArray.set( 0, 2*i );
      presArray.set( 0, 3*i );
      try
      {
        ncfile.write( "time", origin, timeArray );
        ncfile.write( "temp", origin, tempArray );
        ncfile.write( "pres", origin, presArray );
      }
      catch ( IOException e )
      {
        System.err.println( "Failed to write file <" +
                            fileName + "> at index=" + i +
                            ": " + e.getMessage());
        System.exit();

      }
      catch ( InvalidRangeException e )
      {
        System.err.println( "Bad range writing file <" +
                            fileName + "> at index=" + i +
                            ": " + e.getMessage());
        System.exit();
      }
    }
    try
    {
      ncfile.flush();
      ncfile.close();
    }
    catch ( IOException e )
    {
        System.err.println( "Failed to flush/close file <" +
                            fileName + ">: " + e.getMessage());
        System.exit();
    }


Hope that helps. Let me know if you have other questions.

Ethan

> Hello support team!
> 
> I have large plain ascii file with data. How it's better to make
> arrays in NetCDF file if one array is one column in ascii. I've tried
> to read ascii file by string and parse it, but I have to use temporary
> ArrayList or Hashtable to store data before it would be written to
> NetCDF and it takes much RAM. Is there any nice approach to write into
> NetCDF from ascii file without using temporary java constructions such
> as ArrayList()?
> 
> 
> --
> Best regards, Sukhonosov Sergey            mailto:serg@xxxxxxxx
> 
> 


Ticket Details
==================
Ticket ID: WVX-463093
Department: Support netCDF Java
Priority: Normal
Status: Open