Re: [netcdf-java] Fix for GRIB File Descriptor Leak

• To: netcdf-java@xxxxxxxxxxxxxxxx
• Subject: Re: [netcdf-java] Fix for GRIB File Descriptor Leak
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Fri, 16 May 2014 16:43:17 -0600

Hi Aaron:

Yes, we are about to make a 4.3.22 release, so we can incorporate this fix.

John

On 5/16/2014 3:22 PM, Aaron Braeckel wrote:

Hello,

Rob (CCed) emailed in February (subject: "EOFException: Reading
file.grb2.ncx at 0 file length = 0") about a file descriptor leak
observed with the Java netCDF when reading invalid GRIB index files.  I
was able to reliably replicate the file leak locally by manually setting
the index file to 0 size before opening a GRIB file.  I observed the
file leak through the set of open file descriptors (through both the
Java UnixOperatingSystemMXBean class and the lsof command).

The attached source files are a patch for this issue against the
edu.ucar:grib:4.3.21 source code.  This patch is quite simple, below I
describe the individual changes.  Would it be possible to apply these
changes and make a new release?  If so, when would the release be made?
We are currently maintaining our own branch of the GRIB library because
we need this fix in operations, which is undesirable for a number of
reasons.  I can provide a main() method that demonstrates this problem
if it would be of interest.

Thanks!
Aaron and Rob

===============================

The source of the problem was in Grib1CollectionBuilder and
Grib2CollectionBuilder (the code changes to each are identical):

In /readIndex/(String filename) a new RandomAccessFile was being created
but was never closed.  This fix definitively resolved our file leak
issue, and logically the created RandomAccessFile must be closed in this
method since it is only used there.

<     return readIndex( new RandomAccessFile(filename, "r") );
---
 >     RandomAccessFile raf = new RandomAccessFile( filename, "r" );
 >     boolean success = readIndex( raf );
 >     raf.close();
 >     return success;


In /readOrCreateIndex/() there is more code that opens a
RandomAccessFile but may not close it.  This code did not resolve our
issue, but is similar to the fix above and may resolve a file leak in
other situations.  In this case I note that gc.setIndexRaf(indexRaf) is
called just before the changed code segment, and therefore the RAF may
be closed properly without this change.  I don't know the code and flow
of operations well enough to be sure.

<       readIndex(indexRaf);
---
 >       boolean success = readIndex(indexRaf);
 >       if( !success ){
 >         indexRaf.close();
 >       }





_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/

• References:
  • [netcdf-java] Fix for GRIB File Descriptor Leak
    • From: Aaron Braeckel