NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

[decoders] Large GRIB files

NOTE: The decoders mailing list is no longer active. The list archives are made available for historical reasons.

Hi,

I'm trying to read a large GRIB file using the java decoder, but I'm
getting the following exception:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
 at java.lang.StringBuffer.append(StringBuffer.java:320)
at ucar.grib.grib1.Grib1Input.seekHeader(Grib1Input.java:247)
 at ucar.grib.grib1.Grib1Input.scan(Grib1Input.java:129)

Since the grib file has 3.8Gb, the exception is thrown when the class tries
to load the whole file to the memory.

Is there a way to seek the grib headers in-disk?

I'm a newbie in the grib file format, but looking at the source code for
GribInput, the seekHeader method loads a bunch of bytes into the memory and
then look for the header pattern. Is this load really necessary?

Att,

-- 
Ederson Ferreira de Jesus
Matemático Aplicado - UNICAMP
"Time is an illusion. Lunchtime doubly so."
Adams, Douglas
  • 2011 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the decoders archives: