NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

[thredds] Java heap problems with large ASCII downloads

Hi,
I use thredds as opendap server for some large netcdf-files (CF-1.0, size > 1.1GB). When selecting to download the whole file with the 'Get ASCII' or 'GET BINARY', thredds crashes with a 'Java heap error'.
I'm currently using a default memory setup, i.e. -Xmx512MB for tomcat. How much memory will I need to avoid this problem. Does DODS read the whole file at once or variable after variable, or even slice after slice? Or are there some adjustments I can make to reduce memory consumption? 

Best regards,

Heiko




Details:
DODServlet ERROR (anyExceptionHandler): java.lang.OutOfMemoryError: Java heap space 
ReqState:
  serverClassName:    'thredds.server.opendap.NcDODSServlet'
  dataSet:            'XXXX.nc'
  requestSuffix:      'ascii'
  CE:                 ''
  compressOK:          true
  InitParameters:
    DebugOn: ''
    maxNetcdfFilesCached: '10'

java.lang.OutOfMemoryError: Java heap space
        at ucar.nc2.N3raf.readData(N3raf.java:77)
        at ucar.nc2.N3iosp.readData(N3iosp.java:159)
        at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1374)
        at ucar.nc2.Variable._read(Variable.java:923)
        at ucar.nc2.Variable.read(Variable.java:618)
        at thredds.server.opendap.NcSDArray.read(NcSDArray.java:102)
        at thredds.server.opendap.NcSDGrid.read(NcSDGrid.java:59)
        at opendap.servlet.AsciiWriter.writeAsc(AsciiWriter.java:95)
        at opendap.servlet.AsciiWriter.toASCII(AsciiWriter.java:56)
at opendap.servlet.AbstractServlet.doGetASC(AbstractServlet.java:1029) 
        at opendap.servlet.AbstractServlet.doGet(AbstractServlet.java:1630)
at thredds.server.opendap.NcDODSServlet.doGet(NcDODSServlet.java:269) 
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:698)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
...



Thredds-Version:
Built-By: caron
Built-On: 2008-02-20 22:56:05
Implementation-Title: THREDDS Data Server
Implementation-Version: 3.16.33
Implementation-Vendor: UCAR/Unidata

Running on tomcat 6.0.16 with jdk.1.6.0_04 on debian 4.0.

--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY
  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: