Re: [thredds] THREDDS 3.16: Memory issues

• To: Carlos Valiente <carlos.valiente@xxxxxxxxx>
• Subject: Re: [thredds] THREDDS 3.16: Memory issues
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Tue, 04 Mar 2008 11:49:26 -0700

Hi Carlos:

We are looking at removing these kinds of limitations in the TDS 4.0, but it 
wont be for a while. Im wondering what client you use that can handle such big 
output?

Carlos Valiente wrote:

John Caron wrote:

We're starting up Tomcat with -Xmx406m, in order to give it 4 GB of memory, but that does not seem to be enough. We've tried raising that limit to 6 GB, which is the total memory (physicall + virtual) available on that box, but it did not make any difference.

-Xmx406m = 406Mbytes, i assume you mean -Xmx4096m ??

Yep, typo

Are you running 64-bit linux and JVM? Otherwise the JVM maxes out at around 2 Gb.

Yep, it's the 64-bit one.

Are THREDDS memory requirements for aggregation higher than those figures, or are we perhaps doing something not too clever with our setup?

the aggregation itself should not be the problem. But how big is the request?

For the variable that does not work (geopotential), these are the NetCFD files:

$ du -sch *
178M    MM_129_mon_1980.nc
178M    MM_129_mon_1981.nc
178M    MM_129_mon_1982.nc
178M    MM_129_mon_1983.nc
178M    MM_129_mon_1984.nc
178M    MM_129_mon_1985.nc
178M    MM_129_mon_1986.nc
178M    MM_129_mon_1987.nc
178M    MM_129_mon_1988.nc
178M    MM_129_mon_1989.nc
178M    MM_129_mon_1990.nc
178M    MM_129_mon_1991.nc
178M    MM_129_mon_1992.nc
178M    MM_129_mon_1993.nc
178M    MM_129_mon_1994.nc
178M    MM_129_mon_1995.nc
178M    MM_129_mon_1996.nc
178M    MM_129_mon_1997.nc
178M    MM_129_mon_1998.nc
178M    MM_129_mon_1999.nc
178M    MM_129_mon_2000.nc
178M    MM_129_mon_2001.nc
3.9G    total
$

The files for the one that does work are smaller:

$ -sch *
60M     MM_228_mon_1980.nc
60M     MM_228_mon_1981.nc
60M     MM_228_mon_1982.nc
60M     MM_228_mon_1983.nc
60M     MM_228_mon_1984.nc
60M     MM_228_mon_1985.nc
60M     MM_228_mon_1986.nc
60M     MM_228_mon_1987.nc
60M     MM_228_mon_1988.nc
60M     MM_228_mon_1989.nc
60M     MM_228_mon_1990.nc
60M     MM_228_mon_1991.nc
60M     MM_228_mon_1992.nc
60M     MM_228_mon_1993.nc
60M     MM_228_mon_1994.nc
60M     MM_228_mon_1995.nc
60M     MM_228_mon_1996.nc
60M     MM_228_mon_1997.nc
60M     MM_228_mon_1998.nc
60M     MM_228_mon_1999.nc
60M     MM_228_mon_2000.nc
60M     MM_228_mon_2001.nc
1.3G    total
$

 > The current opendap implementation requires the response to be
 > built in memory.

OK, I see. So I guess we need more memory in order to serve that kind of requests...

Thanks for your quick response, John!

C

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx

For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/

• Follow-Ups:
  • Re: [thredds] THREDDS 3.16: Memory issues
    • From: Carlos Valiente

• References:
  • [thredds] THREDDS 3.16: Memory issues
    • From: Carlos Valiente
  • Re: [thredds] THREDDS 3.16: Memory issues
    • From: John Caron
  • Re: [thredds] THREDDS 3.16: Memory issues
    • From: Carlos Valiente