NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi,I'm encountering "java.lang.OutOfMemoryError: Java heap space" errors in aggregating netcdf files. The messages are appearing in the tomcat5/content/thredds/logs/threddsServlet.log and the web page will return a web page showing
Error { code = -1; message = "Server Error on dataset sst-aerosol-nl-aggregation"; };
instead of the normal OPENDAP access page ( http://www.ngdc.noaa.gov/thredds/dodsC/sst-aerosol-nl-aggregation.html ). This page usually returns normally for a few requests, but once the OutOfMemoryError occurs the service is down until I restart tomcat. I did try an upgrade to the latest THREDDS warfile, 3.16.48.0, but still see this error. The TDS is running on Red Hat Enterprise Linux WS release 3 (Taroon Update 9). The java.vm.version = 1.5.0_06-b05, and tomcat version is 5.5.17.
The files we are serving up are SST Aerosol products, with a Time coordinate, with a single time slice per file. The typical filesize is 3.6MB. We have quite a bit of metadata as Global Attributes in each, but I think we can abstract that out with an NcML wrapper file. Here is the wrapper file, which is in the same directory with the data files:
<?xml version="1.0" encoding="UTF-8"?> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; > <attribute name="myAttribute" value="NN-attribute" /> <aggregation dimName="Time" type="joinExisting"><scan location="/data1/thredds/sst/Aerosol_NN_test/" suffix=".nc" />
</aggregation>
</netcdf>
And here is the portion of the catalog.xml that uses this aggregation:
<!-- Aerosol NN, joinExisting with ncml wrapper
-->
<dataset name="SST Aerosol Aggregation NOAA-18(N)" ID="SST-Aerosol-NN-Agg"
urlPath="sst-aerosol-nn-aggregation">
<metadata inherited="true">
<serviceName>multiple</serviceName>
<dataType>Grid</dataType>
</metadata>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
location="/data1/thredds/sst/Aerosol_NN_test/Aerosol_NN_test.ncml" > </netcdf> </dataset>Do you see any troubles with my configuration? Earlier posts to this list indicated that there should not be a problem with aggregating a large number of files. This particular aggregation is 245 files. I definitely get errors right away if I also set up separate but similar aggregation with 135 files in the same catalog.
Thanks for any suggestions, Ken -- = Enterprise Data Services Division =============== | CIRES, National Geophysical Data Center / NOAA | | 303-497-6221 | = Ken.Tanaka@xxxxxxxx =============================
thredds archives: