NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

[thredds] "Too Many Open Files" Error. Dataset too big?


Hi there,

I am trying to aggregate netcdf files for some rather large datasets with tomcat6.0.36/TDS4.19 - 20131008.1358 and am running into a "Too many open files" error.

My catalog scan is:

  <dataset name="ASR 30 km 2d surface analysis"
           ID="ds631.0_anl_2d-AGG"
           urlPath="ds631.0/9/best"
           harvest="true">
     <serviceName>all</serviceName>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
       <aggregation dimName="time" type="Union">
<scan location="/glade/p/rda/data/ds631.0/asr30km.anl.2d/" suffix=".nc" subdirs="false"/>
       </aggregation>
     </netcdf>
   </dataset>

which contains about 4020 files and totals 470G.

The message that the server sends when trying to access the OPeNDAP service is:


Error {
    code = 404;
    message = "/glade/p/rda/data/ds631.0/asr30km.anl.2d/asr30km.anl.2d.20010619.nc 
(Too many open files)";
};



When trying the CdmRemote service catalina.out repeatedly reads:

Oct 24, 2013 11:20:39 AM org.apache.jk.common.ChannelSocket acceptConnections
WARNING: Exception executing accept
java.net.SocketException: Too many open files
    at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
    at java.net.ServerSocket.implAccept(ServerSocket.java:478)
    at java.net.ServerSocket.accept(ServerSocket.java:446)
    at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:311)
at org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:668) at org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:879) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:679)


The cache settings in my threddsConfig.xml are the default settings.

Have I overlooked something?

-kevin.


--
Kevin Manross
NCAR/CISL/Data Support Section
Phone: (303)-497-1218
Email:manross@xxxxxxxx <mailto:manross@xxxxxxxx>
Web:http://rda.ucar.edu
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: