Re: [thredds] "Too Many Open Files" Error. Dataset too big?

• To: thredds@xxxxxxxxxxxxxxxx
• Subject: Re: [thredds] "Too Many Open Files" Error. Dataset too big?
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Thu, 24 Oct 2013 13:23:58 -0600

Hi Kevin:

Aggregations should not cause "Too many open files".
There must be a file leak bug.

heres what would help us debug it:

0. ulimit -a to see how many open files you have (should be 1 or 2k)

1. restarting tomcat will close open files.

2. to monitor what files are open, use:

/usr/proc/bin/pfiles [Tomcat Process ID]

(you may need to use sudo?)

3. then do whatever (minimal) thing to trigger the problem, then run pfiles and 
see what files are open. Send me the pfiles output.


a few more notes are here:

http://www.unidata.ucar.edu/software/thredds/v4.4/tds/internal/Troubleshooting.html


John

On 10/24/2013 11:50 AM, Kevin Manross wrote:
> 
> Hi there,
> 
> I am trying to aggregate netcdf files for some rather large datasets 
> with tomcat6.0.36/TDS4.19 - 20131008.1358 and am running into a "Too 
> many open files" error.
> 
> My catalog scan is:
> 
>    <dataset name="ASR 30 km 2d surface analysis"
>             ID="ds631.0_anl_2d-AGG"
>             urlPath="ds631.0/9/best"
>             harvest="true">
>       <serviceName>all</serviceName>
>       <netcdf 
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>         <aggregation dimName="time" type="Union">
>           <scan location="/glade/p/rda/data/ds631.0/asr30km.anl.2d/" 
> suffix=".nc" subdirs="false"/>
>         </aggregation>
>       </netcdf>
>     </dataset>
> 
> which contains about 4020 files and totals 470G.
> 
> The message that the server sends when trying to access the OPeNDAP 
> service is:
> 
> 
> Error {
>      code = 404;
>      message = 
> "/glade/p/rda/data/ds631.0/asr30km.anl.2d/asr30km.anl.2d.20010619.nc (Too 
> many open files)";
> };
> 
> 
> 
> When trying the CdmRemote service catalina.out repeatedly reads:
> 
> Oct 24, 2013 11:20:39 AM org.apache.jk.common.ChannelSocket 
> acceptConnections
> WARNING: Exception executing accept
> java.net.SocketException: Too many open files
>      at java.net.PlainSocketImpl.socketAccept(Native Method)
>      at 
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
>      at java.net.ServerSocket.implAccept(ServerSocket.java:478)
>      at java.net.ServerSocket.accept(ServerSocket.java:446)
>      at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:311)
>      at 
> org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:668)
>      at 
> org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:879)
>      at 
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
>      at java.lang.Thread.run(Thread.java:679)
> 
> 
> The cache settings in my threddsConfig.xml are the default settings.
> 
> Have I overlooked something?
> 
> -kevin.
> 
> 
> -- 
> Kevin Manross
> NCAR/CISL/Data Support Section
> Phone: (303)-497-1218
> Email:manross@xxxxxxxx <mailto:manross@xxxxxxxx>
> Web:http://rda.ucar.edu
> 
> 
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/
> 

• Follow-Ups:
  • Re: [thredds] "Too Many Open Files" Error. Dataset too big?
    • From: Kevin Manross

• References:
  • [thredds] "Too Many Open Files" Error. Dataset too big?
    • From: Kevin Manross