Re: [thredds] tomcat-crash with thredds4: Too many open files

• To: Richard Signell <rsignell@xxxxxxxxx>
• Subject: Re: [thredds] tomcat-crash with thredds4: Too many open files
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Fri, 25 Sep 2009 12:54:25 -0600

Its useful to clear the TDS cache (need remote monitoring):

 https://motherlode.ucar.edu:9443/thredds/admin/debug?Caches/clearCache
 https://motherlode.ucar.edu:9443/thredds/admin/debug?Caches/forceNCCache
 https://motherlode.ucar.edu:9443/thredds/admin/debug?Caches/forceRAFCache

Then look at

to see what files are still open.

if you can do that, send me the output of pfiles.

Richard Signell wrote:

Heiko,

Did you ever figure this out?  We just had our
thredds4.0.26/tomcat6.0.18 crash with "Too man open files" also.

-Rich

On Fri, Sep 18, 2009 at 4:53 AM, Heiko Klein <Heiko.Klein@xxxxxx> wrote:

Hi,

we just had a crash of the thredds4.0.25/tomcat server and I was able to
analyse a bit the system before the operators restarted it.

Error message was: 'Too many open files' and checking the open files with
lsof or /proc/PID/fd showed 1021 open netcdf-files for the tomcat-pid. We
haven't made any changes to the thredds4 configuration concerning the
file-caches, so they should be between 200-400. We're using WCS, too, so
this might add some more files, but still not up to 1021.

We also use some aggregations of netcdf files, so one 'thredds-aggregation'
might consist of some hundred netcdf-files. Maybe this is counted as one
file in the fileCache?

I currently try to avoid further crashes by increasing the ulimit -n from
1024 to 16384, but I would rather be able to tune the file-cache properly
instead of throwing more resources at it.


Best regards,

Heiko

--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/

• References:
  • [thredds] tomcat-crash with thredds4: Too many open files
    • From: Heiko Klein
  • Re: [thredds] tomcat-crash with thredds4: Too many open files
    • From: Richard Signell