NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
I’ve switched to JRE 1.7. I think it is a good thing, but now I’m seeing the memory usage fluctuate, so sometimes it drops back down. Using JRE 1.6 memory usage pretty much only went up, I don’t recall it ever dropping down. We were starting to think our huge CPU spikes might be related to Java garbage collection. Hopefully JRE 1.7 is a workable solution for us. On Dec 12, 2013, at 10:29 AM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote: > Hi jay: > > Debugging tomcat server performance is complicated. > > The first question is, how many simultaneous requests are there? Each will > consume one thread, until request completes . So very slow connections can > shutdown a server; the default number of threads depends on version i think > maybe 200 for tomcat 7. This can be changed in tomcat/conf/server.xml. > > 1. Start up the tomcat manager application and see what it has to say about > the connections. > > 2. Better, run jvisualvm on the same machine as tomcat (can also be done > remotely) and connect to the running tomcat server. > > there are more complicated things to do; there are commercial versions of > tomcat that add visibility and performance metrics. You might also google > "tomcat performance monitoring tools". > > you want to upgrade to Java 7; also make sure you are on latest version of > tomcat 7. > > send me the threddsServlet.log file for one of the hours where the server > bogged down and ill see if theres anything obvious. > > John > > > > On 12/12/2013 10:47 AM, Gerry Creager - NOAA Affiliate wrote: >> Jay, >> >> What OS, release and Tomcat version are you running? I've seen a similar >> issue on another piece of software (Ramadda). Since I've seen this >> behavior with the standalone server and the tomcat-based server, I'm >> beginning to suspect my Java installation, but have not had sufficient >> time to investigate yet. >> >> There may be an OS correlation here, so I'm interested. I'm running RHEL >> 6 and the various updated flavors of OpenJDK and Tomcat6. >> >> gerry >> >> >> >> >> On Wed, Dec 11, 2013 at 4:33 PM, Jay Alder <alderj@xxxxxxxxxxxxxxxxxxxx >> <mailto:alderj@xxxxxxxxxxxxxxxxxxxx>> wrote: >> >> Hi, we’ve recently released a web application that uses TDS for >> mapping, which is getting a lot of traffic. At one point the server >> stopped responding altogether, which is a major problem. A quick >> restart of tomcat got it going again, so I’m starting to dig into >> the logs. We normally get the GET / request complete behavior, but >> occasionally we’ll have: >> >> GET …url… >> GET …url… >> GET …url… >> GET …url… >> GET …url… >> GET …url… >> GET …url… >> GET …url… >> >> meanwhile having a 100% CPU spike (with 12 CPUs) for a minute or more >> >> request compete >> request compete >> request compete >> request cancelled by client >> request cancelled by client >> request compete >> request compete >> >> While watching the logs the few times I’ve seen this occur it seems >> to pull out of it ok. However the time the server failed, requests >> were never returned. From the logs, requests came in for roughly 40 >> minutes without being completed. Unfortunately do to the high >> visibility we started to get emails from users and the press about >> the application no longer working. >> >> Has anyone experienced this before and/or can you give guidance on >> how to diagnose or prevent this? >> >> Here are some config settings: >> CentOS 5.7 >> Java 1.6 >> TDS 4.3.17 >> only WMS is enabled >> Java -Xmx set to 8Gb (currently taking 5.3, the dataset is 600 Gb of >> 30-arcsecond grids for the continental US, 3.4 Gb per file) >> For better or worse we are configured to use 2 instances of TDS to >> keep the catalogs and configuration isolated. I’m not sure if this >> matters, but I didn’t want to omit it. Since it is a live server I >> can’t easily change to the preferred proxy configuration. >> >> I am trying not to panic yet. However, if the server goes >> unresponsive again, staying calm may no longer be an option. >> >> Jay Alder >> US Geological Survey >> Oregon State University >> 104 COAS Admin Building >> Office Burt Hall 166 >> http://ceoas.oregonstate.edu/profile/alder/ >> >> >> _______________________________________________ >> thredds mailing list >> thredds@xxxxxxxxxxxxxxxx <mailto:thredds@xxxxxxxxxxxxxxxx> >> For list information or to unsubscribe, visit: >> http://www.unidata.ucar.edu/mailing_lists/ >> >> >> >> >> -- >> Gerry Creager >> NSSL/CIMMS >> 405.325.6371 >> ++++++++++++++++++++++ >> “Big whorls have little whorls, >> That feed on their velocity; >> And little whorls have lesser whorls, >> And so on to viscosity.” >> Lewis Fry Richardson (1881-1953) >> >> >> _______________________________________________ >> thredds mailing list >> thredds@xxxxxxxxxxxxxxxx >> For list information or to unsubscribe, visit: >> http://www.unidata.ucar.edu/mailing_lists/ >> > > _______________________________________________ > thredds mailing list > thredds@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ Jay Alder US Geological Survey Oregon State University 104 COAS Admin Building Office Burt Hall 166 http://ceoas.oregonstate.edu/profile/alder/
thredds archives: