[thredds] aggregation caching: cache cleared when Tomcat restarted?

• To: thredds@xxxxxxxxxxxxxxxx
• Subject: [thredds] aggregation caching: cache cleared when Tomcat restarted?
• From: "John Maurer, IV" <jmaurer@xxxxxxxxxx>
• Date: Thu, 16 Dec 2010 09:58:53 -1000

Hi John,
Some of my aggregations are updated with a new file daily but others are more 
permanent. Because of the daily updates, I need to keep my scour set to every 
24 hours. However, the dataset that is more permanent (our tide model, which 
has forecasts out through the end of 2011) is the one causing me the recurring 
slowness. Even though the files in that dataset have not been updated since 
late November nor have any new files been created, it is still taking several 
minutes to access the OPeNDAP URL on what seems to be a daily basis, though I'm 
still trying to tease out what the pattern is. The file for that dataset gets 
re-written in the content/thredds/cache/agg folder when this happens. It'll be 
fine for a day, but then by the next day it's usually slow again on first 
access (~5 minutes to access the OPeNDAP page). It would appear that the scour 
is happening regardless of my 30-day maxAge setting? Don't know what else I can 
send you to help troubleshoot; lemme know... In case it helps, here are the 
datasets that are causing me this problem:
- http://oos.soest.hawaii.edu/thredds/idd/tide_mod.html?dataset=tide_elev_mhi
- http://oos.soest.hawaii.edu/thredds/idd/tide_mod.html?dataset=tide_velo_mhi
- http://oos.soest.hawaii.edu/thredds/idd/tide_mod.html?dataset=tide_elev_bi
- http://oos.soest.hawaii.edu/thredds/idd/tide_mod.html?dataset=tide_velo_bi

Thanks,
John Maurer 

> Hi John:
> 
> 1) Yes, this says every 24 hours, remove files with lastModified 
> date > 
> 30 days. If your aggregations are more permanent, change to a 
> longer 
> time, or set scour to -1 to not scour at all.
> 
> 2) Look under content/thredds/cache/agg:
> 
> each joinExisting should have an XML file in there. the content 
> should 
> be sort of obvious, send it to me if not. anyway, it will get 
> updated 
> when the file collection changes. otherwise, it should survive a 
> restart 
> and should make things fast if most of the files havent changed.
> 
> let me know if it seems something is not working as described.

• Follow-Ups:
  • Re: [thredds] aggregation caching: cache cleared when Tomcat restarted?
    • From: John Caron