Re: [thredds] Problems with large aggregations on JPL THREDDS?

• To: "Signell, Richard" <rsignell@xxxxxxxx>
• Subject: Re: [thredds] Problems with large aggregations on JPL THREDDS?
• From: Roy Mendelssohn - NOAA Federal <roy.mendelssohn@xxxxxxxx>
• Date: Sun, 17 Feb 2013 07:26:44 -0800

Hi Rich:
On Feb 17, 2013, at 4:10 AM, "Signell, Richard" <rsignell@xxxxxxxx> wrote:

> "If you have joinExisting Aggregations, coordinate information will be
> written to a cache directory....Every scour amount of time, any item
> that hasnt been changed since maxAge time will be deleted. Set scour
> to -1 to not scour if you have aggregations that never change."
> 
> The OSCAR dataset is not changing, so JPL could modify their
> threddsConfig.xml to be:
> 
> ...
> <AggregationCache>
> ...
> <scour>-1 hours</scour>
> ...
> </AggregationCache>
> 
> I'm cc'ing the THREDDS list as it seems a common issue.

But then the scour value is set for all datatsets.  I do not know of a way to 
set dataset specific rates.  We run into this problem a lot, though not as 
extreme as the example given.  Recently I have had to start setting up 
featureCollections for some GRIB data, and that has a lot of desired properties 
- you can set collection specific times, and you can have an external process 
force the update.   But, I have not had any success using featureCollections 
for just regular gridded netcdf datasets to replace the older dataset 
aggregation.  I have seen others on this list imply that they haven't quite 
been able to get it working either.

If someone is using featureCollection to aggregate gridded netcdf files  (not 
GRIB files) can they post the URL and include a copy of the relevant portions 
of the catalog.xml?

Thanks,

-Roy M.


**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

• Follow-Ups:
  • Re: [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Jeff McWhirter
  • Re: [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Signell, Richard

• References:
  • [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Signell, Richard