NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [thredds] Problems with large aggregations on JPL THREDDS?

  • To: Roy Mendelssohn - NOAA Federal <roy.mendelssohn@xxxxxxxx>
  • Subject: Re: [thredds] Problems with large aggregations on JPL THREDDS?
  • From: "Signell, Richard" <rsignell@xxxxxxxx>
  • Date: Sun, 17 Feb 2013 11:32:44 -0500
Roy,

>From this page:
http://www.unidata.ucar.edu/projects/THREDDS/tech/tds4.3/reference/collections/FeatureCollections.html

It looks like the only "featureType" values currently allowed in a
FeatureCollection are
TDS 4.2: FMRC, Point, and Station
TDS 4.3: FMRC, Point,  Station and GRIB

We've been doing FeatureCollection FMRC featureTypes with netcdf files
and it works fine.

But I don't think you can use FeatureCollection for joinNew or
joinExisting aggregations of netcdf files yet.

-Rich

On Sun, Feb 17, 2013 at 10:26 AM, Roy Mendelssohn - NOAA Federal
<roy.mendelssohn@xxxxxxxx> wrote:
> Hi Rich:
> On Feb 17, 2013, at 4:10 AM, "Signell, Richard" <rsignell@xxxxxxxx> wrote:
>
>> "If you have joinExisting Aggregations, coordinate information will be
>> written to a cache directory....Every scour amount of time, any item
>> that hasnt been changed since maxAge time will be deleted. Set scour
>> to -1 to not scour if you have aggregations that never change."
>>
>> The OSCAR dataset is not changing, so JPL could modify their
>> threddsConfig.xml to be:
>>
>> ...
>> <AggregationCache>
>> ...
>> <scour>-1 hours</scour>
>> ...
>> </AggregationCache>
>>
>> I'm cc'ing the THREDDS list as it seems a common issue.
>
> But then the scour value is set for all datatsets.  I do not know of a way to 
> set dataset specific rates.  We run into this problem a lot, though not as 
> extreme as the example given.  Recently I have had to start setting up 
> featureCollections for some GRIB data, and that has a lot of desired 
> properties - you can set collection specific times, and you can have an 
> external process force the update.   But, I have not had any success using 
> featureCollections for just regular gridded netcdf datasets to replace the 
> older dataset aggregation.  I have seen others on this list imply that they 
> haven't quite been able to get it working either.
>
> If someone is using featureCollection to aggregate gridded netcdf files  (not 
> GRIB files) can they post the URL and include a copy of the relevant portions 
> of the catalog.xml?
>
> Thanks,
>
> -Roy M.
>
>
> **********************
> "The contents of this message do not reflect any position of the U.S. 
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598



  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: