Re: [thredds] Problems with large aggregations on JPL THREDDS?

• To: Jeff McWhirter <jeff.mcwhirter@xxxxxxxxx>
• Subject: Re: [thredds] Problems with large aggregations on JPL THREDDS?
• From: Kenneth Casey - NOAA Federal <kenneth.casey@xxxxxxxx>
• Date: Mon, 18 Feb 2013 09:25:20 -0500

Jeff - we've been considering doing the same thing using our granule
level Geoportal Server... Allow the user to do a rich search, identify
the granules in a collection that they want, then dynamically generate
an NCML aggregation and present that back to them (would be cool if it
could be automatically loaded int our Live Access Server as well). We
weren't thinking about it as a way to overcome performance problems
with large aggregations though.... We were considering it as a way of
building on-demand aggregations of other feature types like point and
profile data.

Ken

Kenneth S. Casey, Ph.D.
Technical Director
NOAA National Oceanographic Data Center
1315 East-West Highway
Silver Spring MD 20910
301-713-3272 ext 133
http://www.nodc.noaa.gov/

On Feb 18, 2013, at 6:17 AM, Jeff McWhirter <jeff.mcwhirter@xxxxxxxxx> wrote:

> We've encountered the same problem in dealing with aggregations in RAMADDA.   
> The climate collections that Don Murray at NOAA is managing go way beyond 
> what the NCML based aggregation facility can address. Don has 10s of 
> thousands of files that make up one collection - model, scenario, experiment, 
> ensemble, parameter,  etc.
>
> Don and I are putting together a RAMADDA plugin that provides rich search and 
> browse services, a data analysis framework (NCO and CDO), and product 
> generation (netcdf subsets, images, etc.)
>
> We haven't addressed OPeNDAP access yet but I could see a hybrid approach - 
> use the internal metadata database to respond to the OPeNDAP metadata listing 
> request and then for the data request figure out what granules are being 
> requested and create an NCML aggregation on the fly for just the small subset 
> of files.
>
> -Jeff
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/

• Follow-Ups:
  • Re: [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Jeff McWhirter

• References:
  • [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Signell, Richard
  • Re: [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Roy Mendelssohn - NOAA Federal
  • Re: [thredds] Problems with large aggregations on JPL THREDDS?
    • From: Jeff McWhirter