Re: [thredds] How to control caching of Aggregated datasets?

• To: "McCann, Mike" <mccann@xxxxxxxxx>
• Subject: Re: [thredds] How to control caching of Aggregated datasets?
• From: Heiko Klein <Heiko.Klein@xxxxxx>
• Date: Fri, 11 Mar 2011 09:31:40 +0100

Thanks Mike,

Best regards,

Heiko

On 2011-03-10 18:05, McCann, Mike wrote:

Hello Heiko,

I was experiencing the same issue and after some discussion with John Caron
I switched my catalog to use a scan aggregation:

   <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
     <aggregation dimName="TIME" type="joinExisting" recheckEvery="2 hour">
       <scan location="<dir>" suffix="_R_TS.nc" subdirs="false" />
     </aggregation>
   </netcdf>

I needed to create a<dir>  with symbolic links to all the files being
aggregated.  The last file in the aggregation is recreated every two hours
and so far this technique has been working fine.

Hope this helps,
Mike


On 3/10/11 8:44 AM, "Heiko Klein"<Heiko.Klein@xxxxxx>  wrote:

Hi,

I have a thredds aggregated dataset, with the setup in catalog.xml:
     <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
      <aggregation dimName="time" type="joinExisting" recheckEvery="10
seconds">
      <netcdf location="/file1.nc" coordValue="1" />
      <netcdf location="/file2.nc" coordValue="2" />
      </aggregation>
     </netcdf>


After changing the aggregation, i.e. by adding a new file, e.g.
file3.nc, or by changing the coordValue, I usually run reinitialize the
catalogues with /thredds/admin/debug?catalogs/reinit

The changes appear in the opendap stream first after a while (some
hours?). The only way I currently found to make the correct results
really appear in opendap is to do the following shortly after each others:
a) clear file object caches
b) delete the corresponding aggregation cache file from the disk.
(And I'm not sure if this allways works?)

Is there a better way to control aggregation changes in the catalog.xml
file, i.e. to flush any aggregation-caches when the aggregation changes?


Best regards,

Heiko

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/

--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY

• Follow-Ups:
  • Re: [thredds] How to control caching of Aggregated datasets?
    • From: John Caron

• References:
  • Re: [thredds] How to control caching of Aggregated datasets?
    • From: McCann, Mike