Re: [thredds] NcML aggregation strangely pulling files from multiple datasets

• To: Roy Mendelssohn <Roy.Mendelssohn@xxxxxxxx>
• Subject: Re: [thredds] NcML aggregation strangely pulling files from multiple datasets
• From: John Maurer <jmaurer@xxxxxxxxxx>
• Date: Thu, 19 May 2011 12:59:14 -1000

MANY THANKS, ROY! That cleared up all the problems. I'm hugely relieved that
I and/or my disk weren't going haywire. :) OK, catching my breath... phew.
Cheers,
John Maurer

On Thu, May 19, 2011 at 12:48 PM, Roy Mendelssohn
<Roy.Mendelssohn@xxxxxxxx>wrote:

> Hi John:
>
> We have run into this too, particularly when adding new datasets.  Whenever
> we do, we always do the following and find it clears up the problem:
>
> 1.  Delete all of $TOMCAT_HOME/content/thredds/cache/agg
>
> 2. Delete all of $TOMCAT_HOME/content/thredds/cache/ehcache
>
> not certain ehculprit - it looks like the ehcache might assosciate the agg
> cache with certain datasets, and if that gets changed the list gets out of
> whack, but that is a wild guess.  What I do know is that when we have that
> problem, those two steps clear it up.
>
> The downside is that access to large aggregate datasets is slow the first
> time they are accessed.
>
> -Roy
>
> On May 19, 2011, at 2:24 PM, John Maurer wrote:
>
> > Hi all,
> > I'm seeing some weird behavior today with one of our joinExisting NcML
> aggregations for high-frequency radio (HFR). The OPeNDAP page lists
> variables that are from a totally different dataset (e.g. HFR does not
> measure water temperature):
> http://oos.soest.hawaii.edu/thredds/idd/hfr.html?dataset=kokagg. I looked
> at "content/thredds/cache/agg/" at the aggregation cache file for the
> dataset (hioos-hfr-kokagg) and it confirms that it's strangely pulling files
> from 3 unrelated datasets: a glider, an NCOM global FMRC, and the HFR data.
> Since nothing has changed recently in my config file for the dataset and I
> haven't recently upgraded TDS, this has me worried about a possible disk
> problem from recent power outages? The three datasets are not subdirectories
> of each other, so I don't see how TDS can be lumping them together. Any
> ideas what could be causing this?
> >
> > In addition to the above, 5 of my FMRC's are suddenly inaccessible today
> as well and were recently working just fine. It would seem their file
> aggregations may have similarly gone haywire as well, but their cache files
> under "content/thredds/collection/" (right?) are binary (.jdb) so I can't
> confirm whether they're combining multiple datasets. The catalogs give an
> error in threddsServlet.log with the message:
> >         at
> ucar.nc2.ft.fmrc.FmrcDataset.getNetcdfDataset2D(FmrcDataset.java:166)
> >         at ucar.nc2.ft.fmrc.Fmrc.getDataset2D(Fmrc.java:183)
> >
> > What's the best way for me to clear all TDS caches? Is it safe to remove
> "content/thredds/cache/" entirely?, and then restart Tomcat/TDS? Will it get
> re-built then automatically after I access each dataset?
> >
> > I've copied the relevant agg and collection caches, config files, and
> threddsServlet.log to my anonymous ftp server at:
> >
> > ftp.soest.hawaii.edu
> > jmaurer/thredds/
> >
> > Thanks for your help!,
> > John Maurer
> > Pacific Islands Ocean Observing System (PacIOOS)
> > University of Hawaii
> > _______________________________________________
> > thredds mailing list
> > thredds@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
>
>

• Follow-Ups:
  • Re: [thredds] NcML aggregation strangely pulling files from multiple datasets
    • From: McCann, Mike

• References:
  • [thredds] NcML aggregation strangely pulling files from multiple datasets
    • From: John Maurer
  • Re: [thredds] NcML aggregation strangely pulling files from multiple datasets
    • From: Roy Mendelssohn