NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [thredds] over written file has old global attributes

Hi Jean,
The TDS data file cache holds a pointer to each cached file along with all the metadata for the file. The attributes get pulled from the cache while data requests access the file on disk. The cache is supposed to check the last modified date for the file and re-read if it has been modified since the cache was last updated. But what you are reporting sure makes it sound like that capability (checking last modified) might be broken.
Just to make sure, next time you see this behavior, could you take a look at the cache on your server and, if the file is listed, clear the cache and see if the problem goes away? You have to have TDS Remote Management setup (see the TDS Reference for details, http://www.unidata.ucar.edu/projects/THREDDS/tech/reference/).
It will be a few weeks before we will be able to look at this in depth. For now, we have a new 3.16 release with a minor tweak that turns off caching if the max number of cached items is set to zero. You set the max values by editing your threddsConfig.xml file which is in the content/thredds directory. There are three types of caching you can adjust: NetcdfFileCache, NetcdfDatasetCache, and HTTPFileCache. You may only need to modify the NetcdfFileCache settings but NetcdfDatasetCache may come into play if you have datasets described with NcML. To change the settings, find the elements of the same name in your threddsConfig.xml and set max to 0.
You probably are not seeing this behavior every time because the cache occasionally scours out files that aren't as active as others. 

Hope that helps,

Ethan

Jean.C.Newman@xxxxxxxx wrote:
On the sift machine, because we were doing testing, we have "opendap" with & without thredds and "looking" at the file - ncdump & dncdump with "opendap no thredds" both give correct attributes while dcdump with "thredds" gives old attribute information. Jean 

ncdump -h crescent_runup_ha.nc
...
// global attributes:
                 :history = "MOST v2.080 11-05-2007  16:09:22" ;
                 :title = "Aleutian-Cascadia: Mwt 8.4, 21.00*b65" ;
dncdump -h http://sift.pmel.noaa.gov/thredds/dodsC/data/crescent_runup_ha.nc 
...
// global attributes:
                 :history = "MOST v2.080 11-05-2007  11:49:33" ;
                 :title = "Aleutian-Cascadia: Mwt 8.5, 29.00*b25" ;

dncdump -h http://sift.pmel.noaa.gov/opendap/crescent_runup_ha.nc
...
// global attributes:
                 :history = "MOST v2.080 11-05-2007  16:09:22" ;
                 :title = "Aleutian-Cascadia: Mwt 8.4, 21.00*b65" ;
  ___________________________________________________________
   Jean Newman                             Tel: 206-526-6531
   NOAA Center for Tsunami Research
   NOAA/PMEL/OERD2 - UW/JISAO              FAX: 206-526-6485
   7600 Sand Point Way NE, Bldg. 3    jean.c.newman@xxxxxxxx
   Seattle, WA 98115-6349               jcn@xxxxxxxxxxxxxxxx
  _________________________. URL: http://nctr.pmel.noaa.gov/
                          (__________________________________

On Tue, 6 Nov 2007, Jean.C.Newman@xxxxxxxx wrote:


One other thing, the "old attributes" lasts. I made a run yesterday at
11:23 am and it was over written several time the last at 16:09 and today
if I look at that output file via LAS/"dncdump -h" it still shows the "old
stuff" from the 11:23 run. "ncdump -h crescent_runup_ha.nc" has the
correct 16:09 information. Thanks, Jean
 ___________________________________________________________
  Jean Newman                             Tel: 206-526-6531

On Tue, 6 Nov 2007, Jean.C.Newman@xxxxxxxx wrote:


The question:
Is there some way to turn off caching of metadata so the file is checked
every time it's accessed? Jean
 ___________________________________________________________
  Jean Newman                             Tel: 206-526-6531

On Tue, 6 Nov 2007, Jean.C.Newman@xxxxxxxx wrote:


It's not a browser caching problem. I have a very old LAS that accesses
these files. From the LAS, input for a model run is set up then a servlet
is run on 1 of several possible machines that starts a FORTRAN model run
that overwwites files so that the LAS interface has static files/"DODs"
URLs for the output. We are moving to different code but need to keep this
working until the new is up and reliable. The data updates, I can see
that in the plots, but I use global attributes in labeling and I get the
old information sometimes, 2 out of 31 SIM runs. The example I put out on
the sift machine was run on an internal machine and then sftp-ed to sift.
Then I made another run and again sftp-ed it to the sift machine and saw
the same "old attributes" problem, which I really didn't think I would.
If tomcat is restarted then I get the new information so I imagine it is
some caching problem. I did try removing the file before rerunning the
SIM and that did not solve the problem. Is there some way to turn off
caching of metadata so the file is checked every time it's accessed? Jean
 ___________________________________________________________
  Jean Newman                             Tel: 206-526-6531


--
Ethan R. Davis                                Telephone: (303) 497-8155
Software Engineer                             Fax:       (303) 497-8690
UCAR Unidata Program Center                   E-mail:    edavis@xxxxxxxx
P.O. Box 3000
Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
---------------------------------------------------------------------------
  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: