NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Gerry, re: > Could I call later today? Discuss some of these? I will be in my office all day after about 10:30 (dropping my truck off for service). Since there are no meetings scheduled, I should be free to discuss this situation at length. Tom > We're still testing things related to the model runs for SCOOP, and > sasquatch is the main machine on LDM for that. > > Last evening at 2300z, we initiated a test run starting with a set of > 69-70MB netcdfs from UF (sura-uf-pe6600-1.coastal.ufl.edu) of which we > received 9 of 11. If you look around on sasquatch, the matching in > pqact is for those starting with WANA, and the various data can be > found in ~ldm/data/SCOOP/ANAwinds/* where we have specifically been > concentrating on Katrina data, ca 20050826. > > Justin Davis at UFl inserts, then sees what he's getting on his own ldm > stream to verify the inserts. He then reports out the files, sizes and > times, pretty much as gospel. > > I'm also told another machine, at the Virginia Institute of Marine > Sciences (VIMS) (sura-vims-pe6600-1.vims.edu) received all the files, > but ran out of disk space before writing them all out. > > I saw no particular indication we got all the files via LDM here. I'm > getting overloaded and trying to figure this one out. I've got a window > running notifyme running here at home now. I'd like to capture whenever > we see these files and have some indication of success. So far, I've > not seen 'em all. > > I'm also trying to recraft our regex matching to make sure we're not > shooting ourselves in the foot. We do a fair bit of processing for > database entry of the files and their associated metadata when we see a > new file come in. We're taking some bigtime database hits because of > the number of products we're seeing and the number of duplicates that > come through the system. I'm thinking we may have to approach the > database issues differently from our current approach, and we're working > on it. > > Finally, while the load, memory and idle stats are pretty good, since > we're both relaying and processing on sasquatch, might we be binding > things up ourselves? Looking last night at the stats from your site, > we're moving a lot of data back and forth. Here's a current 'top': > > top - 09:19:40 up 34 min, 2 users, load average: 0.26, 0.60, 0.70 > Tasks: 161 total, 1 running, 160 sleeping, 0 stopped, 0 zombie > Cpu0 : 0.0% us, 0.3% sy, 0.0% ni, 98.0% id, 0.3% wa, 0.3% hi, 1.0% si > Cpu1 : 2.3% us, 1.3% sy, 0.0% ni, 94.7% id, 0.0% wa, 0.3% hi, 1.3% si > Mem: 8163060k total, 5151236k used, 3011824k free, 43956k buffers > Swap: 15647288k total, 0k used, 15647288k free, 3422820k cached > > I'm gonna propose SCOOP host a 1-day meeting for the folks running LDM > and that we invite you or Steve Emmerson out to help us tweak things. > We've several cooperative folks and a few rugged individualists, so > getting agreement and concensus has proven difficult. I'm thinking that > getting the experts in would help a lot there. > > Could I call later today? Discuss some of these? > > Thanks, Gerry > Tom Yoksas wrote: > >>From: Gerry Creager <address@hidden> > >>Organization: TAMU > >>Keywords: 200606072219.k57MJCSZ012682 IDD-SCOOP > > > > > > Hi Gerry, > > > > re: > > > >>to perhaps get on sasquatch and make sure things are reasonably happy? > > > > > > I logged on a couple of minutes ago, and did not see anything amiss > > (but I deleted some old core files in ~ldm). > > > > Was there something in particular that you wanted me to look at? > > > > Cheers, > > > > Tom > > -- > Gerry Creager -- address@hidden > Texas Mesonet -- AATLT, Texas A&M University > Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 > Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 > > Cheers, Tom **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: CIL-628524 Department: Support IDD SCOOP Priority: Normal Status: Closed