NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
My rule of thumb is that if you're an: *End node* (i.e not feeding another LDM) -- you need only as much as the largest product you'll see in the stream plus some overhead. We tend to dump large model files into the queue, sometimes as large as 800MB. You want to make sure you can process and save that product to disk so the queue size here would be about 4 times that size or about 3Gb. But since you're doing NOAAPort, most of the products are small by that comparison. I tend to keep a queue size large enough to store no more than 5 min of data just so I can monitor what's coming across. So I use the default (500MB) which gives me about 3-5 minutes of storage in the queue. *Relay node* -- you need a large enough queue to store data for an outage. At my work, typical outages are between 5-10 minutes, mostly from system reboots after OS patching. Sometimes we have small networking outages that last 10 to 15 minutes. We then compute the LDM throughput to get a queue size. So if you take 15 minutes, to be safe, with about 200MB per minute, you'd get 3Gb. If you're pushing more than NOAAPort data, this will increase the size of the queue. We have some systems with systems with large queues, well over 5GB. I'm not one for keeping an hour in the queue because the number of products from NOAAPort would make restarts very slow as the downstream LDM would spend a lot of time sync'ing all the products from the upstream LDM. So it's good to find the right balance, large enough to recover from an outage but not too large to tie up all your disk space or make queue sync'ing difficult. Dan. On Fri, Oct 1, 2021 at 3:52 PM Patrick L. Francis <wxprofessor@xxxxxxxxx> wrote: > > so does that means gilbert, you agree we should keep an hour's worth of > data in the queue? or is time not that important a factor? :) > > > cheers, > > --patrick > > > > _______________________________________________ > NOTE: All exchanges posted to Unidata maintained email lists are > recorded in the Unidata inquiry tracking system and made publicly > available through the web. Users who post to any of the lists we > maintain are reminded to remove any personal information that they > do not want to be made public. > > > ldm-users mailing list > ldm-users@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > https://www.unidata.ucar.edu/mailing_lists/ > -- *Dan Vietor* *Senior Research Meteorologist* CIRA, Colorado State Univ Aviation Weather Center Kansas City, MO 816.584.7211
ldm-users archives: