NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hey Patrick, Good to see you back! To answer your question easily, let's take a look at the College of DuPage's dish stats: https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?noaaport3.cod.edu As you can see, their max peak hourly was about 30 GB in one hour; their average is around 20 GB/hr. NOAAAport is a 120 mbsec feed now, and on December 1, 2021, as super-res level 3 radar gets sent across, and during major events, you better believe that number will bounce up further! Gilbert On Fri, Oct 1, 2021 at 3:31 PM Patrick L. Francis <wxprofessor@xxxxxxxxx> wrote: > > Karen, > > There's a name I haven't seen for awhile, great to see your name again!1 > > My queues are typically for the processing of real-time data. I don’t > generally keep an hour of data in the queue because I have redundancy and > in general if data coming across is more than 20-30 minutes old I’m going > to ignore it anyway. I know that most data is archived elsewhere and I’m > not interested in keeping an archive— just wanting to process what I get in > real-time. > > Here's wher it get interesting... or perhaps this is just an old man that > gets excited about silly things... :) I like both having the dish for the > speediness of the delivery, but also to backup friends down the road.. When > I was a professor it was backing up others way back in the day that there > were only a few of us out there.. like gerry at A&M had the massive feeds > going... Well now I'm not a professor anymore, but I like backing up other > weather peeps in the commercial sector too :) > > So, if I have one of my noaaport relays well here.. let me elaborate: > > for those that may not remember I take the feed from the dish to the > novra, then split that multicast feed from the novra to two (minimum) > noaaport relays.. the job of the noaaport relays is to balance the feed to > all of my production servers (where the actual work on the data is done) > ... well, I also have friends out there that depend on timely data, and as > an old unidata member, I like to share, and help others too, who in turn > help me :) So those that have a downstream backup feed from me, may have > an issue with their setup and need to reboot their system, or start up > another and need that data they missed... here is where it gets critical... > > If someone is making a product, and had a hard time getting wherever they > needed to go to boot up their ingest feed, and I am one of their backups, > or if it's a university setup that backs up from me that is running > research, or whatever.... how much data should be kept in the queu e for > them to fire up? > > In the beginning years ago 500m was fine... but by a few years ago we > needed several gig to cover an hour, and we have since had goesR added and > more model products added... > > what I do on each of my boxes is build the ramdisk to keep the queue in > because it's much much faster (less io and such) at xferring data from the > queue to be processed on the fly triggered from receipt, but also because > someone that might be downstream from me as a backup might need to pull > some data too, though I setup my backup feeds from the relays and not the > production servers... > > anyway, I'm just one of the weird guys that thinks about these things, and > years ago more people would chiume in and chat about things I guess lol :) > > so that's my 2 cents, with your 2 cents, so does that mean that together > we make less sense? :) > > thanks for chiming in karen.. good to see your name again :) > > cheers, > > --patrick > > > > _______________________________________________ > NOTE: All exchanges posted to Unidata maintained email lists are > recorded in the Unidata inquiry tracking system and made publicly > available through the web. Users who post to any of the lists we > maintain are reminded to remove any personal information that they > do not want to be made public. > > > ldm-users mailing list > ldm-users@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > https://www.unidata.ucar.edu/mailing_lists/ > -- ---- Gilbert Sebenste Consulting Meteorologist AllisonHouse, LLC
ldm-users archives: