NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Elliott, re: > Since yesterday afternoon, we have started seeing data from > idd.unidata.ucar.edu being rejected because it is too old. I have checked > the NEXRAD3 and NGRID feeds, but it is likely happening in others. OK. re: > We are in Oklahoma on CST, so 14:19 CST is 20:19 UTC. The data below appears > to be just over an hour old: > Nov 5 14:14:23 irads-ingest0 idd.unidata.ucar.edu[19432] INFO: Ignoring > too-old product: 279 20181105190549.724 NEXRAD3 82757661 SDUS54 KFWD > 051859 /pNVLDFW > Nov 5 14:14:23 irads-ingest0 idd.unidata.ucar.edu[19432] INFO: Ignoring > too-old product: 6145 20181105190549.725 NEXRAD3 82757662 SDUS33 KBIS > 051859 /pN3SBIS > Nov 5 14:14:23 irads-ingest0 idd.unidata.ucar.edu[19432] INFO: Ignoring > too-old product: 8300 20181105190549.741 NEXRAD3 82757663 SDUS76 KSGX > 051904 /pN0ZNKX > Nov 5 14:14:23 irads-ingest0 idd.unidata.ucar.edu[19432] INFO: Ignoring > too-old product: 4107 20181105190549.742 NEXRAD3 82757664 SDUS54 KFWD > 051859 /pNCRDFW > > Nov 5 14:19:05 irads-ingest0 idd.unidata.ucar.edu[19442] INFO: Ignoring > too-old product: 3016 20181105190006.857 NGRID 37304044 FOUS15 KWNO > 051800 /pNBHUSA > Nov 5 14:19:05 irads-ingest0 idd.unidata.ucar.edu[19442] INFO: Ignoring > too-old product: 3426 20181105190006.857 NGRID 37304045 FOUS15 KWNO > 051800 /pNBHUSA > > We saw latency drift upward yesterday afternoon till LDM started rejecting: > > https://status.irads.ou.edu/images/L3latency.png > > The green line is from the host connected to the IDD. The blue line is > from a different host that receives data from a local NOAAPort that is > functioning properly. Our real-time stats pages also show that your feed latencies have grown: Unidata HomePage http://www.unidata.ucar.edu Data -> IDD Operational Status http://rtstats.unidata.ucar.edu/rtstats/ Real-time IDD Statistics -> Statistics by Host http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex irads-ingest0.net.ou.edu [6.12.14] http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?irads-ingest0.net.ou.edu The higher volume feeds from idd.unidata.ucar.edu are all showing unacceptably high latencies. For instance: NEXRAD3 http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NEXRAD3+irads-ingest0.net.ou.edu NGRID http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NGRID+irads-ingest0.net.ou.edu CONDUIT http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+irads-ingest0.net.ou.edu re: > We aren't seeing any issues from other sources. We receive the NEXRAD2 > feed directly from TDS/RDS and those feeds are functioning properly > as well. > > Please let us know if this is a known issue or if there is something we > need to do on our end. We are not seeing anything out of the ordinary on the real-server backend node of our top level IDD relay cluster, idd.unidata.ucar.edu, that is feeding delta-ingest0.irads.ou.edu. In fact, we are feeding NEXRAD3 and some other feeds on a single REQUEST to a machine at UIowa, and its latencies are on the order of a second or two. We noticed that the latencies for your 5-way split of CONDUIT feed started getting bad (going south) yesterday, November 4 at approx. 9 UTC: http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+irads-ingest0.net.ou.edu The interesting thing is that the latencies dropped down to near zero for awhile, increased again, then dropped between 18 and 21 UTC (approx) and then have gotten steadily worse since. Question: - was anything done with the networking yesterday morning? Suggested test: - try changing your NEXRAD3 feed REQUEST from idd.unidata.ucar.edu to idd.meteo.psu.edu And, please let us know the results of this test. Cheers, Tom -- **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: ZAJ-969398 Department: Support IDD Priority: Normal Status: Closed =================== NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.