NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
I know ucnids has gone through several iterations over the years. I checked my ingest for TLX NVW and it seems to be decompressing them well: -rw-r--r--. 1 ldm ops 8936 21-02-25 21:31:53 202102252122_NVW.nids -rw-r--r--. 1 ldm ops 2753 21-02-25 21:31:53 202102252122_NVW.nidz -rw-r--r--. 1 ldm ops 8796 21-02-25 21:41:35 202102252131_NVW.nids -rw-r--r--. 1 ldm ops 2727 21-02-25 21:41:35 202102252131_NVW.nidz -rw-r--r--. 1 ldm ops 8984 21-02-25 21:51:09 202102252141_NVW.nids -rw-r--r--. 1 ldm ops 2823 21-02-25 21:51:09 202102252141_NVW.nidz -rw-r--r--. 1 ldm ops 9090 21-02-25 22:00:46 202102252151_NVW.nids -rw-r--r--. 1 ldm ops 2888 21-02-25 22:00:46 202102252151_NVW.nidz -rw-r--r--. 1 ldm ops 9110 21-02-25 22:10:21 202102252200_NVW.nids -rw-r--r--. 1 ldm ops 2893 21-02-25 22:10:21 202102252200_NVW.nidz -rw-r--r--. 1 ldm ops 8200 21-02-25 22:20:01 202102252210_NVW.nids -rw-r--r--. 1 ldm ops 2688 21-02-25 22:20:01 202102252210_NVW.nidz -rw-r--r--. 1 ldm ops 7946 21-02-25 22:29:35 202102252219_NVW.nids -rw-r--r--. 1 ldm ops 2700 21-02-25 22:29:35 202102252219_NVW.nidz -rw-r--r--. 1 ldm ops 7864 21-02-25 22:39:10 202102252229_NVW.nids -rw-r--r--. 1 ldm ops 2569 21-02-25 22:39:10 202102252229_NVW.nidz -rw-r--r--. 1 ldm ops 7940 21-02-25 22:48:49 202102252239_NVW.nids -rw-r--r--. 1 ldm ops 2592 21-02-25 22:48:49 202102252239_NVW.nidz I don't know if there is a newer version out there? I could give you my ucnids source. Dan. On Thu, Feb 25, 2021 at 3:57 PM Gregory Grosshans via ldm-users < ldm-users@xxxxxxxxxxxxxxxx> wrote: > We are spinning up a new Vad Wind Profiler dataflow and using the "ucnids" > program to decompress the data via LDM. Sometimes the output file is > decompressed, and other times it is NOT decompressed. I don't see any > orphaned ucnids processes or entries in the ldmd.log file until I turn on > verbose logging. The pqact entry is: > > NEXRAD3 ^SDUS3. .... ([0-3][0-9])([0-2][0-9])([0-5][0-9]).*/p(NVW)(...) > PIPE -close /nfsops/ops_users/ldmopsng1/util/ucnids -n - > radar/nflowvad/\5/prod48/(\1:yy)(\1:mm)\1\2.\3 > > For the TLX 88D site in the past 24 hours there are about 151 files saved > (i.e. data is scrubbed after 24 hours). Of these 151 files, 73 files (i.e. > ~half of the files) are not UNcompressed. > > > -rw-r--r--. 1 ldmopsng1 users 8598 Feb 25 20:53 21022520.43 > -rw-r--r--. 1 ldmopsng1 users 8780 Feb 25 21:03 21022520.53 > -rw-r--r--. 1 ldmopsng1 users 8968 Feb 25 21:12 21022521.03 > -rw-r--r--. 1 ldmopsng1 users 3946 Feb 25 21:22 21022521.12 NOT > decompressed > -rw-r--r--. 1 ldmopsng1 users 3946 Feb 25 21:31 21022521.22 NOT > decompressed > > > The version of LDM is 6.13.11. > > When I have set ldmd.log to be verbose for the pqact processing this data > I see entries like this, but the file was NOT decompressed (2131Z > 21022521.22 file): > > 20210225T212217.975508Z pqact[11328] > palt.c:processProduct:1340 INFO 2650 20210225212217.885403 > NEXRAD3 1145245 SDUS34 KOUN 252112 /pNVWTLX > 20210225T212217.978774Z pqact[11328] > filel.c:fl_removeAndFree:425 INFO Deleting closed PIPE entry: > pid=6990, cmd="-close /nfsops/ops_users/ldmopsng1/util/ucnids -n - > radar/nflowvad/TLX/prod48/21022521.12" > 20210225T212217.978880Z pqact[6990] filel.c:pipe_open:1798 > INFO Executing decoder > "/nfsops/ops_users/ldmopsng1/util/ucnids" > > It is odd the INFO for executing the decoder is in the log file after the > entry about LDM deleting the closed PIPE. > > Below are the log entries for the 2113Z 21022521.03 file that WAS > decompressed: > > 20210225T211240.207544Z pqact[11328] > palt.c:processProduct:1340 INFO 2699 20210225211240.103390 > NEXRAD3 1125246 SDUS34 KOUN 252103 /pNVWTLX > 20210225T211240.209361Z pqact[17988] filel.c:pipe_open:1798 > INFO Executing decoder > "/nfsops/ops_users/ldmopsng1/util/ucnids" > 20210225T211240.211086Z pqact[11328] > filel.c:fl_removeAndFree:425 INFO Deleting closed PIPE entry: > pid=17989, cmd="-close /nfsops/ops_users/ldmopsng1/util/ucnids -n - > radar/nflowvad/TLX/prod48/21022521.03" > > > So both log entries look similar, and I'm not sure why one file is > decompressed and another is not. The only thing I see different is the > order of the log entries, the one out of order isn't DEcompressed, but I > suspect this has more to do the with system logging and is a coincidence. > > The limits for the number of open files has been increased on this > server. The OPEN_MAX, ulimit -a, limit descriptors, was changed from 1021 > for the LDM user to 40,000 soft limit and 40960 for the hard limit. > > Has anyone seen something like this with ucnids, or another type of > decoder, determined what was going on and have suggestions? > > Thanks, > Gregg > > -- > ********************** > *I'm primarily telecommuting. If you need to reach me, call my work cell > 405-249-5310.* > > > > > *========================================================================Email > seems to be generating increasing inefficiencies in organizations. I > learned from a manager a Stanford Computer Science professor no longer uses > email for communication, but uses SNAIL mail, telephone calls, and person > to person visits. I'm considering the same. 405-325-2462* > > *Storm Prediction Center* > > *120 David L. Boren Blvd, Suite 2330Norman, OK 73072* > _______________________________________________ > NOTE: All exchanges posted to Unidata maintained email lists are > recorded in the Unidata inquiry tracking system and made publicly > available through the web. Users who post to any of the lists we > maintain are reminded to remove any personal information that they > do not want to be made public. > > > ldm-users mailing list > ldm-users@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > https://www.unidata.ucar.edu/mailing_lists/ > -- *Dan Vietor* *Senior Research Meteorologist* CIRA, Colorado State Univ Aviation Weather Center Kansas City, MO 816.584.7211
ldm-users archives: