Re: 20021114: pqsurf in LDM 5.2

• To: "Neil R. Smith" <neils@xxxxxxxxxxxxxxxxxx>
• Subject: Re: 20021114: pqsurf in LDM 5.2
• From: Robb Kambic <rkambic@xxxxxxxxxxxxxxxx>
• Date: Tue, 19 Nov 2002 18:52:19 -0700 (MST)

Neil,

Here's some compile hints at:

http://www.unidata.ucar.edu/glimpse/ldm/3185

Hope this helps,
Robb...



On Tue, 19 Nov 2002, Neil R. Smith wrote:

Robb Kambic wrote:

> > On Tue, 19 Nov 2002, Neil R. Smith wrote: > > > Robb Kambic wrote:

> > >
> > > On Mon, 18 Nov 2002, Neil R. Smith wrote:
> > >
> > > > Hi Robb,
> > > > I sent you the log files from interactive pqsurf as requested.
> > > > Did you receive it? Does it provide any clues? Do I need
> > > > to provide more info? Thanks, -Neil
> > > >
> > > > Robb Kambic wrote:
> > > > >
> > > > > On Thu, 14 Nov 2002, Unidata Support wrote:
> > > > >
> > > > > >
> > > > > > ------- Forwarded Message
> > > > > >
> > > > > > >To: support@xxxxxxxxxxxxxxxx
> > > > > > >From: "Neil R. Smith" <neils@xxxxxxxxxxxxxxxxxx>
> > > > > > >Subject: pqsurf crashing in ldm 5.2
> > > > > > >Organization: UCAR/Unidata
> > > > > > >Keywords: 200211142352.gAENq6L05743
> > > > > >
> > > > > > Our pqsurf is crashing in our recently upgraded ldm 5.2 (from
> > > > > > 5.0.6). We are useing the same pqsurf.conf file.
> > > > > >
> > > > > > Here are the ldmd.conf errors:
> > > > > > Nov 14 23:02:01 5Q:coriolis rpc.ldmd[188942]: Starting Up (built: 
Nov 13
> > > > > > 2002 21:54:31)
> > > > > > Nov 14 23:02:01 5Q:coriolis pqbinstats[162465]: Starting Up (188942)
> > > > > > Nov 14 23:02:01 5Q:coriolis pqexpire[187333]: Starting Up
> > > > > > Nov 14 23:02:01 5Q:coriolis pqact[181011]: Starting Up
> > > > > > Nov 14 23:02:01 5Q:coriolis pqsurf[169954]: Starting Up (188942)
> > > > > > Nov 14 23:02:01 5Q:coriolis pqact[187141]: Starting Up
> > > > > > .
> > > > >
> > > > > Neil,
> > > > >
> > > > > A couple things to do is to make a few startup changes:
> > > > >
> > > > > -Comment out the pqexpire line with a "#" in the etc/ldmd.conf file. 
That
> > > > > process is not needed any more and it might be causing some problems.
> > > > > - remake the ldmqueue and pqsurfqueue. ie
> > > > >
> > > > > % ldmadmin stop
> > > > > % ldmadmin delqueue
> > > > > % ldmadmin mkqueue
> > > > > % ldmadmin delsurfqueue
> > > > > % ldmadmin mlsurfqueue
> > > > > % ldmadmin start
> > > > >
> > > > > > . (no intervening proc.#169954 messages)
> > > > > > .
> > > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: surface_split: Can't 
handle
> > > > > > MESSAGE_TYPE_UNKNOWN
> > > > > > Nov 14 23:03:33 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: surface_split: Can't 
handle
> > > > > > MESSAGE_TYPE_UNKNOWN
> > > > > > Nov 14 23:03:34 3Q:coriolis pqsurf[169954]: HDR + PIL: SXHI
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: child 187141 terminated 
by
> > > > > > signal 10
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Exiting
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:   Queue usage (bytes):
> > > > > > 72784
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]:            (nregions):
> > > > > > 377
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of products 27
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of observations 
135
> > > > > > Nov 14 23:03:36 5Q:coriolis pqsurf[169954]: Number of dups 2
> > > > > >
> > > > >
> > > > > - another idea is to run the pqsurf in verbose mode from the command 
line.
> > > > >
> > > > > - comment out pqsurf in the etc/ldmd.conf file
> > > > > - run pqsurf from command line as user ldm
> > > > >
> > > > > % pqsurf -v -l -
> > > > >
> > > > > This send output to the terminal.  If you replace -l - with -l log it 
will
> > > > > put the output in file log
> > > > >
> > > > >
> > > > > See comment below first
> > > > >
> > > > > > Here is the ldmd.conf entry:
> > > > > > exec    "pqsurf -p ^S[APX] -Q /data/pqsurf.pq 
/unidata/ldm/etc/pqsurf.conf"
> > > > >
> > > > > I would change the above line to
> > > > >
> > > > > exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -q /data/pqsurf.pq
> > > > > /unidata/ldm/etc/pqsurf.conf"
> > > > >
> > > > > You might be getting some radar products or some other products 
wrongly
> > > > > causing pqsurf to crash.
> > > > >
> > > > > If none of the above works, send me info about your machine and the 
pqsurf
> > > > > log file from the command line.
> > > > >
> > > > > Robb...
> > > > >
> > > > > >
> > > > > > Here are the pqsurf.conf entries:
> > > > > > WMO     ^sao .. (...) ([0-3][0-9])([0-2][0-9])
> > > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > > WMO     ^sao ... (...) ([0-3][0-9])([0-2][0-9])
> > > > > >         DBFILE  /data/ddplus/db_sao/(\2:yy)(\2:mm)\2\3.SA_DB \1
> > > > > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > > > > WMO     ^speci (....) ([0-3][0-9])([0-2][0-9])
> > > > > >         DBFILE  /data/ddplus/db_metar/(\2:yy)(\2:mm)\2\3.METAR_DB \1
> > > > > >
> > > > > > Is any of the above config. antiquated and causing the
> > > > > > problem?
> > >
> > > Neil,
> > >
> > > We had a power outage today, it's a pita to get back your environment etc.
> > > Anyhow, I look at the logs today. It seems the problem isn't with pqsurf
> > > per se, it's the pqact running on the pqsurf.conf that exited.  Since
> > > pqact usuallly doesn't fail unless a entry in the file causes a problem.
> > > I notice all your entires are DBFILE ones.  I wondering if you could
> > > comment those out with a '#' and make a entry that writes to a file?  This
> > > way we could determine if it's the DBFILE entries or not.  How about using
> > > a entry like this:
> > >
> > > WMO     ^metar (....) ([0-3][0-9])([0-2][0-9])
> > >         file    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo

> > Neil, > > My mistake, file action must be uppercase, ie FILE. > > WMO ^metar (....) ([0-3][0-9])([0-2][0-9])

>         FILE    /data/ddplus/(\2:yy)(\2:mm)\2\3.wmo

> > The metars products arrive at a rate of 1 a sec. You can scan

> the queue looking for product via pqcat, ie

> > % pqcat -vl - -q <yourPQSURFqueue> > /dev/null > > For more info on pqcat look at:

> http://www.unidata.ucar.edu/cgi-bin/man-cgi?pqcat+1

> > Robb...

> > >
> > > This will file all the metars in a file like:  /data/ddplus/02111812.wmo
> > > Rememeber the white spaces needs to be tabs.
> > >
> > > Let me know what you find out.
> > >
> > > Robb...
> >
> > Hi Robb,
> > Nope. That didn't produce any file output. Is there a diagnostic
> > I can look at or generate for analysis? Also, what is the frequency
> > of these products so I can know when to expect them in the queue?
> > Thanks, -Neil

OK. The telltail seems to be in your second bit of info - how to look at the queue with pqcat. When I do

coriolis{ldm}52%  pqcat -vl -  -q /data/surf.pq > /dev/null
Nov 20 00:43:57 pqcat: Starting Up (28721)
Nov 20 00:43:57 pqcat: pq_open failed: /data/surf.pq: No such file or
directory
Nov 20 00:43:57 pqcat: Exiting
Nov 20 00:43:57 pqcat: Number of products 0

Looks like an empty queue. To review the ldmd.conf entry:

exec    "pqsurf -p ^S(A....|P....|XUS8.|XUS91) -q /data/pqsurf.pq
/unidata/ldm/etc/pqsurf.conf"
which I copied from your email ... which cited pqsurf.pq with -q instead

of -Q. OK. My fault. Well, correcting the q/Q issue, I get a pqsurf.pq with products I can see qith pqcat. With 'file' changed to 'FILE' in the recommended step in pqsurf.conf, I do indeed get the files created in

the

expected directory.

Now, is this a dbm issue? What did I need to pay attention to during
ldm package build?
-Neil
--
Neil R. Smith, Comp. Sys. Mngr.         neils@xxxxxxxxxxxxxxxxxx
Dept. Atmospheric Sci., Texas A&M Univ.     979/845-6272 FAX:979/862-4466

==============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
rkambic@xxxxxxxxxxxxxxxx                   WWW: http://www.unidata.ucar.edu/
==============================================================================