NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
>From: "Thomas L. Mote" <address@hidden> >Organization: University of Georgia >Keywords: 200001042125.OAA01383 McIDAS-XCD setup Tom, re: you using NIS+ >You are correct! I am running NIS+. I added the files entry >under services in my /etc/nsswitch.conf file. After >rebooting, I was able to do a a telnet cacimbo.ggy.uga.edu >500 and get it to respond as it should. Excellent! I just pointed my McIDAS session at your machine and verified that I can talk to your ADDE remote server: DATALOC ADD RTIMAGES cacimbo.ggy.uga.edu DSINFO IMAGE RTIMAGES Dataset Names of Type: IMAGE in Group: RTIMAGES Name NumPos Content ------------ ------ -------------------------------------- ANTARCTIC 10 Antarctic IR Composite EDFLOATER-I 10 Educational Floater EDFLOATER-II 10 Educational Floater II GE-IR 10 GOES-East North America IR GE-IRTOPO 10 GOES-East IR/TOPO Composite GE-VIS 10 GOES-East North America VIS GE-VISTOPO 10 GOES-East VIS/TOPO Composite GE-WV 10 GOES-East North America H2O GEW-IR 10 GOES-East/West IR Composite GEW-IRTOPO 10 GOES-East/West IR/TOPO Composite GEW-VIS 10 GOES-East/West VIS Composite GEW-VISTOPO 10 GOES-East/West VIS/TOPO Composite GEW-WV 10 GOES-East/West H2O Composite GW-IR 10 GOES-West Western US IR GW-IRTOPO 10 GOES-West IR/TOPO Composite GW-VIS 10 GOES-West Western US VIS GW-VISTOPO 10 GOES-West VIS/TOPO Composite GW-WV 10 GOES-West Western US H2O MDR 10 Manually Digitized Radar MDRTOPO 10 MDR/TOPO Composite MOLL-IR 10 Mollweide Composite IR MOLL-IRTOPO 10 Mollweide IR/TOPO Composite MOLL-WV 10 Mollweide Composite H2O RESFLOATER 10 Research Floater DSINFO -- done I also verified that I can see images on your system: IMGLIST RTIMAGES/GE-IR.ALL Image file directory listing for:RTIMAGES/GE-IR Pos Satellite/ Date Time Center Band(s) sensor Lat Lon --- ------------- ------------ -------- ---- ---- ------------ 1 G-8 IMG 6 JAN 00006 12:15:00 23 71 4 2 G-8 IMG 6 JAN 00006 13:15:00 23 71 4 3 G-8 IMG 6 JAN 00006 14:15:00 23 71 4 4 G-8 IMG 6 JAN 00006 15:15:00 23 71 4 5 G-8 IMG 6 JAN 00006 16:15:00 23 71 4 6 G-8 IMG 6 JAN 00006 19:15:00 23 71 4 7 G-8 IMG 6 JAN 00006 20:15:00 23 71 4 8 G-8 IMG 6 JAN 00006 21:15:00 23 71 4 9 G-8 IMG 6 JAN 00006 22:15:00 23 71 4 10 G-8 IMG 6 JAN 00006 11:15:00 23 71 4 IMGLIST: done Next, I tried to see if your point source data files were accessible: DATALOC ADD RTPTSRC cacimbo.ggy.uga.edu Group Name Server IP Address -------------------- ---------------------------------------- RTPTSRC CACIMBO.GGY.UGA.EDU <LOCAL-DATA> indicates that data will be accessed from the local data directory. DATALOC -- done PTLIST RTPTSRC/PTSRCS.ALL FORM=FILE PTLIST: No MD files found PTLIST: Done This tells me that either no MD files are being created from XCD processes or that the REDIRECTions in the 'mcidas' account are not valid. re: setup the 'mcadde' account to have the same HOME directory as 'mcidas' >OK. This I didn't realize. I also (before rebooting) >changed the shell on user mcadde to /bin/false and changed >the home directory for mcadde to /unidata/home/mcidas. I >also changed my inetd.conf file. I have attached these >files. Very good. re: testing point source data access >OK. As before, I had already gone through all of these >steps before I sent the question. (Not trying to be >smart... just want you to know I'm making an effort.) I understand completely. I included the information just in case you hadn't done the steps. >I went through the steps again after getting the ADDE server >responding. I have attached the LOCDATA.BAT file. I also >checked the redirections with a REDIRECT LIST. Everything >is set to /data/mcidasd. > >I then tried a SFCPLOT T USA. No dice. This is not good. The failure on your side matches my test above. >I tried a DATALOC ADD RTPTSRC LOCAL-DATA before the SFCPLOT >command. Nada. This must mean that the REDIRECTions are wrong or that MD files are not being created by the XCD decoders. >So, I went to the SFCPLT command. Still not finding data. >BTW, I see no MD files in my /data/mcidasd directory. Bingo! >I will assume this is significant! I have all sorts of files, >but no MD files. I have attached a listing of my >/data/mcidasd directory. The other thing I discovered is that I can't access GRID files from your system. re: turning on GRID decoding >I had set up the GRIB decoding before writing. (Again, not >trying to be smart...) DECINFO does show the GRIB decoder >to be active. OK. I am baffled right now. >I still wonder about the MD files. I should be getting >them. You should if the following was done: o added the xcd_run entries to pqact.conf and restarted your LDM o copied SCHEMA from the ~mcidas/data directory to /data/mcidasd (the point source decoders need the information in the schema file in order to know how to write the data) >I still have all of the "old" decoders running in >addition to the XCD. Old decoders? Do you mean ldm-mcidas decoders? If so, I need to remind you that the point source and grid files that used to be in the Unidata-Wisconsin channel back on July 1 of 1999. The only products coming in the Unidata-Wisconsin datastream anymore are images. >I have attached my pqact.conf file. Great. >Maybe the problem is there. (A pqactcheck shows it to >be OK.) I am glad that you tested the file contents integrity! >Some of this stuff is probably out of date since I >was gone for over a year. I haven't had a chance to update >the pqact. OK. >I'm not sure what to try next, but we do seem to be >chipping away at problems. Yes, we are making progress. The fact that remote access works to your ADDE server is good. The whole ADDE thing is what is so cool about McIDAS! To prove this to yourself, try the following from a McIDAS-X session started as the user 'mcidas': DATALOC ADD RTPTSRC ADDE.UNIDATA.UCAR.EDU SFCPLOT T USA DATALOC ADD RTPTSRC cacimbo.ggy.uga.edu Pretty cool, huh!? Comments on files you attached: services:: ... # UNIDATA stuff ldm 388/udp unidata # UCAR Unidata LDM ldm 388/tcp unidata # UCAR Unidata LDM mcserv 500/tcp # McIDAS ADDE port mccompress 503/tcp # McIDAS ADDE compression port Looks good. inetd.conf:: mcserv stream tcp nowait mcadde /unidata/home/mcidas/bin/mcservsh mcservsh -H /unidata/home/mcidas mccompress stream tcp nowait mcadde /unidata/home/mcidas/bin/mcservsh mcservsh -H /unidata/home/mcidas This also looks correct. The routine /unidata/home/mcidas/bin/mcservsh is a Bourne shell script that will read in information from the file /unidata/home/mcidas/.mcenv. '.mcenv', contains McIDAS envirionment information (e.g. MCDATA, MCPATH, MCGUI, PATH, etc.) that is needed to run McIDAS applications. Since I can list out (and display) imagery from your machine, I figure that '.mcenv' is setup correctly. LOCDATA.BAT:: This file was zero lenght in your message. pqact.conf:: My comments are sorrounded by '******************************' below. ##### #Entries for XCD decoders # DDPLUS|IDS ^.* PIPE xcd_run DDS HRS ^.* PIPE xcd_run HRS ****************************** Looks good. The other things that are needed are (you probably already did these, but others may read these transactions to see what could be wrong): o copy xcd_run to ~ldm/decoders directory o edit ~ldm/decoders/xcd_run and adjust MCDATA, MCPATH, PATH, LD_LIBRARY_PATH to match your McIDAS installation o make sure that xcd_run has execute permission set ****************************** ... #### MCIDAS # Pattern/actions for the Unidata/Wisconsin (i.e. McIDAS) data stream. # See the man page documentation for gunrv2, lwfile, lwtmd2, and lwtoa3 # for more info on how to customize these. # # GRID files: # MCIDAS ^(GUNRV2 .*) PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/gunrv2 -d /data/mcidasd -v # Opaque files: # MCIDAS ^(LWFILE .*) PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/lwfile -d /data/mcidasd -v # MD files: # MCIDAS ^(LWTMD2 .*) PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/lwtmd2 -d /data/mcidasd -v # AREA files: # MCIDAS ^(LWTOA3 .*) PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/lwtoa3 -d /data/mcidasd -v # Profiler files: # MCIDAS ^(LWFILE PROFILER.CDF) PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/lwfile -d /data/mcidasd -v MCIDAS ^(LWFILE) (PROFILER.CDF) .* DIALPROD=(..) EXEC cdftomd -d /data/mcidasd \3 \2 81 # Fronts for McIDAS WMO ^ASUS01 .... ([0-3][0-9])([0-2][0-9]) FILE /data/surface/front/ASUS1%j ****************************** You can comment out the entries for GUNRV2, LWFILE, and LWTMD2 above. Again, the only data in the Unidata-Wisconsin datastream since July 1 is imagery. If you are going to want FSL wind profiler data decoded into McIDAS MD files, you need to add the following entries to pqact.conf: FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)\..* PIPE -close /unidata/home/mcidas/ldm-mcidas/bin/proftomd -v -l /usr/local/ldm/logs/ldm-mcidas.log -d /data/mcidasd U2 WPRO 81 FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(.*)\..* PIPE /unidata/home/mcidas/ldm-mcidas/bin/proftomd -v -l /usr/local/ldm/logs/ldm-mcidas.log -d /data/mcidasd U6 WPR6 91 This decoder requires that the McIDAS SCHEMA file be in the ouput data directory, /data/mcidasd. ****************************** #### WSI # Nexrad data # ****************************** The current release of McIDAS-X has an ADDE server for both NIDS and WSI NOWrad (tm) data. It can use the raw files the same way that GEMPAK/GARP can, so there is no need to turn those products into AREA files. ****************************** #### # NLDN decode actions # # # nldn format: YYJJJHHMbMe -- YY year, JJJ julian day, HH hour # Mb begin minute bin, Me end minute bin ****************************** The format of the NLDN broadcast product header as of January 1, 2000 is: # nldn format: CCYYJJJHHMbMe -- CCYY year, JJJ julian day, HH hour This means that the regular expression for the following NLDN actions will no longer be valid (they would match 9YJJJHHMbMe. If you want to process NLDN data, you should change the regular expressions from: ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) to: ^([0-9][0-9][0-9]|[0-9][0-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) This will match both YYJJJHHMbMe and CCYYJJJHHMbMe formats for the NLDN data products. Also, you will probably what to check the pathname for the McIDAS NLDN decoder as the one in the following lines does not follow what you have for lwtoa3 above. ****************************** # # Store 6 min bins with group parts # NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) FILE -close /data/nldn/\1\2\3\4\5.ingest # # Store 6 min bins verbatim NLDN (.*) FILE -close /data/nldn/\1.ingest # # Store transmitted, binary form as YYJJHH file NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) FILE -close /data/nldn/\1\2\3.ingest # # Decode transmitted binary form as GEMPAK ship file (named YYMMDDHH.gem) NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) PIPE /usr/local/ldm/decoders/bin/dcnldn -m 10000 -s minute10 -d /data/gempak/logs/dcnldn.log -p /unidata/home/gempak/gempak5.4/tables/pack/nldn.pack /data/gempak/nldn/YYMMDDHH.gem # # Decode transmitted, binary form as McIDAS MD file NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) PIPE -close /usr/local/ldm/util/bin/nldn2md -v -d /data/mcidasd 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CNN # ... ****************************** The next two NLDN actions appear to be repeats of ones above: ****************************** # Store transmitted, binary form as YYJJHH file NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) FILE -close /data/mcidasd/\1\2\3.ingest # Decode transmitted, binary form as McIDAS MD file NLDN ^(9[3-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9]) PIPE -close /usr/local/ldm/util/bin/nldn2md -v -d /data/mcidasd 70 NLDN DIALPROD=LD \1\2 \3\400 DEV=CCN ****************************** OK, the XCD stuff in pqact.conf looks good. There some mods needed as I indicate above, but these would not keep the XCD point source decoders from working. ****************************** data.mcidasd:: numerous .IDX files -rw-r--r-- 1 ldm apps 128 Jan 5 18:44 AVM00005.IDX ... numerous AREA files, but no composites. More on this later. -rw-rw-rw- 1 ldm apps 607776 Jan 6 07:34 AREA0060 ... .XCD files (these contain ALL of the DDPLUS|IDS data for a single day): -rw-rw-r-- 1 ldm apps 80072960 Jan 5 18:59 DD000050.XCD -rw-r--r-- 1 ldm apps 177427840 Jan 6 18:09 DD000060.XCD XCD rapid access files: -rw-rw-r-- 1 mcidas apps 142596 Jan 5 09:39 FOUS14.RAP -rw-rw-r-- 1 mcidas apps 6617092 Jan 5 09:39 RAOB.RAP -rw-rw-r-- 1 mcidas apps 26088580 Jan 5 09:39 SAOMETAR.RAP -rw-rw-r-- 1 mcidas apps 6364676 Jan 5 09:39 SYNOPTIC.RAP -rw-rw-r-- 1 mcidas apps 4896388 Jan 6 18:09 TERMFCST.RAP -rw-rw-r-- 1 ldm apps 6346420 Jan 6 18:09 TERMFCST.RAT HRS spool file: -rw-rw-r-- 1 ldm apps 16781313 Jan 6 18:09 HRS.SPL ROUTE.SYS, SYSKEY.TAB, SCHEMA: -rw-rw-r-- 1 mcidas apps 14080 Jan 6 18:06 ROUTE.SYS -rw-rw-r-- 1 mcidas apps 432640 Jan 5 09:39 SCHEMA -rw-r--r-- 1 mcidas apps 24000 Jan 6 18:34 SYSKEY.TAB The *.IDX and *.XCD files look normal. The *.RAP files were created when you ran 'BATCH XCDDEC.BAT'. Each one of these should have a corresponding *.RAT file. The only one I see is the one for terminal forecasts. This goes along with you not decoding any point source data. The HRS.SPL file shows that model data is being written to the spool, but the lack of GRID files shows that the GRIB data monitor, DMGRID, is not decoding any of them into McIDAS data files. Where do we stand? It seems that there is a problem running the point source and grid XCD decoders (duh! :-). The question is what is the problem? To learn more, we both need to look at the contents of: o ~mcidas/workdata/XCD_START.LOG o ~mcidas/.mcenv o the output from 'DMAP LWPATH.NAM' (if this file is not in ~mcidas/workdata, we have a problem) o the contents of ~mcidas/workdata/LWPATH.NAM o the output from 'DMAP MD' o a directory listing of ~mcidas/workdata Tom