NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
>From: Karthik Samudram Jayaraman <address@hidden> >Organization: Western Michigan University >Keywords: 200402021728.i12HSip1008370 IDD CRAFT Karthik, >This is Karthik From Western Michigan University. We need the NEXRAD >Level II data to be ingested to our server S400.wood.wmich.edu, >which currently is not been ingested. > >Could you please guide me in how to proceed with the same? There are three things involved in getting any data through the IDD: 1) getting access to the appropriate datastream and then setting up the request line in the LDM ldmd.conf file (located in /cutrim1/ldm/etc/ldmd.conf on s400) 2) decoding the data into a directory/directories where there is enough room 3) scouring the data so that your disk does not fill up Here is the information you need for the NEXRAD Level II data: 1) you can request the Level II feed from thelma.ucar.edu: LDM upsteam server: thelma.ucar.edu IDD feedname: CRAFT You should decide which radars you want the Level II data from, because there is a LOT of data in the CRAFT feed. I setup the /cutrim1/ldm/etc/ldmd.conf request line for the radars in Michigan (GRR, APX, DTX) and those immediately to the south (IWX and LOT) and west (MKX, GRB, MQT). 2) the Level II data comes in chunks of 100 radials each of which are compressed using bzip2. The simpliest way to decode these is to use the GEMPAK decoder, dcnexr2. Since decoding these data puts a high demand on a systems CPU, we found that it is best to run this decoder from its own ldmd.conf action. For s400, the action would look like: exec "pqact -f CRAFT /cutrim1/ldm/etc/pqact.conf_craft" I setup this action in the /cutrim1/ldm/etc/ldmd.conf file. The contents of pqact.conf_craft would look like: # CRAFT CRAFT ^L2-BZIP2/(....)/([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])([0-2][0-9][0-5][0-9])([0-9][0-9]) PIPE decoders/dcnexr2 -s \1 -d /dev/null /cutrim1/ldm/data/craft/\1/\1_\2_\3 WARNING: white space in pqact.conf actions may be tabs, not spaces. In this example, tabs are used: - between CRAFT and ^L2-BZIP2 - before PIPE - between PIPE and decoders/dcnexr2 - before -s \1... IMPORTANT: No line in pqact.conf* should begin with a space. I setup /cutrim1/ldm/etc/pqact.conf_craft for you. 3) we scour Level II NEXRAD data using the LDM 'scour' utility run from cron. To do this, 'scour' should be run at least once per day. Here is an example that runs 'scour' multiple times per day: # # Scour data directory 0 1,4,7,10,13,16,19,22 * * * bin/ldmadmin scour > /dev/null 2>&1 Scouring was already setup on s400 exactly as in this example. To configure 'scour' (which is run by 'bin/ldmadmin scour'), one must setup entries in the etc/scour.conf file (located in /cutrim1/ldm/etc/scour.conf on s400). Here is the appropriate entry for scouring Level II data being saved in the /cutrim1/ldm/data/craft directory hierarchy: /cutrim1/ldm/data/craft 1 I setup this entry so that the Level II data should get scoured assuming that the cron jobs contiue to run. >Thanks You must keep an eye on the ingest/decode/scour setup until you are certain that it is working correctly. Otherwise, you could easily fill your disk which would result in you no longer ingesting and decoding data. Tom -- NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publically available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.