McIDAS, ldm-mcidas, GOES Ingest and NOAAPort Status Report
Winter 2007/2008

Tom Yoksas
April 2, 2008

Executive Overview

SSEC concerns about "illegal" use of McIDAS prompted a restatement of who Unidata is licensed to distribute its version of McIDAS to
Unidata McIDAS-X v2007 was released on August 16, 2007
ldm-mcidas v2006/7 undergoing revisions before release
Unidata-ingested GOES-10 imagery has been included in Unidata-Wisconsin GOES-East composite image sectors since January 31, 2007
Unidata NOAAPort ingest package continues as a reliable source for IDD HDS, IDS|DDPLUS, NGRID, NIMAGE, and NNEXRAD data

Vision

Efforts referenced within this report are based on five of the six endeavors articulated in the Unidata 2008 Proposal:

Endeavor 1: Responding to a broader and more diverse community
Endeavor 2: Comprehensive Support Services
Endeavor 4: Software to analyze and visualize geoscience data
Endeavor 5: Distributed, organized collections of digital material
Endeavor 6: Improved data access infrastructure

Unidata McIDAS-X/-XCD Distribution License (Yoksas)

Concerns about distribution of Unidata McIDAS-X to institutions outside of the United States (e.g., CPTEC/INPE in Brazil and CONAE in Argentian) resulted in a letter a letter of clarification for distribution of Unidata McIDAS-X to better identify who may be allowed to legally download Unidata's distribution of McIDAS-X. The letter reaffirms Unidata's right to distribute McIDAS-X to the "international university community" while providing guidelines for which institutions may be regarded to be members of that community.

Unidata McIDAS-X, -XCD (Yoksas)

McIDAS-X/-XCD version 2007 were released by SSEC on May 25, 2007, and June 04, 2007, respectively. The Unidata McIDAS-X/-XCD v2007 distribution was released on August 16, 2007.
- The most significant new feature of v2007 is the support of NCEP-created GRIB/GRIB2 fields in the GRIB ADDE server. Speed of access to model fields using the GRIB filer approach (which uses MySQL) is greatly improved over the decode option.
  Solutions to problems that remain with decoding GRIB2 messages will be made available in an addendum as soon as possible.
- ADT was added by SSEC to v2007. ADT Estimates tropical cyclone intensity using the Advanced Dvorak Technique>
A McIDAS v2007 training workshop was held on August 7-10, 2007 in UPC offices in Boulder.

A special McIDAS training session for INPE/CPTEC (Brazil) is slated to be held from December 1-14 at the CPTEC training facility in Cachoeira Paulista. CPTEC is fully funding this training event.
As of August 16, 2007 22 sites downloaded Unidata McIDAS 2007. For a historical comparison, 102 sites downloaded version 2004; approx. 60 downloaded v2005; and approx. 50 downloaded v2006.
The majority of new interest in Unidata McIDAS is coming from organizations outside of the US.
Nine bugfix/enhancement updates have been released for v2007. Many modifications have been related to the new configuration capability in the package and modifications to better support 64-bit building under SunOS (x86_64 and SPARC).
Direct access of PNG-compressed AREA format images in the IDD UNIWISC (Unidata-Wisconsin) and GINI format Nexrad Level III composites in the IDD FNEXRAD datastreams initially added in addendum v2006f was operationalized in v2007.
Continuing support/bug fixes for Unidata and SSEC McIDAS-X, -XCD.
Internet2 bandwidth usage by McIDAS ADDE ranks as the second highest Advanced Application user of I2 bandwidth at approx. 0.5 TB/week. Notably the LDM ranks first in I2 bandwidth use for Advanced Applications.

ldm-mcidas Decoders (Yoksas)

A new, as yet unreleased, version, v2006, includes support for decoding of USPLN lightning data that could be made available from Embry Riddle University (active). Release of the new version awaits User Committee decisions on provision of USPLN data to Unidata community, and migration of McIDAS-specific decoders (i.e., nldn2md and proftomd) to the Unidata McIDAS distribution.
McIDAS-specific decoders (nldn2md and proftomd) are being moved from the ldm-mcidas package into the Unidata McIDAS package. After the move, the ldm-mcidas package will only contain decoders for the compressed imagery being distributed in the IDD (Zlib-compressed NOAAPort GINI imagery; PNG-compressed Nexrad Level III composites in GINI format; and PNG-compressed Unidata-Wisconsin imagery in AREA format).

GOES Ingest Capabilities (Yoksas, Schmidt)

During the Fall 2006 Policy Committee meeting, a response to the call for internal proposals for the UOP Director's STORM fund was developed. The proposal, representing a collaboration among Unidata, and NCAR's Earth Observing and Research Applications Laboratories, requested funding for the following activities:
- Purchase of equipment needed to ingest satellite imagery from the GOES-10 spacecraft using an existing satellite dish installed at the Mesa Lab
- Purchase of new equipment, and upgrades to leverage existing equipment to ingest GOES-East/West imagery using the satellite dishes installed at FL-2
- Provision of real-time, programmatic access to the data ingested from all three GOES platforms using Unidata THREDDS technologies
In October, 2006, UCAR submitted this proposal to NESDIS which funded it for the full requested amount of $30K.
On the evening of January 30, Unidata successfully began ingesting GOES-10 (referred to internally as GOES-South) imagery using equipment that had been surplused by various groups in NCAR/UCAR and the Unidata community (e.g., retired USAN satellite dish from NCAR/CISL; TeraScan (tm) satellite receiver (SeaSpace Inc.) from NCAR/RAL; PC satellite ingest card from the Weather Underground, Inc., etc.), and McIDAS SDI software licensed from the Space Science and Engineering Center (SSEC) of the University of Wisconsin-Madison.
On Wednesday, January 31 GOES-South imager sectors were integrated into the Unidata-Wisconsin datastream (IDD feedtype UNIWISC) for the benefit of the US and increasingly international Unidata community, especially university participants in Latin America. Any system participating in the IDD has access to the data (currently over 400 systems at over 210 sites). GOES-South imagery is now accessible to any ADDE-enabled application (e.g., McIDAS, IDV, etc.) with Internet access.
The establishment of GOES-10 ingest capabilities in UCAR was a large portion of Rick Anthes President's Corner article in the March 2007 UCAR newsletter.
SDI interface cards donated to the UPC by the LSU/SRCC and St. Louis University will be used to establish a GOES-East and GOES-West ingest and ADDE serving capability in UCAR and as spare parts (active, Schmidt, Yoksas).
Activities:
March-May, 2007:
- recertify the satellite dish de-ice subsystem (done)
- advertise the availability via McIDAS ADDE of real-time GOES-South data to the Unidata community (done)
- incorporate real-time GOES-South, GOES-East, and GOES-West data into THREDDS and ADDE server installations on motherlode.ucar.edu (in progress)
Summer-Fall, 2007:
- new LNBs were installed on the GOES-East and GOES-West dishes located at the front of FL-2. The LNBs were purchased by RAL and EOL as part of their contribution to the STORM project.
- move GOES-South ingest to satellite dish located between FL-1 and FL-2 that was given to Unidata by RAL for this activity. This dish is newer and easier to motorize than the Mesa USAN dish that was originally used for GOES-South ingest. The Mesa dish will be reporposed for GOES-East or GOES-West ingest.
Fall-Winter, 2007/2008:
- motorize GOES-South ingest dish to track GOES-10 as its orbit degrades (March, 2008)
- tap signal and data off of existing NCAR/RAL TeraScan GOES-East and GOES West reception systems (using equipment surplused by UCAR/JOSS; LSU/SRCC; and St. Louis University), and ingest the data directly into formats that can be accessed by McIDAS ADDE (GOES-East ingest began on June 22; GOES-West ingest is set to begin with the next month).
  Ingest of GOES-East imagery has been accomplished and is routinely available via ADDE. Ingest of GOES-West imagery has been tested.
- archive GOES-South, GOES-East, and GOES-West data on the NCAR MassStore (preliminary planning)
- work with NCAR/EOL to make the GOES-South data available in TeraScan TD format (upcoming)
Full resolution GOES-East/West imagery is also available via ADDE from a Unidata-owned, toplevel IDD relay node housed in the SSEC Data Center. This same machine hosts a rolling multi-month, realtime archive of images broadcast in the IDD Unidata-Wisconsin datastream.

NOAAPort Ingest (Chiswell, Schmidt, Emmerson, Yoksas, Weber)

The Unidata-developed DVB-S NOAAPort ingest system as been In use at the UPC and several (7 operational + 2 in progress) Unidata sites since the NWS transition to DVB-S broadcast. At least 4 non-Unidata sites are known to be using the ingest system, but no effort is spent in tracking its use.

SSEC provides and supports the Unidata DVB-S NOAAport ingest system for its licensed users of McIDAS-XCD.
User Support occasionally gets questions on the NOAAPort ingest software package, but the support burden caused by the inquiries is insignificant.
The NOAAPort ingest system installed at the Universidad de Costa Rica in San Jose, Costa Rica continues to function as designed. Problems, however, regularly occur as oxidation builds up in a N-connector coupling that was added by the company that re-aimed the satellite for NOAAPort use. After the oxidation is removed by UCR personnel, data ingestion returns to normal. Notably, UCR will be connecting to Internet2 in the not too distant future. This augmentation of internet connectivity will allow for the receipt of high resolution model data (e.g., CONDUIT data) that will be used in local modeling efforts. It will also allow the UCR to take a leading role in the provision of real-time data in Central America through the establishment of the IDD-Caribe.
This NOAAPort reception system installation is a high profile undertaking at the UCR, so its continued success remains important for Unidata. (Yoksas)

by Tom Yoksas.