Re: forecast aggregations

• To: "Steven.Anthony" <Steven.Anthony@xxxxxxxx>
• Subject: Re: forecast aggregations
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Fri, 09 Mar 2007 03:42:18 +0000

Steven.Anthony wrote:

Hey John,

I'm just wondering if you have had any luck with this (its still driving me nuts ;). Thanks for the help.

Thanks,
- Steven

John Caron said the following on 2/22/2007 1:21 PM:

Hi Steven:

Can you send me a directory listing and the resulting FMRC catalog?

If the server is running and I can access, just send me the catalog URL

thanks

Glenn.Rutledge wrote:

The following from Steve Anthony- Glenn

Hey,

Our problems with the FMRC Aggregation are from a date issue. It appears that when the data is read, the dates are getting thrown off by an offset.

In the catalog.xml file, we have copied (almost verbatim) the syntax that UCAR uses for its Motherlode server.

from catalog.xml:

<datasetFmrc name="gfs-test-FMRC" collectionType="ForecastModelRuns" harvest="true" path="fmrc/gfs_3"> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"; enhance="true"> <aggregation dimName="run" type="forecastModelRunSingleCollection" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" timeUnitsChange="true" > <scanFmrc location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129"

       regExp=".*\.grb$"
       runDateMatcher="#gfs_3_#yyyyMMdd_HH"
       forecastOffsetMatcher="HHH#.grb#"/>
         </aggregation>
</netcdf>

<fmrcInventory location="/nomads3_data/raid2/noaaport/merged/gfs-hi/200611/20061129" suffix=".grb" fmrcDefinition="NCEP-GFS-Global_onedeg.fmrcDefinition.xml" />

</datasetFmrc>

When I open up the "Forecast Model Run" folder in the FMRC dataset, I see 4 links (like I should) but the run dates are horribly off, as you can see below, the run dates are jumping by almost a month! - these should only be 6 hours apart.

gfs-test-FMRC_RUN_2007-02-12T00:00:00Z
gfs-test-FMRC_RUN_2007-01-18T00:00:00Z
gfs-test-FMRC_RUN_2006-12-24T00:00:00Z
gfs-test-FMRC_RUN_2006-11-29T00:00:00Z

The "Constant Forecast Offset" folder appears as it should, forecast offsets out to 180 hours, and at a 3 hour interval.

The "Constant Forecast Date" folder, however, also has problems, but they are a combination of the 2 previous ones.

gfs-test-FMRC_ConstantForecast_2006-11-29T00:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-11-29T06:00:00Z
...

gfs-test-FMRC_ConstantForecast_2006-12-06T12:00:00Z << looks normal up to this point gfs-test-FMRC_ConstantForecast_2006-12-24T00:00:00Z << then it jumps 18 days, and does this 2 more times in the dataset.

gfs-test-FMRC_ConstantForecast_2006-12-24T03:00:00Z
gfs-test-FMRC_ConstantForecast_2006-12-24T06:00:00Z
...

Questions:

Is my syntax wrong in the catalog.xml file? (we have one grib1 file per forecast hour per model run) What is in the NCEP-GFS-Global_onedeg.fmrcDefinition.xml file, and where is it?

Is there a way to customize them to possibly fix this?
Has anyone else experienced issues with the this FMRC capability.

if you need any more information, feel free to ask.

As for the ROTFS, we currently don't host it on our TDS server, but we can get some off of our nomads6 machine and put it up on the TDS for you.

Thanks a lot for your help,
- Steven Anthony


Steve Hankin wrote the following on 2/22/2007 12:12 AM:

Hi Glenn,

I'm at the OPeNDP workshop with John Caron this week -- an opportunity to discuss the problems with the forecast aggregation server.

Would you mind describing the nature (and details) of the problem(s) that you encountered with the configuring the forecast aggregation server for the RTOFS GRIB files?

   - Steve

P.S. John -- the OPeNDAP URL to the GDS server directory is http://nomads6.ncdc.noaa.gov:9090/dods/rtofs I guess the THREDDS catalog isat http://nomads6.ncdc.noaa.gov:9090/dods/rtofs.thredds.

--
Steven Anthony
Computer Systems Analyst
NOMADS Project
STG Inc. Federal Contractor
Veach-Baley Federal Building
151 Patton Avenue
Asheville, NC 28801-5001
Phone: 828.271.4474

http://nomads.ncdc.noaa.gov

Hi Steven: