NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

20060316:IMPORTANT!!!!-NCEP Global Ensemble Output!!

NOTE: The nws-changes mailing list is no longer active. The list archives are made available for historical reasons.

The message included below was sent earlier on our communications lines. It is also posted to the NWS Data Management Change Notices web page at the following URL. 
http://weather.gov/datamgmt/notices.shtml 2006/05/30
DM 03-06.07 MAJOR PRODUCT CHANGES TO NCEP GLOBAL ENSEMBLE OUTPUT.

NOXX10 KWBC 162115
DATA MGT MESSAGE 03-06.07

TO  AWIPS/NOAAPORT USERS.. FAMILY OF SERVICES/FOS/SUBSCRIBERS...
   EMERGENCY MANAGERS WEATHER INFORMATION NETWORK /EMWIN/
   NATIONAL WEATHER SERVICE /NWS/ CUSTOMERS... WAFS USERS...
   GTS USERS... NWS FAX CHART USERS

FROM  RTH WASHINGTON DATA MANAGEMENT

EFFECTIVE DATE MAY 30 2006

MAJOR PRODUCT CHANGES TO NCEP GLOBAL ENSEMBLE OUTPUT -- PLEASE READ CAREFULLY
Starting with the 12 UTC cycle on 30 May 2006 NCEP Central Operations will implement changes to the global ensemble processing that will result in drastic changes to its products. These changes are 
necessary to allow us to:

1. Streamline our ensemble processing
2. Provide additional ensemble members at higher resolutions on public servers
3. Eliminate redundant global ensemble products on these servers.

These changes will impact products available on both the NCEP ftp server
(ftpprd.ncep.noaa.gov/pub/data/) and the NWS ftp server (tgftp.nws.noaa.gov/SL.us008001/ST.opnl/). 
Users of these products will need to modify their automated retrieval and processing 
routines so that

they can continue to receive and use these data.

The nature of these changes are as follows:

1.  Directory Structure - [Affects both the NCEP and NWS ftp servers]


   The directory tree of global ensemble data will be drastically changed to allow 
for better system
performance on the NCEP Central Computer System (CCS).  This change is being brought 
to the NCEP and

NWS ftp servers as it will help with both access times and directory listings.

   1A. On the NCEP ftp server the directories will change as follows:

   Current - ftpprd.ncep.noaa.gov/pub/data/nccf/com/mrf/prod/ens.{YYYYMMDD}


   New - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/{FILE_TYPE}

   where YYYYMMDD is the current date, CC is the cycle time (00, 06, 12, or 18), 
and FILE_TYPE is a
descriptor for the type of data files contained in the subdirectory.  Examples of 
FILE_TYPE that will

be on the NCEP ftp server include: pgrba, pgrbb, ensstat, and wafs.


   Users who use the standard pressure GRIB files (ens*.pgrbf*) files on the NCEP 
ftp server will be

able to access these data from one of the following locations:

   ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/ 
  or
   ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrbb/

   The exact location of the parameters you may need is given in item 3 below.

   Users of the ensemble data on WAFS grids will be able to access these at:

   ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/wafs/


   1B.  On the NWS ftp server (tgftp.nws.noaa.gov) the directories will change as 
follows:

   Current - 
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1/

   New - 
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.high/
       - 
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.low/

   The exact location of the parameters you may need is given in item 3 below.


2.  Six hourly output - [Affects both the NCEP and NWS ftp servers]


   The global ensemble data sets will begin to be available in 6-hour increments 
out through 384
hours.  Currently 6-hour increment data are only available to 84 hours, then data 
are available in

12-hour increments to 384 hours.



3.  Break Up of Pressure GRIB data sets (pgrb files) - [Affects both the NCEP and 
NWS ftp servers]

   To facilitate the removal of redundant data sets the individual member ensemble 
files are being
broken up into two files per time step.  The first file will contain all of the 
parameters currently
contained in the enspost format files (see Item 6 below) at one degree resolution 
out to 384 hours.
These are being termed the A files and this is an increase in their resolution for 
the 90 to 384 hour
time period.  The second set (the B files) will contain the rest of the parameters 
at one degree
resolution to 84 hours and 2.5 degree resolution from 90 to 384 hours.  This is the 
same resolution

that these parameters are currently made available.

   On the NCEP ftp server, files will change as follows:


   Current - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/mrf/prod/ens.{YYYYMMDD}/ensp1.t{CC}z.pgrbf24

   New - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/gep01.t{CC}z.pgrbaf24 
and
       - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrbb/gep01.t{CC}z.pgrbbf24

   On the NWS ftp server, the files will change as follows:


   Current - 
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1/fh.0024_pa.membrp0

1_tl.press_gr.onedeg

   New - For the "A" files:

tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.high/fh.0024_pa
.membrp01_tl.press_gr.onedeg

       - For the "B" files:

tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.low/fh.0024_pa.
membrp01_tl.press_gr.onedeg


   This change will allow for a subset of higher resolution (one degree) data to be 
available out to
180 hours, whereas today it is only available to 84 hours.  Parameter lists for the 
"A" and "B" files

are located at:

   http://www.nco.ncep.noaa.gov/pmb/changes/gefs_20060530_Alist   and
   http://www.nco.ncep.noaa.gov/pmb/changes/gefs_20060530_Blist



4.  File Names Changing due to new ensemble transform mechanism - [Affects both the 
NCEP and NWS ftp

servers]


   When this change is implemented, NCEP will be creating perturbations in its 
ensemble members via a
ensemble transform breeding method.  [For specific information about this change, 
please see
http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/ENS_IMP.html]  Under this method, there 
will no longer be
positive and negative pairs of ensemble members.  NCEP will therefore cease 
identifying its members as
either positive or negative and instead use positive numbers for member 
identification.  For example,

on the NCEP ftp server:


   Current - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/mrf/prod/ens.{YYYYMMDD}/ensn1.t{CC}z.pgrbf24

   New - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/gep01.t{CC}z.pgrbaf24 


and/or

       - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrbb/gep01.t{CC}z.pgrbbf24

   The names of these files will range from gep01 to gep{NN}, where NN is the 
number of ensemble
members.  A low resolution control run will be added for all cycles and will be 
named as gec00.t{CC}z.*.

   On the NWS ftp server, the names will change as follows:

   Current -
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1/fh.0024_pa.membrn01_
tl.press_gr.onedeg

   New - For the "A" files:


tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.high/fh.0024_pa
.membrp01_tl.press_gr.onedeg

             - For the "B" files

tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.low/fh.0024_pa.
membrp01_tl.press_gr.onedeg

   The names of these new files will range from fh.{FFFF}_pa.memberp01.* to

fh.{FFFF}_pa.memberp{NN}.*, where NN is the number of ensemble members.  The control 
run for all

cycles will be named as fh.{FFFF}_pa.memberc00.*.


5.  Renaming and restructuring Ensemble mean files (enstat.*) - [Affects NCEP ftp 
server only]

   The ensstat.* files on the NCEP ftp server will no longer be available.  These 
grids contain
ensemble mean and spread information for a single parameter for all ensemble members 
and all forecast
hours.  They will instead be replaced by files that contain multiple parameters for 
a single ensemble
member and a single forecast hour.  With this configuration the creation of these 
data will be made
while the ensemble members are running on the NCEP CCS.  Currently these are not 
available until the

end of the ensemble run which creates a large amount of data to download at 
once.

   The current data sets are named as follows:


ftpprd.ncep.noaa.gov/pub/data/nccf/com/mrf/prod/ens.{YYYYMMDD}/ensstat.t{CC}z.{Parameter-name}

   The new data sets will be named as follows:


ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/geavg.t{CC}z.pgrbaf{FFF} 


and

ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/gespr.t{CC}z.pgrbaf{FFF}



6.  Removal of enspost, and ensemble Data Files - [Affects both the NCEP and NWS ftp 
servers]

   To reduce product redundancy and to allow for the additional data sets outlined 
in items number 2
and 3 above, the enspost.* (fh.xxxx.* files at NWS ftp) and ensemble.* data sets 
will no longer be
produced.  These data sets contain redundant information that is already available 
in the pressure
GRIB data sets.  They are being removed and users should instead use the "A" files 
located on the NCEP

ftp server at:

ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/ge{NNN}.t{CC}z.pgrbaf{FFF}

   (where NNN is the member name, c00, p01-p10) and on the NWS ftp server at:

tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1_RE.high/fh.{FFFF}_
pa.membr{NNN}_tl.press_gr.onedeg


   Even though these new "A" grids contain the same information as the enspost.* 
(fh.xxxx.* files at
NWS ftp) and ensemble.* data sets, they are organized differently.  The existing 
files are created at
the end of the ensemble processing and each file contains information for a single 
parameter for all
ensemble members and all forecast hours.  The new "A" files will be made available 
throughout the
production time of the ensemble processing on the NCEP CCS and they will each 
contain multiple

parameters for a single ensemble member and a single forecast hour.


   There is one exception to this change.  The 24-hour bias corrected ensemble 
products created at 00
UTC and available only the NCEP ftp server, will continue to be made available in 
the following new

location:



ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/ensstat/ensstat.pqpf_24h 
and

ensstat.pqpf_24hbc


7.  Corrected ensemble extensions in GRIB1 PDS - [Affects both the NCEP and NWS ftp 
servers]

   The current global ensemble data sets do not properly use the extended Product 
Definition Section
(PDS) of GRIB1.  When this change is implemented the GRIB1 data will be correctly 
labeled as ensemble
data sets.  This information is given in PDS octets 41 and higher.  A description of 
NCEP's use of

these octets is given at:

   http://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html

8.  Removal of all GRIB index files - [Affects both the NCEP and NWS ftp 
servers]


   The GRIB index files are no longer required by NCEP production and are being 
removed from the ftp

servers.  These files are currently named as follows:


   NCEP - 
ftpprd.ncep.noaa.gov/pub/data/nccf/com/mrf/prod/ens.{YYYYMMDD}/ens{NN}.t{CC}z.pgrbif24

   NWS -
tgftp.nws.noaa.gov/SL.us008001/ST.opnl/MT.ensg_CY.{CC}/RD.{YYYYMMDD}/PT.grid_DF.gr1/fh.{FFFF}i_pa.memb
r{NNN}_tl.press_gr.{XXX}deg


   If users still need GRIB index files, they can be created with NCEP software 
located at:

   http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/util/sorc/grbindex.fd/


9.  Removal of the 12 UTC "e2" Member -- Addition of Low Resolution Control Run for 
all Four Cycles

   With this implementation all four cycles per day will have a low resolution 
control run.  The need
for the "e2" member at 12 UTC will go away at that time.  The NCEP CCS resources 
that were used to run
the "e2" member will be used for the low resolution control member.  The low 
resolution control member

data will be found at:


ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrba/gec00.t{CC}z.pgrbaf{FFF} 


and

ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.{YYYYMMDD}/{CC}/pgrbb/gec00.t{CC}z.pgrbbf{FFF}


10.  6-hour Accumulations for Precipitation and Maximum and Minimum Temperature


   In the current global ensemble system accumulation variables alternate between 
6-hour and 12-hour
amounts depending upon the forecast hour.  With this upgrade all accumulation 
amounts will be based on

a 6-hour period.

TEST DATA:


At this time we are not able to provide parallel global ensemble data sets for user 
to compare the old
versus new.  We are however able to provide a static data set.  These data can be 
viewed at:

ftpprd.ncep.noaa.gov/pub/data1/nccf/com/gens/prod/gefs.20060306/

note that these static data are located under /pub/data1 not /pub/data.


It is anticipated that we will be able to establish a real time feed of the parallel 
data sets by the
end of March 2006.  Once this parallel feed of the new ensemble data sets is 
available, a new message

will be sent out via this mechanism.

Please direct any questions on these changes to:

NCEP.List.PMB-PCSP@xxxxxxxx

FOR FOS/NOAAPORT AND NON-AWIPS CUSTOMERS - IF YOU  HAVE ANY
QUESTIONS REGARDING ROUTING OR WOULD LIKE TO ADD THESE
PRODUCTS TO YOUR DATASTREAM PLEASE CONTACT.

Data Management
EMail   nws.dm@xxxxxxxx

THIS INFORMATION IS ALSO POSTED ON THE NWS DATA MANAGEMENT CHANGE

NOTICES WEB PAGE. THE URL IS LISTED BELOW.

http://weather.gov/datamgmt/index.html DATA MANAGEMENT
TELECOMMUNICATION OPERATIONS CENTER
RTH WASHINGTON SENDS
--
The NOAA's National Weather Service has a Web Based Customer
Relationship Management System that responds to customers'
needs for information.  The URL address for our CRM System
is listed below.

http://help.nws.noaa.gov/cgi-bin/doit.cgi?201~-506,,,,1:467:1








   
  
   





   
 
   

    
    
     
  • 
      2006 messages navigation, sorted by:
    
      
      
       
    1. Thread
      2. 
       
    2. Subject
      3. 
       
    3. Author
      4. 
       
    4. Date
      5. 
       
    5.  ↑ Table Of Contents 
      6. 
      
     
     
    • 
     
  • 
      Search the nws-changes archives:
    
      
    
         
       
      
    
      
     
    •