Re: [conduit] partial files (based on bytes inserted) for f003 and f006 (GFS 0p50 & 1p00)

• To: Paul Prestopnik <prestop@xxxxxxxx>
• Subject: Re: [conduit] partial files (based on bytes inserted) for f003 and f006 (GFS 0p50 & 1p00)
• From: David Ovens <ovens@xxxxxxxxxxxxxxxxxxxx>
• Date: Mon, 12 Aug 2019 10:55:30 -0700

Hello,

I can confirm what Paul has noticed.  I have noticed this ever since
the start of the FV3 GFS switchover.  My daily summaries show that the
received file sizes are always just a little bit smaller for 0p25,
0p50, and 1p00 resolution grids.

                         Received        Expected
File name                Bytes            Bytes
------------------------ ---------  ---------------------
gfs.t12z.pgrb2.0p25.f003  331571643 vs status = 332129957
gfs.t12z.pgrb2.0p25.f006  336373643 vs status = 337032238

gfs.t12z.pgrb2.0p50.f003   99179578 vs status =  99358388
gfs.t12z.pgrb2.0p50.f006  100337802 vs status = 100543868

gfs.t12z.pgrb2.1p00.f003   28352037 vs status =  28406326
gfs.t12z.pgrb2.1p00.f006   28598264 vs status =  28661206

I'm not sure what fields are missing from these forecast hours, but it
is consistent, day after day, run after run.  It doesn't affect any of
our operations, so I have not brought it up before now.

-- 
David Ovens              e-mail: ovens@xxxxxxxxxxxxxxxxxxxx
Research Meteorologist    phone: (206) 685-8108
Dept of Atm. Sciences      plan: Real-time MM5 forecasting for the
Box 351640                        Pacific Northwest
University of Washington          https://atmos.uw.edu/mm5rt
Seattle, WA  98195               Weather Graphics and Loops
                                  https://atmos.uw.edu/~ovens/loops




On Mon, Aug 12, 2019 at 10:45:37AM -0600, Paul Prestopnik wrote:
>  For only the 3 and 6 forecast hours, I am getting status files that never
> show fully inserted model files. For example:
> 
> $ more 20190812_i12_f003_GFS003.status
> > data/nccf/com/gfs/prod/gfs.20190812/12/gfs.t12z.pgrb2.1p00.f003 complete
> > (28406326 bytes) at Mon Aug 12 15:27:36 2019
> > Inserted *28352037 of 28406326*
> >
> 
> Is this the expected behavior?  I'm not sure when it started happening, but
> I think it was sometime in the last few months.  Prior to that we would get
> status files showing the entire model file was inserted.
> 
> Here are the relevant lines from my pqact:
> 
> > CONDUIT
> > ^.status.data/nccf/com/gfs/prod/gfs.([0-9]{8})/../gfs.t([0-9]{2})z.pgrb2.1p00.f([0-9]{3})
> >         FILE
> >  /var/autofs/mnt/rapdmg1/data/grib/tmp/GFS003/\1/\1_i\2_f\3_GFS003.status
> >
> > CONDUIT
> > ^.status.data/nccf/com/gfs/prod/gfs.([0-9]{8})/../gfs.t([0-9]{2})z.pgrb2.0p50.f([0-9]{3})
> >         FILE
> >  /var/autofs/mnt/rapdmg1/data/grib/tmp/GFS004/\1/\1_i\2_f\3_GFS004.status
> >
> 
> Thanks,
> Paul

> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> 
> 
> conduit mailing list
> conduit@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit: 
> https://www.unidata.ucar.edu/mailing_lists/ 

• References:
  • [conduit] partial files (based on bytes inserted) for f003 and f006 (GFS 0p50 & 1p00)
    • From: Paul Prestopnik