NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Daryl, Afraid the company lawyer would fire me and kill me, not necessarily in that order. However, here are a couple that I'm sure are very much "public" knowledge... deleting NIL= lines changing OOOOOKT (capital oh's) to 00000KT (zeros) changing 0/0KT and 000/00KT to 00000KT (zeros) changing G//KT to KT replacing ttXdd or tt-dd temp/dewpt groups with tt/dd replacing "<" with "/" (mostly Canadian SA format stations) I've noticed the "future obs" problem as well. That's one that's awaiting my attention... Steve On Wed, Aug 9, 2017 at 12:07 PM, daryl herzmann <akrherz@xxxxxxxxxxx> wrote: > Steve, > > In the vane of data-curation, would you mind sharing that list of 25 > things? :) > > daryl > > On Wed, 9 Aug 2017, Stephen Adams wrote: > > Greg, >> >> Ah... the joys of processing the metar "standard". I made a C program that >> makes corrections to certain obvious errors in the metars. Over the years >> I've been adding more and more "events". It's up to 25 now, with 2 more >> waiting for my attention... The international sites are the most error >> prone, but good ole USA sites are not immune. Quality control at data >> entry >> sounds like a wonderful and logical idea. Therefore, according to Murphy's >> Law, it'll never happen! >> >> Steve >> >> >> On Wed, Aug 9, 2017 at 10:21 AM, Greg Thompson <gthompsn@xxxxxxxx> wrote: >> >> >>> Daryl, >>> >>> The number of *exceptions* for METARs, TAFs, and many other NWS products >>> is insane. My software will ingest the incorrect MR0C just fine, and my >>> program that seeks out unknown 4-letter IDs found in the METAR DB I >>> maintain but are not known lat/lon will reveals many sites that were >>> typos. >>> >>> When will someone ever employ software checking upon data entry to >>> prevent >>> things like this? Never at the rate we're going. The worse condition in >>> my opinion is when the time stamp is completely wrong - such as >>> POST-dated >>> METAR data. It happens all the time and they merrily end up in final >>> files >>> as supposedly good info. >>> >>> --Greg >>> >>> >>> On Tue, Aug 8, 2017 at 10:14 PM, daryl herzmann <akrherz@xxxxxxxxxxx> >>> wrote: >>> >>> >>>> Oh the fun of processing METARs. Behold the METAR site MROC Juan >>>> Santamaria, Costa Rica. Here's the 0z ob >>>> >>>> SACS31 MROC 090000 >>>> METAR >>>> MROC 090000Z 25005KT 5000 BR FEW005 SCT015 23/23 A2993 NOSIG= >>>> >>>> and here's the 1z ob: >>>> >>>> SACS31 MROC 090100 >>>> METAR >>>> MR0C 090100Z 24005KT 3000 BR FEW002 BKN010 23/22 A3000 NOSIG= >>>> >>>> Notice what's wrong? :) So the Oh became a zero! So the KWBC >>>> collective >>>> then missed it. >>>> >>>> SACA35 KWBC 090100 >>>> METAR >>>> ...snipped... >>>> MROC NIL= >>>> >>>> So should I account for this in my METAR ingesting code somehow or alas, >>>> the 1z ob on 9 August 2017 is lost forever for MROC :( >>>> >>>> daryl >>>> >>>> -- >>>> /** >>>> * daryl herzmann >>>> * Systems Analyst III -- Iowa Environmental Mesonet >>>> * https://mesonet.agron.iastate.edu >>>> */ >>>> >>>> _______________________________________________ >>>> NOTE: All exchanges posted to Unidata maintained email lists are >>>> recorded in the Unidata inquiry tracking system and made publicly >>>> available through the web. Users who post to any of the lists we >>>> maintain are reminded to remove any personal information that they >>>> do not want to be made public. >>>> >>>> Data-curation mailing list >>>> Data-curation@xxxxxxxxxxxxxxxx >>>> For list information, to unsubscribe, or change your membership options, >>>> visit: http://www.unidata.ucar.edu/mailing_lists/ >>>> >>>> >>> >>> _______________________________________________ >>> NOTE: All exchanges posted to Unidata maintained email lists are >>> recorded in the Unidata inquiry tracking system and made publicly >>> available through the web. Users who post to any of the lists we >>> maintain are reminded to remove any personal information that they >>> do not want to be made public. >>> >>> Data-curation mailing list >>> Data-curation@xxxxxxxxxxxxxxxx >>> For list information, to unsubscribe, or change your membership options, >>> visit: http://www.unidata.ucar.edu/mailing_lists/ >>> >>> >>> >> > -- > /** > * daryl herzmann > * Systems Analyst III -- Iowa Environmental Mesonet > * https://mesonet.agron.iastate.edu > */ >
data-curation archives: