NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Reinoud-John and I discussed this offline and the problem that the IDV (and maybe ToolsUI) is having is that some of the fields end up as netCDF DataTypes (Sequences and Structures (IDV variables seq1 and seq2) that the IDV was assuming were just plain numbers. John's going to look at whether this is the correct way to handle these, but in the mean time, the nightly build of the IDV should handle these BUFR files now. I just checked in another fix to make it even more robust, so you might want to wait until tomorrow's nightly build. Let us know if you find more problems with these and/or other BURF files that are problematic.
Don On 11/12/10 1:54 AM, Reinoud Bokhorst wrote:
Hi John, Tell me about the complexity of BUFR... The last two files were meteorological SYNOP files coded according to the WMO standard tables for FM94 BUFR Edition 4: http://www.wmo.int/pages/prog/www/WMOCodes/TDCFtables.html It uses the common sequence 3-07-080 (= Sequence for representation of synoptic reports from a fixed land station suitable for SYNOP data) from table D. I also use a ECMWF bufr decoder package (http://www.ecmwf.int/products/data/software/bufr.html). Attached is 'decode_bufr.txt' which is a dump of test1.bufr using the decode_bufr program in this package. The output compares well with what I see in toolsUI 4.2. However I now discovered when I press the 'Detail Info' button in toolsUI, I get a similar conversion error as reported for IDV (see attached). The reason why I am interested in this data is that the WMO GTS system (for SYNOP, TEMP, SHIP, BUOY, etc) is migrating from the alphanumeric code formats to the binary formats (http://www.wmo.int/pages/prog/www/WMOCodes.html). The deadline was due this month but I believe it has been postponed a while. Regards, Reinoud idvusers-request@xxxxxxxxxxxxxxxx wrote:---------------------------------------------------------------------- Message: 1 Date: Wed, 10 Nov 2010 16:27:09 -0700 From: John Caron <caron@xxxxxxxxxxxxxxxx> To: idvusers@xxxxxxxxxxxxxxxx Subject: Re: [idvusers] BUFR support in IDV Message-ID: <4CDB2A4D.7050808@xxxxxxxxxxxxxxxx> Content-Type: text/plain; charset=ISO-8859-1; format=flowed HI all: Apparently the BUFR reading is ok, and it reads ok into the FeatureType adapters, buts its not clear if IDV is using the new adapter. Reinoud: We can read well-formed BUFR messages into the CDM, but figuring out what they mean in order to get them geolocated and visualized in the IDV is a hard problem. BUFR is a particularly complex format where providers have too many choices on how to do the encoding, with little or no guidance or best practices AFAICT. The original example you sent (MSSARG_00217064.bufr) is good. The latest two are, um, difficult, and the possibility of us being able to understand arbitrary BUFR messages like this is not good. Perhaps if you could track down any documentation on that "template", we will see what we can do. John On 11/9/2010 8:36 AM, Don Murray wrote:Hi Reinoud- Thanks for trying this out. The BUFR support in the IDV has not really been tested. It relies on the netCDF-Java package to do the reading, so by default, some things "just work", while others (as you found) don't. Please submit this to support-idv@xxxxxxxxxxxxxxxx and the developers can take a look. It'll probably be passed on to the netCDF-Java support people for action. Don On 11/9/10 2:16 AM, Reinoud Bokhorst wrote:Hi Don, After a short break I have started looking into this again. Attached two surface SYNOP files in BUFR that load fine in toolsUI 4.2 but not in IDV 2.9u2. I can open the files in IDV as "Netcdf /Gempak point data files" but when I try to view the data I get the following error: ucar.ma2.ForbiddenConversionException at ucar.ma2.ArrayStructure.getFloat(ArrayStructure.java:1022) at ucar.ma2.StructureDataW.getScalarFloat(StructureDataW.java:139) at ucar.ma2.StructureDataW.convertScalarFloat(StructureDataW.java:96) at ucar.unidata.data.point.PointObFactory.makePointObs(PointObFactory.java:1910) at ucar.unidata.data.point.NetcdfPointDataSource.makeObs(NetcdfPointDataSource.java:360) at ucar.unidata.data.point.NetcdfPointDataSource.makeObs(NetcdfPointDataSource.java:322) at ucar.unidata.data.point.PointDataSource.getDataInner(PointDataSource.java:1266) at ucar.unidata.data.DataSourceImpl.getData(DataSourceImpl.java:2238) at ucar.unidata.data.DirectDataChoice.getData(DirectDataChoice.java:332) at ucar.unidata.data.DataChoice.getData(DataChoice.java:637) at ucar.unidata.data.DataInstance.getData(DataInstance.java:243) at ucar.unidata.data.DataInstance.getData(DataInstance.java:207) at ucar.unidata.data.DataInstance.dataOk(DataInstance.java:295) at ucar.unidata.data.point.PointDataInstance.init(PointDataInstance.java:83) at ucar.unidata.data.point.PointDataInstance.<init>(PointDataInstance.java:69) at ucar.unidata.idv.control.ObsDisplayControl.doMakeDataInstance(ObsDisplayControl.java:795) at ucar.unidata.idv.control.DisplayControlImpl.initializeDataInstance(DisplayControlImpl.java:3084) at ucar.unidata.idv.control.DisplayControlImpl.setData(DisplayControlImpl.java:3066) at ucar.unidata.idv.control.StationModelControl.setData(StationModelControl.java:1459) at ucar.unidata.idv.control.StationModelControl.init(StationModelControl.java:463) at ucar.unidata.idv.control.DisplayControlImpl.init(DisplayControlImpl.java:1333) at ucar.unidata.idv.control.DisplayControlImpl.init(DisplayControlImpl.java:1034) at ucar.unidata.idv.ControlDescriptor.initControl(ControlDescriptor.java:986) at ucar.unidata.idv.ControlDescriptor$1.run(ControlDescriptor.java:911) at ucar.unidata.util.Misc$3.run(Misc.java:1090) Regards, Reinoud Don Murray wrote:Hi Reinoud- On 10/19/10 7:43 AM, Reinoud Bokhorst wrote:Thank you for your fast response and fix.No problem. It was my bug, I figured I should fix it and I had a few spare cycles. ;-)This particular example now works. But when I try loading other samples from GTS (Global Telecom System) data it doesn't.. I am a bit confused now about the level of support of BUFR in IDV (after your fix that made it work) but as I understand it, it is only partly because the CDM supports it (correct me if I am wrong ;). Do you want more sample data at this point? Or rather wait until BUFR is officially going to be supported?Can you load all these other files in the ToolsUI? If they conform to what the CDM thinks are point or station obs, then they should be readable in the IDV. Although, the CDM implementation might not be correct. The IDV won't be able to handle soundings (profiles) or trajectories which the CDM might handle. Sample files would help.HP, yes I'll probably send a note to the User's Committee, although by no means I am an expert on this field.Cheers, Don------------------------------ _______________________________________________ idvusers mailing list idvusers@xxxxxxxxxxxxxxxx For list information, to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ End of idvusers Digest, Vol 21, Issue 6 ***************************************_______________________________________________ idvusers mailing list idvusers@xxxxxxxxxxxxxxxx For list information, to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
-- Don Murray NOAA/ESRL/PSD and CIRES 303-497-3596 http://www.esrl.noaa.gov/psd/people/don.murray/
idvusers archives: