-----Original Message-----
From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx]
Sent: 05 August 2008 20:03
To: Kellett, Stanley
Cc: Wright, Bruce; Jeff Ator; Ross, Gil; Michelle Mainelli; Scott
Jacobs; Paul Hamer; Chris Macdermaid; Robb Kambic
Subject: Re: table versions
Hi Stan:
Thanks so much for your reply. I hope the flies arent biting as hard as
in Wyoming where I was camping last week.
If new versions of the WMO master table can only add new elements, this
would be very good news. It would mean that one really only needs the
latest version of the table. Is there somewhere that is explicitly
stated in the WMO BUFR specification?
The worry that I have is that if a mistake is made in a published
version of a table, and is used to code a BUFR message, that message
will be incorrectly read by a decoder using the corrected version of the
table. Theres no way to unambiguously know what table the coder used.
We have been studying the differences between versions of the tables,
using the mel-bufr tables. While we are not sure of their exact
provenance, they seem to be widely used.
Below is a summary of the differences in scale,offset and/or bit width
among mel-bufr versions 2 through 13. The difference is always with the
current version (13); "B3M-000-003-B.diff" for example means it is the
difference with mel-bufr table version 3. For example the line
0-2-42
width 6 != 2 (B2M-000-002-B.diff)
means that the bit width for field 0-2-42 was 6 for version 2, and 2 for
version 13.
--------------------------------
0-2-42
width 6 != 2 (B2M-000-002-B.diff)
width 6 != 2 (B3M-000-003-B.diff)
width 6 != 2 (B3M-000-004-B.diff)
width 6 != 2 (B3M-000-005-B.diff)
0-2-91
scale 0 != 4 (B3M-000-008-B.diff)
refer 4 != 0 (B3M-000-008-B.diff)
scale 0 != 4 (B3M-000-009-B.diff)
refer 4 != 0 (B3M-000-009-B.diff)
scale 0 != 4 (B3M-000-010-B.diff)
refer 4 != 0 (B3M-000-010-B.diff)
0-2-140
scale 2 != 0 (B2M-000-002-B.diff)
scale 2 != 0 (B3M-000-003-B.diff)
scale 2 != 0 (B3M-000-004-B.diff)
scale 2 != 0 (B3M-000-005-B.diff)
0-2-151
width 4 != 11 (B3M-000-006-B.diff)
width 4 != 11 (B3M-000-007-B.diff)
width 4 != 11 (B3M-000-008-B.diff)
width 4 != 11 (B3M-000-009-B.diff)
0-7-9
refer 0 != -1000 (B3M-000-008-B.diff)
0-10-9
refer 0 != -1000 (B3M-000-008-B.diff)
0-11-52
width 14 != 13 (B3M-000-008-B.diff)
0-11-82
width 13 != 14 (B3M-000-008-B.diff)
0-13-60
refer -10 != -1 (B3M-000-008-B.diff)
refer -10 != -1 (B3M-000-009-B.diff)
refer -10 != -1 (B3M-000-010-B.diff)
0-21-85
refer -3000 != 0 (B2M-000-002-B.diff)
refer -3000 != 0 (B3M-000-003-B.diff)
refer -3000 != 0 (B3M-000-004-B.diff)
refer -3000 != 0 (B3M-000-005-B.diff)
refer -3000 != 0 (B3M-000-006-B.diff)
0-22-39
width 12 != 13 (B2M-000-002-B.diff)
width 12 != 13 (B3M-000-003-B.diff)
width 12 != 13 (B3M-000-004-B.diff)
width 12 != 13 (B3M-000-005-B.diff)
width 12 != 13 (B3M-000-006-B.diff)
width 12 != 13 (B3M-000-007-B.diff)
width 12 != 13 (B3M-000-008-B.diff)
width 12 != 13 (B3M-000-009-B.diff)
width 12 != 13 (B3M-000-010-B.diff)
width 12 != 13 (B3M-000-011-B.diff)
width 12 != 13 (B3M-000-012-B.diff)
0-24-1
scale -1 != -11 (B2M-000-002-B.diff)
scale -1 != -11 (B3M-000-003-B.diff)
scale -1 != -11 (B3M-000-004-B.diff)
scale -1 != -11 (B3M-000-005-B.diff)
0-24-2
scale -1 != -11 (B2M-000-002-B.diff)
scale -1 != -11 (B3M-000-003-B.diff)
scale -1 != -11 (B3M-000-004-B.diff)
scale -1 != -11 (B3M-000-005-B.diff)
0-25-50
scale 0 != 4 (B2M-000-002-B.diff)
refer 0 != -131072 (B2M-000-002-B.diff) width 7 != 18
(B2M-000-002-B.diff) scale 0 != 4 (B3M-000-003-B.diff) refer 0 !=
-131072 (B3M-000-003-B.diff) width 7 != 18 (B3M-000-003-B.diff) scale
0 != 4 (B3M-000-004-B.diff) refer 0 != -131072 (B3M-000-004-B.diff)
width 7 != 18 (B3M-000-004-B.diff) scale 0 != 4 (B3M-000-005-B.diff)
refer 0 != -131072 (B3M-000-005-B.diff) width 7 != 18
(B3M-000-005-B.diff) scale 0 != 4 (B3M-000-006-B.diff) refer 0 !=
-131072 (B3M-000-006-B.diff) width 7 != 18 (B3M-000-006-B.diff)
0-25-51
width 12 != 7 (B2M-000-002-B.diff)
width 12 != 7 (B3M-000-003-B.diff)
width 12 != 7 (B3M-000-004-B.diff)
width 12 != 7 (B3M-000-005-B.diff)
width 12 != 7 (B3M-000-006-B.diff)
0-29-2
width 2 != 3 (B2M-000-002-B.diff)
width 2 != 3 (B3M-000-003-B.diff)
width 2 != 3 (B3M-000-004-B.diff)
width 2 != 3 (B3M-000-005-B.diff)
If we believe these mel-bufr tables (and we havent made some mistake in
analysing them), scale/reference/width seems to be allowed to change.
For example, the last field, 0-29-2, bit width was changed from 2 to 3
at version 6. If so, then we have no choice but to maintain versioned
code tables, and assume that the version number (BUFR section 1 octet
11) is correct.
Of course the real question is, what values did the coders use? We have
some hope of automatically detecting when bit width is wrong by counting
bits in the message and comparing to the data size. If scale/offset is
wrong, it will take humans and/or more complex algorithms to detect
that.
It would be very helpful to hear what the actual practice is with tables
at the various generating centers. Or to hear that the mel-bufr tables
are wrong, or some other flaw in our conclusion.
Thanks for any thoughts,
John
Kellett, Stanley wrote:
John,
First of all I am sorry for late response as I have just gotten back
from a camp trip with my family.
No scale, width and reference values can't be changed once
operational.
What has probably happened is there was a mistake in the last
published tables which was corrected.
Stan
-----Original Message-----
From: John Caron [mailto:caron@xxxxxxxxxxxxxxxx]
Sent: 24 July 2008 18:20
To: Wright, Bruce; Jeff Ator; Ross, Gil; Michelle Mainelli; Scott
Jacobs; Kellett, Stanley; Paul Hamer; Chris Macdermaid; Robb Kambic
Subject: table versions
We are trying to understand how the different versions of the WMO
master table differ from each other. We are looking at the "mel-bufr"
tables, which seem to be the current best representations of the
different versions of the WMO master tables, although we are not sure
how they were generated.
We compare these tables against our version of the WMO master table
version 13, which we extracted from the Word doc. Typical are the
following difference of table 12 with table 13:
1) *** WARNING **** Discrepency
Scale Ref Width Units for key 0-13-81
3;0;14;S m-1 from B3M-000-012-B
3;0;14;Siemens m-1 from B4M-000-013-B
2) *** WARNING **** Discrepency
Scale Ref Width Units for key 0-15-11
3;14000;13;log from B3M-000-012-B
3;14000;13;log(1/m2) from B4M-000-013-B
3) *** WARNING **** Discrepency
Scale Ref Width Units for key 0-22-39
3;-5000;12;m from B3M-000-012-B
3;-5000;13;m from B4M-000-013-B
Difference 1) and 2) seems to be clarifications of the units. In 1) "S
m-1" has been changed to "Siemens m-1" and in 2) "log" was changed to
"log(1/m2)". It seems likely that those changes are clarifications or
corrections that should be used in version 12 BUFR messages.
Difference 3) is a change in bit width of field 0-22-39. Should this
be understood that version 12 messages use bit width 12, while version
13 uses bit width 13? Obviously this is a critical thing to know.
When we compare table 10 with 13, we see:
4) *** WARNING **** Discrepency
Scale Ref Width Units for key 0-2-91
0;4;10;a from B3M-000-010-B
4;0;10;a from B4M-000-013-B
5) *** WARNING **** Discrepency
Scale Ref Width Units for key 0-13-60
1;-10;17;kg m-2 from B3M-000-010-B
1;-1;17;kg m-2 from B4M-000-013-B
For difference 4) I might guess that the scale and reference were
mistakenly flipped, that is, it corrects a typo in version 10.
Possibly difference 5), which changes the reference value from -10 to
-1, corrects a typo also?
Are new table versions really allowed to change Scale,Ref,Width or
Units of existing table entries? Because if they are, we really have
to carefully track what changes with each version.
Are there archives of previous versions of WMO master tables? Are the
mel-bufr tables accurate?
Any thoughts or advice would be appreciated.