Hi Jeff:
Thanks so much for taking the time to summarize these important events.
This seems to me to be very good news, namely 1) to clarify the existing
situation with regards to versions, and 2) to agree to a mechanism to
make canonical, machine readable tables available. I think this will
make BUFR a much more reliable format for exchange and archive.
Keeping separate table versions for the future, while more trouble, at
least is now manageable as long as we have a reliable archive of the
versions.
One issue that I am still wondering about, is in regards to "allow(ing)
incorrect descriptors to be corrected in future versions". For example,
apparently table B version 13 has 0-22-39 with a data width of 13, but
it was apparently 12 bits in previous versions, and is back to 12 in the
(preliminary) version 14. (Perhaps it was a typo when version 13
document was prepared?) Anyway, should i assume that a version 13 BUFR
message descriptor 0-22-39 uses 12 bits or 13 bits?
If version 13 tables are to be used also for previous versions, it seems
like the version 13 table should be corrected to use the correct
bitwidth of 12 for descriptor 0-22-39. Yet its possible that somebody
used bitwidth 13 to encode their data, and theres no way to know for
sure without knowing who wrote it and the software used.
Best Regards,
John
Jeff Ator wrote:
Everyone,
To follow-up on the below issue, the WMO codes group discussed this at
length during its annual meeting two weeks ago. After much discussion
and debate, it was agreed to allow scale factors, reference values
and/or bit widths (but *not* descriptor names nor units!) to change
concurrent with the introduction of a new version of the BUFR tables.
This policy will take effect beginning with the upcoming version #14.
The current version #13 will remain a superset of all previous editions,
so it will only practically be necessary for centers to maintain version
#13 and then all versions going forward from that point. In particular,
this will allow incorrect descriptors to be corrected in future versions
as well as allow deprecated descriptors to be removed altogether, since
such descriptors will remain valid for all time within the previous
table version(s) in which they existed. So all archived BUFR messages
will always remain valid and readable, but all users (not to mention
their encoding and decoding software!) will now need to pay careful
attention to which version of the tables they are working with at any
given time.
It was well understood and agreed at the meeting that, for this new
policy to succeed, the WMO secretariat will need to maintain official
copies of the current and all future tables available on its web site,
and in one or more formats which users can easily download and ingest
into their local software (currently only MSWord and PDF formats are
available). To that end, WMO has agreed to devote the necessary
resources towards this task, and they will be assisted for an initial
period of time by a small group of experts from the meeting, in sort of
an ad-hoc advisory role, in order to get this process started. WMO has
also agreed to set up an email list to notify users whenever such table
updates are made to the web site.
The exact policy and procedures will be spelled out in the final report
of the meeting (not yet available!), but at this point I wanted to give
everyone a "heads-up" that this is coming. In particular, one procedure
that was agreed upon was that any and all corrections to existing
descriptors within a new table version must be made simultaneously when
that table is first released for pre-operational use, in order to
prevent any confusion amongst users who may engage in pre-operational
dissemination of messages using that new table version before it
officially becomes operational.
Let me know if there are any questions, and I also invite Milan, Eva,
Stan and any other members of the codes group who happen to be on this
email list to add to or clarify anything I've said.
Best regards,
-Jeff
