NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi all,
Thanks for the vigorous discussion about GRIB variable names and the
future release of the netCDF-java 4.3 library and TDS 4.3.
First, I want to assure everyone that the transition will be done in a
careful and controlled manner with continued input from and discussion
with the community and, of course, plenty of time for testing. The
CDM/TDS group has been discussing this for some time with the IDV group
and, more recently, with the McIDAS-V group. We will work with the
community to develop a solution to the problem and deploy it in a way
that minimizes impact to the users (more on the road forward below).
It seems clear that we all understand that variable names need to
change due to a number of changes in our understanding of the GRIB
format and the GRIB tables. We all, also, clearly want to minimize the
pain involved in this transition while still making sure we figure out
the underlying problem as best we can so we won't have to go through
anything like this again. Where the disagreement seems to reside is in
the form of the variable names (opaque or human readable) and how that
will impact users. I'm going to try to summarize the pros and cons:
- Opaque variable names:
- Pro: Names are very stable and accurately represent
the GRIB variable.
- Con: All IDV bundles and other scripts that access GRIB
data by variable name will break.
- Con: Users will need to be shown the netCDF variable's
"long_name" attribute to interpret the meaning of the
variable.
- Human readable variable names:
- Pro: Users can interpret the meaning of the variable from
the netCDF variable name.
- Pros: Not all IDV bundles and other scripts that access
GRIB data by name will break (but some will since some
variable names are currently wrong).
- Con: The names are not stable and are fragile. As GRIB
tables evolve (versioned and un-versioned changes)
either variable names will change over time (not stable)
OR effort will need to be spent to track and minimize
visible table changes (fragile, prone to human error).
Either way, this is an opportunity for us to more deeply understand the
interoperability problems we have with GRIB files (and more generally).
It is also an opportunity to improve the flexibility of our systems and
the robustness of these systems in the face of change.
The Road Forward:
After continued community feedback and discussion and discussion with
Unidata's Users Committee, we will:
1) Finalize on a GRIB variable naming scheme along with the set of
variable attributes the GRIB to netCDF mapping will include (both CF and
GRIB specific attributes).
2) Develop a GRIB variable name mapping API that client applications can
use to map old variable names to new variable names and vice versa.
3) Create CDM and TDS 4.3 release candidates.
4) Install a test TDS 4.3 server on motherlode to support testing and
evaluation in parallel with the current TDS 4.2 installation. Work with
other groups (like NCDC -- Thanks Glenn!) to setup test TDS 4.3 servers
at their sites.
5) Work with the community and our committees to agree on a time table
for the parallel testing and evaluation mentioned in 4 above.
6) Iterate through steps 1-5 above, as necessary.
7) At the agreed upon time, switch the main TDS on motherlode to 4.3.
Work with the community during the transition of other servers.
Thanks,
Ethan (and all of us here at Unidata)
PS I want to chime in with others with a BIG thanks to John Caron for
immersing himself in the GRIB waters. He has spent a lot of time trying
to tease apart the tangled web of GRIB tables and develop a long term
solution. Thanks John!
--
Ethan Davis UCAR Unidata Program
edavis@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu
thredds archives: