Jim,
> I'm confused about a few things. If you don't mind spending a bit of
> time clarifying, I'd appreciate it.
Oops, sorry, it looks like I'm the one who is confused!
> Here are my confused questions.
>
> * Isn't the _FillValue attribute the one that maps to '_'? (See the
> discussion of _FillValue and missing_value here
> <http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Attribute-Conv
> entions.html>.)
Yes, you're right of course. Sorry for mixing up "_FillValue" and
"missing_value". The latter was at one time deprecated, but now that
deprecation has been removed from the latest CF conventions standard.
> * Is the flag_values attribute considered applicable in the case of
> floating point variables?
That was my understanding. CF 1.5 says:
... The flag_values attribute is the same type as the variable to
which it is attached, and contains a list of the possible flag
values.
I can't see anywhere that restricts flag_values to integers. Of course
the "flag_masks" attribute wouldn't have much meaning for floating-point
values, but I wasn't suggesting using "flag_masks".
> * In what sense is this suggestion making a change to the
> interpretation of the missing_value attribute?
If "this suggestion" means extending missing_value to permit multiple
values, then the only change it makes is changing a one-to-one mapping
into a one-to-many mapping. Since my example of breaking ncdump/ncgen
backward compatibility doesn't apply to the "missing_value" attribute,
that may be harmless, unless there is other existing software that
depends on "missing_value" having a single value. So I withdraw my
confused objection to using "missing_value".
However, I still think "flag_values" would also work for this purpose,
especially if there were other reasons for special values than that they
indicated a special type of missing value. Also "missing_values" might
be more descriptive than "missing_value" for a multiple-value attribute,
following the example of "flag_values", but I'm also in favor of keeping
the additions of new CF attributes to a minimum, which argues for just
using "missing_value".
--Russ
_____________________________________________________________________
Russ Rew UCAR Unidata Program
russ@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu
> Grace and peace,
>
> Jim
>
> On 7/15/2011 3:09 PM, Russ Rew wrote:
> > Hi Jim,
> >
> >> In the ESIP Federation CF-Satellite session yesterday (July 14, 2011), I
> >> suggested that it would be good to have a standard scheme for handling
> >> multiple marker values in satellite data that indicate missing data.
> >> There is confusion in the current documentation (between netCDF and CF)
> >> about the proper use of the missing_value attribute, in particular about
> >> whether or not it is acceptable for it to be a vector attribute. I
> >> think there is good reason to allow it to be a vector attribute, and
> >> that allowing the attribute to be a vector quantity is useful beyond the
> >> satellite / remote sensing arena. The problem that arises with having a
> >> vector missing_value attribute is the assignment of meanings to the
> >> different values.
> >>
> >> One way to deal with this is to have a missing_value_meanings
> >> attribute. The missing_value_meanings attribute would be formed in the
> >> same way as the flag_meanings attribute. There would be one string of
> >> non-whitespace characters per element of the missing_value attribute,
> >> separated by spaces. Each string would be a human-readable phrase
> >> (delineating the words of the phrase using underscores or camel-casing)
> >> that explained the meaning associated with the corresponding
> >> missing_value element. Here's an example fragment of cdl.
> >>
> >> float foo(points);
> >>
> >> foo:missing_value = -999.1, -999.2;
> >> foo:missing_value_meanings = "InputsOutOfRange
> >> SolutionDidNotConverge";
> > I agree that it would be very useful to have an attribute for multiple
> > special values for satellite data as well as other kinds of data. IIRC,
> > you gave an example of 10 different kinds of missing values for an NPP
> > sensor.
> >
> > I'm a little concerned about breaking backward compatibility by
> > extending the "missing_value" attribute for this purpose, because there
> > may be software that depends on a one-to-one mapping between a
> > missing_value for a variable and some alternate representation for it,
> > such as the '_' notation ncdump already uses. The fact that the ncdump
> > and ncgen utilities are inverses of each other, converting netCDF data
> > to CDL and back to netCDF without changing the data, requires a single
> > value for the missing value to avoid loss of information when converting
> > from '_' back to the corresponding value.
> >
> > I suggest just using the existing CF flag conventions for this purpose
> > instead, using the variable attributes "flag_values", and
> > "flag_meanings" for any number of special values for a variable.
> >
> > If one of those values is a missing value, the associated flag_meaning
> > could just be "missing_value", but flag_meanings can also specify an
> > arbitrary number of other nuanced kinds of missing values or various
> > other kinds of special values.
> >
> > http://cfconventions.org/documents/cf-conventions/1.5/cf-conventions.htm
> l#flags
> >
> > --Russ
> > _____________________________________________________________________
> >
> > Russ Rew UCAR Unidata Program
> > russ@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu
>
> --
> Jim Biard
>
> Government Contractor, STG Inc.
> Remote Sensing and Applications Division (RSAD)
> National Climatic Data Center
> 151 Patton Ave.
> Asheville, NC 28801-5001
>
> jim.biard@xxxxxxxx
> 828-271-4900
>
>
> --Boundary_(ID_xMNVEAaJBvpOih/JfomxKw)
> Content-type: text/html; charset=ISO-8859-1
> Content-transfer-encoding: 7BIT
>
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
> <html>
> <head>
> <meta content="text/html; charset=ISO-8859-1"
> http-equiv="Content-Type">
> </head>
> <body text="#000000" bgcolor="#ffffff">
> Russ,<br>
> <br>
> I'm confused about a few things. If you don't mind spending a bit
> of time clarifying, I'd appreciate it.<br>
> <br>
> Here are my confused questions.<br>
> <ul>
> <li>Isn't the _FillValue attribute the one that maps to '_'? (See
> the discussion of _FillValue and missing_value <a
> href="http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Attribute-Conve
> ntions.html">here</a>.)<br>
> </li>
> <li>Is the flag_values attribute considered applicable in the case
> of floating point variables?</li>
> <li>In what sense is this suggestion making a change to the
> interpretation of the missing_value attribute?</li>
> </ul>
> Grace and peace,<br>
> <br>
> Jim<br>
> <br>
> On 7/15/2011 3:09 PM, Russ Rew wrote:
> <blockquote cite="mid:10685.1310756965@xxxxxxxxxxxxxxxx" type="cite">
> <pre wrap="">Hi Jim,
>
> </pre>
> <blockquote type="cite">
> <pre wrap="">In the ESIP Federation CF-Satellite session yesterday (J
> uly 14, 2011), I
> suggested that it would be good to have a standard scheme for handling
> multiple marker values in satellite data that indicate missing data.
> There is confusion in the current documentation (between netCDF and CF)
> about the proper use of the missing_value attribute, in particular about
> whether or not it is acceptable for it to be a vector attribute. I
> think there is good reason to allow it to be a vector attribute, and
> that allowing the attribute to be a vector quantity is useful beyond the
> satellite / remote sensing arena. The problem that arises with having a
> vector missing_value attribute is the assignment of meanings to the
> different values.
>
> One way to deal with this is to have a missing_value_meanings
> attribute. The missing_value_meanings attribute would be formed in the
> same way as the flag_meanings attribute. There would be one string of
> non-whitespace characters per element of the missing_value attribute,
> separated by spaces. Each string would be a human-readable phrase
> (delineating the words of the phrase using underscores or camel-casing)
> that explained the meaning associated with the corresponding
> missing_value element. Here's an example fragment of cdl.
>
> float foo(points);
>
> foo:missing_value = -999.1, -999.2;
> foo:missing_value_meanings = "InputsOutOfRange
> SolutionDidNotConverge";
> </pre>
> </blockquote>
> <pre wrap="">
> I agree that it would be very useful to have an attribute for multiple
> special values for satellite data as well as other kinds of data. IIRC,
> you gave an example of 10 different kinds of missing values for an NPP
> sensor.
>
> I'm a little concerned about breaking backward compatibility by
> extending the "missing_value" attribute for this purpose, because there
> may be software that depends on a one-to-one mapping between a
> missing_value for a variable and some alternate representation for it,
> such as the '_' notation ncdump already uses. The fact that the ncdump
> and ncgen utilities are inverses of each other, converting netCDF data
> to CDL and back to netCDF without changing the data, requires a single
> value for the missing value to avoid loss of information when converting
> from '_' back to the corresponding value.
>
> I suggest just using the existing CF flag conventions for this purpose
> instead, using the variable attributes "flag_values", and
> "flag_meanings" for any number of special values for a variable.
>
> If one of those values is a missing value, the associated flag_meaning
> could just be "missing_value", but flag_meanings can also specify an
> arbitrary number of other nuanced kinds of missing values or various
> other kinds of special values.
>
> <a class="moz-txt-link-freetext" href="http://cfconventions.org/documents/c
> f-conventions/1.5/cf-conventions.html#flags">http://cfconventions.org/documen
> ts/cf-conventions/1.5/cf-conventions.html#flags</a>
>
> --Russ
> _____________________________________________________________________
>
> Russ Rew UCAR Unidata Program
> <a class="moz-txt-link-abbreviated" href="mailto:russ@xxxxxxxxxxxxxxxx";>russ@
> unidata.ucar.edu</a> <a class="moz-txt-link-freetext" hre
> f="http://www.unidata.ucar.edu";>http://www.unidata.ucar.edu</a>
> </pre>
> </blockquote>
> <br>
> <pre class="moz-signature" cols="72">--
> Jim Biard
>
> Government Contractor, STG Inc.
> Remote Sensing and Applications Division (RSAD)
> National Climatic Data Center
> 151 Patton Ave.
> Asheville, NC 28801-5001
>
> <a class="moz-txt-link-abbreviated" href="mailto:jim.biard@xxxxxxxx";>jim.biar
> d@xxxxxxxx</a>
> 828-271-4900
> </pre>
> </body>
> </html>
>
> --Boundary_(ID_xMNVEAaJBvpOih/JfomxKw)--
