NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
> Date: Mon, 23 Apr 2001 22:03:30 -0600 (MDT) > From: Don Hooper <hoop@xxxxxxxxxxxx> > Subject: Re: Wanted: examples of uses of missing data AND scale/offset > > John Caron, > > Well, the following data sets at ftp://ftp.cdc.noaa.gov/Datasets/: > coads cpc_us_precip interp_OLR kaplan_sst msu ncep ncep.pac.ocean > ncep.reanalysis noaa_hrc nodc.woa94 recon_reynolds_sst reynolds_sst > udel.airt.precip > seem to fit the bill that you describe, although it is definitely > for others to decided if the constitute "important holdings". They're > important to CDC, at least. ;) You should be able to find them via http > protocol, starting at: > http://www.cdc.noaa.gov/PublicData/ > They are also available via DODS, using URLs that start with: > http://www.cdc.noaa.gov/cgi-bin/nph-nc/Datasets/ > Now, when we got started with our conventions, we felt that the > User's Guide's discussion of conventions was indeed ambiguous regarding > the type of the valid_range attribute, merely saying it needed to match > that of its data variable, without mentioning the complication that such > a variable would have two types (external/internal, AKA packed/unpacked). > We made the same call that was described earlier in the discussion, namely > that it should be more of a human readable concept, as our software > blithely > used the missing_value attribute (in the external/packed type) to discern > the validity of a value. I recall hearing a couple of complaints about > our typing of the valid_range attribute. I doubt it prevented anyone from > making good use of our data, however. ;) I hope this is of some use in > the on-going discussion. > We did use some of this data. We had to change the valid_range so our software could read it without treating everything as missing. > Date: Tue, 24 Apr 2001 15:41:21 -0600 (MDT) > From: Brian Eaton <eaton@xxxxxxxxxxxxx> > Subject: Re: ncdigest V1 #588 (standard handling of scale/offset and missi > ng data) > > > > There are two cases. > > > _FillValue is a valid value if it is within a valid range defined by > > > valid_range, valid_min, valid_max. > > > If none of these three attributes are defined then it is assumed that > > > _FillValue is a missing value & > > > should be used to define one end of the valid range. > > > > > > So if you want to use _FillValue to initialize a variable to a valid > value > > > 0, then you must define at > > > least one of valid_range, valid_min, valid_max. > > > > this seems reasonable to me. Does anyone have datasets where this > > algorithm would be wrong? > > A dataset that was written with a _FillValue within a specified valid > range > would not have that _FillValue treated as the valid default data value by > many existing applications, i.e., such a dataset would break most existing > applications. It's confusing to have _FillValue take on opposite meanings > depending on whether or not a valid range is defined. I'd prefer that > your > interface didn't support this interpretation, especially since the UG > recommends against it. > I do not understand how this would break most existing applications. For example, say we have unscaled float data with: valid_range = -1000.0, 1000.0 _FillValue = 0.0 Surely 0.0 has to be treated as a valid value, doesn't it? Otherwise there would be two valid ranges separated by a gap around 0.0 which would require four comparisons! But I can see there might be some doubt if one had valid_max = 1000.0 _FillValue = 0.0 In this case it is not clear whether the valid min should be -infinity or a number slightly larger than 0.0. The U.G. implies it should be -infinity.
netcdfgroup archives: