NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Wed, Apr 15, 2015 at 10:12 AM, John Caron <caron@xxxxxxxx> wrote: > netcdf-java has several levels of API. The lowest level (NetcdfFile) > doesnt touch the data, so you just get whatever is in there. this is the > level that users of the C library are used to. > > the next highest level of the netcdf-java API (NetcdfDataset) process > missing data, scale/offset, CF conventions, and other things. At this layer > the various attribute conventions are used to identify missing data. > got it -- and I didn't say it, b ut I was thinking that in a data-model type lib, you probably want missing values handles well. But I still don't think I would want values outside the bounds specified as getting turned into missing values unless I specifically asked for that in some sort of QA/QC flag. Which it seemed the OP was expecting. -Chris > John > > On Wed, Apr 15, 2015 at 10:27 AM, Chris Barker <chris.barker@xxxxxxxx> > wrote: > >> As I read those docs, and as I've always interpreted netcdf, those >> "rules" are there to inform the user of the data -- I don't expect client >> software reading the data to replace values in the the arrays for me -- at >> least not low-level generic data reading libraries. >> >> What were you expecting, and why? >> >> Higher level libs that represent a data model _may_ do some QA/QC on the >> data, I suppose, but even then, I'd want to clearly specify what I wanted >> it to do -- certainly not silently toss out values because the were outside >> the limits specified in the meta-data. >> >> -Chris >> >> >> >> On Wed, Apr 15, 2015 at 9:18 AM, John Caron <caron@xxxxxxxx> wrote: >> >>> The netcdf-java library does indeed ignore that rule. We should revisit >>> the manual and clarify things i guess. >>> >>> On Wed, Apr 15, 2015 at 9:33 AM, Russ Rew <russ@xxxxxxxxxxxxxxxx> wrote: >>> >>>> Hi Phil, >>>> >>>> I think that's a misinterpretation of the Users Guide attribute >>>> conventions. Under the description for _FillValue, the Guide states: >>>> >>>> The fill value ... is normally outside the valid range and therefore >>>> treated as missing when read by generic applications. It is legal (but not >>>> recommended) for the fill value to be within the valid range. >>>> >>>> >>>> The last sentence implies that the valid range is not determined by the >>>> _Fill_Value. >>>> >>>> I think it's intended that the rule about _Fill_Value under the >>>> description for valid_range only applies in case none of valid_min, >>>> valid_max, or valid_range are specified, so it wouldn't apply to your >>>> example: >>>> >>>> If neither valid_min, valid_max nor valid_range is defined then generic >>>> applications should define a valid range as follows. ... >>>> >>>> >>>> However, I suspect that the rule is confusing enough that writers of >>>> generic clients might just ignore it, even in that case. >>>> >>>> --Russ >>>> >>>> On Wed, Apr 15, 2015 at 8:55 AM, Bentley, Philip < >>>> philip.bentley@xxxxxxxxxxxxxxxx> wrote: >>>> >>>>> Hi folks, >>>>> >>>>> I was hoping someone can clarify for me the correct use of the >>>>> valid_min, valid_max and valid_range attributes by *well-behaved* netCDF >>>>> clients. >>>>> >>>>> Given the netCDF file shown below (in CDL form), and considering the >>>>> rules for handling the aforementioned attributes, as defined in Appendix A >>>>> of the NetCDF user guide (see >>>>> http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html), >>>>> I *think* I'd expect conforming netCDF clients to represent the data for >>>>> each of the variables var1..var3 as an array with values (_, -272, 0, 100, >>>>> 9999, _), where _ signifies missing data values. >>>>> >>>>> --- >>>>> netcdf mditest { >>>>> >>>>> dimensions: >>>>> dim1 = 6; >>>>> >>>>> variables: >>>>> float var1(dim1); >>>>> var1:long_name = "var1"; >>>>> var1:units = "1"; >>>>> var1:valid_min = -273.0f; >>>>> var1:valid_max = 1.0e5f; >>>>> >>>>> float var2(dim1); >>>>> var2:long_name = "var2"; >>>>> var2:units = "1"; >>>>> var2:valid_min = -273.0f; >>>>> var2:_FillValue = 1.0e5f; // constrains valid_max >>>>> >>>>> float var3(dim1); >>>>> var3:long_name = "var3"; >>>>> var3:units = "1"; >>>>> var3:_FillValue = -273.0f; // constrains valid_min >>>>> var3:valid_max = 1.0e5f; >>>>> >>>>> // global attributes >>>>> :Conventions = "CF-1.0"; >>>>> >>>>> data: >>>>> var1 = -300, -272, 0, 100, 9999, 1e6 ; >>>>> var2 = -300, -272, 0, 100, 9999, 1e6 ; >>>>> var3 = -300, -272, 0, 100, 9999, 1e6 ; >>>>> } >>>>> --- >>>>> >>>>> However, having tried several different netCDF clients - some C, some >>>>> Java, some Python - none of them appear to adhere consistently to the >>>>> aforementioned rules for handling the valid min/max/range attributes. The >>>>> python-based clients, in particular, only seem to honour the _FillValue >>>>> attribute, reflecting, I believe, the current behaviour of the >>>>> netcdf4-python module. >>>>> >>>>> Am I perhaps misinterpreting the nc attribute-handling conventions? >>>>> >>>>> Regards, >>>>> >>>>> -- >>>>> Phil Bentley PhD, Climate Science IT Apps Group >>>>> Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB, United Kingdom >>>>> Tel: +44 (0)1392 886881 >>>>> Email: philip.bentley@xxxxxxxxxxxxxxxx Web: >>>>> http://www.metoffice.gov.uk/ >>>>> >>>>> >>>>> _______________________________________________ >>>>> netcdfgroup mailing list >>>>> netcdfgroup@xxxxxxxxxxxxxxxx >>>>> For list information or to unsubscribe, visit: >>>>> http://www.unidata.ucar.edu/mailing_lists/ >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> netcdfgroup mailing list >>>> netcdfgroup@xxxxxxxxxxxxxxxx >>>> For list information or to unsubscribe, visit: >>>> http://www.unidata.ucar.edu/mailing_lists/ >>>> >>> >>> >>> _______________________________________________ >>> netcdfgroup mailing list >>> netcdfgroup@xxxxxxxxxxxxxxxx >>> For list information or to unsubscribe, visit: >>> http://www.unidata.ucar.edu/mailing_lists/ >>> >> >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R (206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> Chris.Barker@xxxxxxxx >> >> _______________________________________________ >> netcdfgroup mailing list >> netcdfgroup@xxxxxxxxxxxxxxxx >> For list information or to unsubscribe, visit: >> http://www.unidata.ucar.edu/mailing_lists/ >> > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@xxxxxxxx
netcdfgroup archives: