[thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP

• To: thredds@xxxxxxxxxxxxxxxx
• Subject: [thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP
• From: John Kerfoot <johnkerfoot@xxxxxxxxx>
• Date: Wed, 27 Apr 2016 10:52:17 -0400

Hello,

I've run into a problem when trying to serve NetCDF files that contain
ushort datatypes.  Here's an example:

http://opendap.oceanobservatories.org:8090/thredds/catalog/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/catalog.html?dataset=ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc

When I go to the OPeNDAP Access form for this dataset:

http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc.html

and look at the description of the variable 'external_temp_raw', I see that
it is a 16-bit Unsigned Integer, though the _FillValue == -1, which is not
possible.  If I request the first 10 records of this variable:

http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc?external_temp_raw[0:1:10]

I get values that are within the range of the ushort data type (0-65535)

Here's where things get strange:  If I use ncdump to inspect the data
setvia the opendap url:

$ ncdump -v external_temp_raw
http://opendap.oceanobservatories.org:8090/thredds/dodsC/ooi/_nouser/20160426T023221-GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered/deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc
| grep external_temp_raw
short external_temp_raw(obs) ;
external_temp_raw:_FillValue = -1s ;
external_temp_raw:comment = "Temperature external to the instrument
measured in counts" ;
external_temp_raw:units = "counts" ;
external_temp_raw:coordinates = "time lat lon" ;
external_temp_raw:long_name = "Raw External Temperature, counts" ;
external_temp_raw:_Unsigned = "true" ;
external_temp_raw:_ChunkSizes = 9689 ;
 external_temp_raw = -24937, -24937, -24933, -24931, -24939, -24937, -24937,

The data type is shown as short and the first 7 values of the variable are
returned as negative values.  THREDDS is treating these datatypes as shorts
instead of ushorts.

If I download the NetCDF file via HTTPServer and run the same ncdump
command, here's what I get:

$ ncdump -v external_temp_raw ~/Downloads/
deployment0001_GI01SUMO-RID16-01-OPTAAD000-recovered_host-optaa_dj_dcl_instrument_recovered_20140924T200016-20140930T130103.001000.nc
| grep external_temp_raw
        ushort external_temp_raw(obs) ;
external_temp_raw:_FillValue = 65535US ;
external_temp_raw:comment = "Temperature external to the instrument
measured in counts" ;
external_temp_raw:units = "counts" ;
external_temp_raw:coordinates = "time lat lon" ;
external_temp_raw:long_name = "Raw External Temperature, counts" ;
 external_temp_raw = 40599, 40599, 40603, 40605, 40597, 40599, 40599, 40599,

So, the actual file shows the datatype as ushort and a _FillValue ==
65335US and the data values are within the expected range for that datatype .

Any ideas as to why THREDDS appears to be casting the ushort datatype to
short and changing the actual values of that variable when it is accessed
via OPeNDAP?

Thanks,
John

• Follow-Ups:
  • Re: [thredds] Problem serving datasets with ushort data types via THREDDS OPeNDAP
    • From: dmh@xxxxxxxx