[thredds] TDS NCSS: issue with netCDF4 files as input and netCDF4 requested as output

• To: "thredds@xxxxxxxxxxxxxxxx" <thredds@xxxxxxxxxxxxxxxx>
• Subject: [thredds] TDS NCSS: issue with netCDF4 files as input and netCDF4 requested as output
• From: Jolibois Tony <tjolibois@xxxxxx>
• Date: Thu, 10 Dec 2015 15:29:42 +0000

Hi Thredds team,

We're currently testing TDS netCDF Subset Service, its performance, 
robustness.. These tests include different netCDF format, to study the impact 
on performance and robustness with different compression levels.
To study this impact, we run 4 scenarios for input/output data files for TDS 
NCSS.
Input netCDF3 => output netCDF3
Input netCDF4 => output netCDF3
Input netCDF3 => output netCDF4
Input netCDF4 => output netCDF4
We're facing an error only with the 4th scenario, netCDF4 => netCDF4 (NCSS REST 
parameter &accept=netcdf4).


java.io.IOException: -101: NetCDF: HDF error

        at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2821) 
~[netcdf4-4.6.3.jar:4.6.3]

        at ucar.nc2.jni.netcdf.Nc4Iosp.writeData(Nc4Iosp.java:2729) 
~[netcdf4-4.6.3.jar:4.6.3]

        at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:953) 
~[cdm-4.6.3.jar:4.6.3]

        at ucar.nc2.NetcdfFileWriter.write(NetcdfFileWriter.java:937) 
~[cdm-4.6.3.jar:4.6.3]

        at ucar.nc2.FileWriter2.copyAll(FileWriter2.java:427) 
~[cdm-4.6.3.jar:4.6.3]

        at ucar.nc2.FileWriter2.copyVarData(FileWriter2.java:380) 
~[cdm-4.6.3.jar:4.6.3]

        at 
ucar.nc2.dt.grid.CFGridWriter2.writeOrTestSize(CFGridWriter2.java:244) 
~[cdm-4.6.3.jar:4.6.3]

        at ucar.nc2.dt.grid.CFGridWriter2.writeFile(CFGridWriter2.java:112) 
~[cdm-4.6.3.jar:4.6.3]

        at 
thredds.server.ncss.controller.GridResponder.writeGridFile(GridResponder.java:250)
 ~[classes/:4.6.3]

        at 
thredds.server.ncss.controller.GridResponder.writeLatLonSubset(GridResponder.java:166)
 ~[classes/:4.6.3]

        at 
thredds.server.ncss.controller.GridResponder.getResponseFile(GridResponder.java:97)
 ~[classes/:4.6.3]

        at 
thredds.server.ncss.controller.NcssController.handleRequestGrid(NcssController.java:175)
 ~[classes/:4.6.3]

        at 
thredds.server.ncss.controller.NcssController.handleRequest(NcssController.java:140)
 ~[classes/:4.6.3]

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_65]

        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_65]

        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_65]

        at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_65]

The error comes from netcdf4 java library used inside TDS version 4.6.3 (latest 
stable version).

The error appears only if we have a geographical subset (works with no subset 
or temporal aggregation).

Different NetCDF-4 input files have been tested:

*         original Netcdf3 files converted with the command nccopy -k4 -d3 
input.nc output.nc4

*         original Netcdf4 generated with python library

For information on software versions:

misgw-qt-new:/home/misgw# ls -ltr

total 246300

-rw-r--r-- 1 root  root    9130958 oct  7 21:09 apache-tomcat-8.0.28.tar.gz

-rw-rw-r-- 1 misgw misgw 181260798 nov  9 10:48 jdk-8u65-linux-x64.gz

-rw-r--r-- 1 root  root   61808104 nov  9 11:03 thredds_4.6.3.war


Thanks for the support.
Tony Jolibois

-
Tony Jolibois
Product Dissemination Services
Space Oceanography
+33 5 61 39 37 97

[logo_cls_small]
http://www.cls.fr/

• Follow-Ups:
  • Re: [thredds] TDS NCSS: issue with netCDF4 files as input and netCDF4 requested as output
    • From: Christian Ward-Garrison