Re: [netcdf-java] NullPointerException while parsing hdf5 header

• To: John Caron <jcaron1129@xxxxxxxxx>
• Subject: Re: [netcdf-java] NullPointerException while parsing hdf5 header
• From: David Kegley <kegs@xxxxxx>
• Date: Fri, 19 Mar 2021 19:03:42 +0000
• Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=b23.io; dmarc=pass action=none header.from=b23.io; dkim=pass header.d=b23.io; arc=none
• Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=loCZn16GSBWrjpLUIbZoSfY87CZCjpXuP4hx7mwlENk=; b=kNJYEq/liT6nm6qbZF4qI8GQjg0D8DyucPLV4ffVDKVDALxjh7QcOSoUSawfj6QjmLAmYy9yb2bCx8dRGuCEZ/9+WrwRQN5e9d0RlkwlWs7H02PFt9q+DWOPo6PtaSRjeuGVvqHYFhX1wJk6+QnDKQ27Dl7x/Sew1Rb0jzjH62bIgjhw7EmIE1aYBpkxdB1b4FqzGK7Ct1hl62/JEDyRNuv7jmY6ex2p11JKN4cvNSdmPLbHhd7YjnaXWSFRSPmo8ZIu4oG/ymPDJigQpJVz8csY7H7Wa/fRwFHWS+3BUZN6ou7QYgNkMPs6BKJXe6LgO1NqDw1AOeuAFItljp2Euw==
• Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NfF2RQdeqxvyNX8JQPMBBzTK5lXIB8WmnP0rbC7D1ECM7ALyCyJbULwiz4IbOw7xeIFPSsHei3PYlfecrv33SlZy4+rWQ3P1LgM1hUW7YfwEC6OXq7Delswcf2O4VQmQjC1q8zRqUsgDa65vHpCSFpAST0vRWfFFyDWtgc8X6dPEqUYEC0BaFQ0HB3T51HnjxRMcNOgHVI/4hoFFXw2V1SL9n92/ZQNLhVWbpKMq4cxGL2w1sA/MYdKf+d7V7W1vbIuEZXpfUhLfW1xyveEI3rC45djdStOkqFpIjHyjpYBE6RYigh9ieV4fhNkGz6e6gSNnXJgwlmU9clAaniB50Q==
• Authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=b23.io;

Thank you, John! Turns out there was a UUID attribute being stored as a 128-bit 
integer that was causing problems. I was able to move past that error by 
re-writing the file without the UUID attr for now.

From: John Caron <jcaron1129@xxxxxxxxx>
Date: Friday, March 12, 2021 at 10:23 PM
To: David Kegley <kegs@xxxxxx>
Cc: "netcdf-java@xxxxxxxxxxxxxxxx" <netcdf-java@xxxxxxxxxxxxxxxx>
Subject: Re: [netcdf-java] NullPointerException while parsing hdf5 header

On the face of it, you have a file with a 16 byte integer type, which is not 
supported in Java.

On Fri, Mar 12, 2021 at 9:02 AM David Kegley <kegs@xxxxxx<mailto:kegs@xxxxxx>> 
wrote:
Hi All,

I’m attempting to parse a HDF5 file using the netcdf-java library and I’m 
getting the following NullPointer exception while parsing the file header.

[main] WARN ucar.nc2.iosp.hdf5.H5header - HDF5 file /tmp/data.h5 not handling 
hdf integer type (0) with size= 16
java.io.IOException: java.lang.NullPointerException
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:384)
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:343)
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:326)
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:312)
 at gov.noaa.pfel.coastwatch.griddata.NcHelper.openFile(NcHelper.java:712)
 at gov.noaa.pfel.coastwatch.griddata.NcHelper.ncdump(NcHelper.java:104)
 at 
gov.noaa.pfel.erddap.dataset.EDDTableFromMultidimNcFiles.generateDatasetsXml(EDDTableFromMultidimNcFiles.java:228)
 at gov.noaa.pfel.erddap.GenerateDatasetsXml.doIt(GenerateDatasetsXml.java:656)
 at gov.noaa.pfel.erddap.GenerateDatasetsXml.main(GenerateDatasetsXml.java:974)
Caused by: java.lang.NullPointerException
 at ucar.nc2.iosp.hdf5.H5header.readAttributeData(H5header.java:1154)
 at ucar.nc2.iosp.hdf5.H5header.makeAttribute(H5header.java:1002)
 at ucar.nc2.iosp.hdf5.H5header.makeAttributes(H5header.java:978)
 at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:562)
 at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:488)
 at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:197)
 at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:108)
 at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1590)
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:782)
 at ucar.nc2.NetcdfFile.open(NetcdfFile.java:381)
 ... 8 more

The version info from my netcdf-java jar manifest:

Manifest-Version: 1.0
Implementation-Title: CDM core library
Implementation-Version: 5.3.3
Built-By: ubuntu
Implementation-Vendor-Id: edu.ucar
Implementation-URL: https://docs.unidata.ucar.edu/thredds/netcdf-java/
 5.3/userguide/index.html
Created-By: Gradle 3.5.1
Build-Jdk: 1.8.0_161
Built-On: 2020-06-17T05:48:56+0000
Implementation-Vendor: UCAR/Unidata
Main-Class: ucar.nc2.NCdumpW

Additional background:

I’m using the ERDDAP toolkit (https://github.com/BobSimons/erddap/tree/v2.11) 
which utilizes this library, so I’m not calling these methods directly myself.
ERDDAP docker image: axiom/docker-erddap:2.11

The same hdf5 file can be parsed successfully using h5ls:

$ h5ls -frv /tmp/data.h5
Opened "/tmp/data.h5" with sec2 driver.
…

Any thoughts on this would be very helpful. I’m not very familiar with this 
library so if I’m overlooking any relevant details for debugging then please 
let me know.
Unfortunately I cannot share the hdf5 file itself or its schema

Thanks!
David



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx<mailto:netcdf-java@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/

• Follow-Ups:
  • Re: [netcdf-java] NullPointerException while parsing hdf5 header
    • From: John Caron

• References:
  • [netcdf-java] NullPointerException while parsing hdf5 header
    • From: David Kegley
  • Re: [netcdf-java] NullPointerException while parsing hdf5 header
    • From: John Caron