NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

Re: [netcdf-java] Reading sequences from BUFR with NetcdfDataset

Hi John,

many thanks for your help! You saved my day!

All read data looks very well. 

BTW: 
The exception, which I found also happens in ToolsUI:
- Open toolsUI-5.0.0.jar
- Click Viewer
- Enable Enhance
- Open my BUFR file
- Click on Show data as shown in the screenshot

The full exception in the window is:
java.lang.ClassCastException: ucar.ma2.ArrayObject$D1 cannot be cast to 
ucar.ma2.ArrayStructure
        at ucar.nc2.dataset.StructureDS.convert(StructureDS.java:268)
        at ucar.nc2.dataset.SequenceDS.read(SequenceDS.java:84)
        at ucar.nc2.ui.op.DatasetViewer.showData(DatasetViewer.java:566)
        at ucar.nc2.ui.op.DatasetViewer.access$1000(DatasetViewer.java:59)
        at 
ucar.nc2.ui.op.DatasetViewer$NestedTable$3.actionPerformed(DatasetViewer.java:416)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.AbstractButton.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown 
Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$500(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at 
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
 Source)
        at 
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
 Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at 
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown
 Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)

Kind regards

Netty 

Gesendet: Dienstag, 03. September 2019 um 03:55 Uhr
Von: "John Caron" <jcaron1129@xxxxxxxxx>
An: netty.jawn@xxxxxx
Cc: "NetCDF-Java community" <netcdf-java@xxxxxxxxxxxxxxxx>
Betreff: Re: [netcdf-java] Reading sequences from BUFR with NetcdfDataset

Hi netty:
 
Sorry for the crappy documents and confusion. Sequences are strange, they dont 
behave like normal Arrays. As sort-of described here:
 
https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html#readSequences
 
you cant call read(), you can only call getStructureIterator() and iterate, 
because sequences dont have a fixed size. You have nested sequences: a BUFR 
file consists of a variable number of BUFR messages (top level sequence), and 
each BUFR message has a variable number of Long_time_period_or_displacement 
fields (nested sequence). Heres some code that can get you started:
 
public void testReadSequenceEnhanced() throws IOException {try (NetcdfDataset 
ncd = NetcdfDataset.openDataset("yourFilename")) {SequenceDS record = 
(SequenceDS) ncd.findVariable("obs");
try (StructureDataIterator iter = record.getStructureIterator()) {int 
recordCount = 0;while (iter.hasNext()) {StructureData sdata = 
iter.next();Assert.assertNotNull(sdata); recordCount++;
ArraySequence nestedSequence = 
sdata.getArraySequence("Long_time_period_or_displacement");try 
(StructureDataIterator nestedIter = nestedSequence.getStructureDataIterator()) 
{int nestedCount = 0;while (nestedIter.hasNext()) {StructureData nestedData = 
nestedIter.next();Assert.assertNotNull(nestedData);NCdumpW.printStructureData(new
 PrintWriter(System.out), nestedData); nestedCount++; 
}Assert.assertEquals(1685, nestedCount); } }Assert.assertEquals(1, 
recordCount); } } }
 
Theres also this doc that may be useful:
https://www.unidata.ucar.edu/software/netcdf-java/current/reference/StructureData.html[https://www.unidata.ucar.edu/software/netcdf-java/current/reference/StructureData.html]
Good luck! 

On Mon, Sep 2, 2019 at 8:18 AM <netty.jawn@xxxxxx[mailto:netty.jawn@xxxxxx]> 
wrote:Hi,

I have to read the attached BUFR-File with netcdf-java and it works a little 
bit.

I opened my BUFR-File with ToolsUI to examine the content. Here was the first 
surprise: The icon 
https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/images/netcdf-java/tutorial/cdmdatasets/enhanceButton.jpg[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/images/netcdf-java/tutorial/cdmdatasets/enhanceButton.jpg]
shows "add Coordinates..." as tooltip. But it is actually the enable/disable of 
the enhance-mode, as stated here
https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html]
The tooltip should be changed.

Reading variables was easy thanks to the enhance-mode:
try (NetcdfDataset ncFile = NetcdfDataset.openDataset(filename);) {
  VariableDS variable = (VariableDS) 
ncFile.findVariable("obs.Software_identification_and_version_number");
  System.out.println(variable.readScalarString());
  }
Float and all other types are correctly enhanced.

Naively I thought reading sequences is easy too. I have to read the sequence 
"obs.Long_time_period_or_displacement" (as an example). Aunt Google showed me 
the function "showNestedSequence" from 
https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html[https://www.unidata.ucar.edu/software/netcdf-java/current/reference/Cookbook.html]
 (can't find it in the current documentation). This function shows me all 
values, but not enhanced. Furthermore 2 nested loops were used. I would prefer 
ONE loop (if possible).

After (I think) thousands of failed attempts, I came to this
try (NetcdfDataset ncFile = NetcdfDataset.openDataset(filename);) {
  VariableDS variable = (VariableDS) 
ncFile.findVariable("obs.Long_time_period_or_displacement");
  System.out.println("Type: " + variable.getDataType());
  Array value = variable.read();
  System.out.println("THIS LINE IS NOT PRINTED");
  }
The following exception is thrown:
java.lang.ClassCastException: class ucar.ma2.ArrayObject$D1 cannot be cast to 
class ucar.ma2.ArrayStructure (ucar.ma2.ArrayObject$D1 and 
ucar.ma2.ArrayStructure are in unnamed module of loader 
org.apache.catalina.loader.ParallelWebappClassLoader @ebbfb89)
        at ucar.nc2.dataset.StructureDS.convert(StructureDS.java:268)
        at ucar.nc2.dataset.SequenceDS.read(SequenceDS.java:84)
        at de.sample.BUFRTest.showNestedSequenceNew(BUFRTest.java:88)
Line 88 is the "variable.read();".
I called your netcdf-lib from a servlet, therefore the "org.apache.catalina", 
but this should not be the problem.

So it seems this sentence is not correct:
https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html[https://docs.unidata.ucar.edu/netcdf-java/5.0/userguide/netcdf_dataset.html]
"Note that NetcdfDataset is a subclass of NetcdfFile, and so can be used 
wherever a NetcdfFile is used."

Someone suggested to have a look at NCdumpW, which I also did, but this program 
does not enhance any values.

Since I don't want to open my file as NetcdfDataset and again as NetcdfFile, I 
would try to open it as NetcdfFile and manually enhance the values. But no idea 
how to do this. Any good solution for my problem is also appreciated.

Thanks

Kind regards

Netty_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx[mailto:netcdf-java@xxxxxxxxxxxxxxxx]
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/[https://www.unidata.ucar.edu/mailing_lists/]

Attachment: bug-toolsui.jpg
Description: JPEG image