Re: [netcdf-java] Multi-threading and gridsets - help

• To: Steven Hawes <steven.hawes@xxxxxxxxxxxxx>
• Subject: Re: [netcdf-java] Multi-threading and gridsets - help
• From: Christian Ward-Garrison <cwardgar@xxxxxxxx>
• Date: Fri, 28 Jul 2017 14:20:59 -0600

Hi Steven,

Are you sure you need NetcdfDataset.initNetcdfFileCache() ? It is generally
used when you expect to access the same file multiple times and you don't
want to re-read it and re-construct its metadata each time. Is that your
use case?

As for the threading issue, can you provide a complete example that
demonstrates it, including code and resources?

Thanks,
Christian

On Fri, Jul 28, 2017 at 12:49 AM, Steven Hawes <steven.hawes@xxxxxxxxxxxxx>
wrote:

> Hello,
>
> I’m having trouble with threading and using GridDataSets. I was wondering
> what I am doing wrong.
>
> Using the tutorial I initially create the file cache as such
>
> *NetcdfDataset.initNetcdfFileCache*(...)
>
> Then I’m opening the grid data set using the open command, which uses the
> NetcdfDataset.aquire… to use the file cache
>
> GridDataset gds = ucar.nc2.dt.grid.GridDataset.open(location);
>
> Then when I want to read the data from a grid I use:
>
> Array data  = grid.readDataSlice(0, 0, xy[1], xy[0])
>
> However, when running this code with threads, it throws out this exception
> (works fine with one thread). It appears to access 
> ucar.unidata.io.RandomAccessFile.read,
> which I thought the cache prevented it from using?
>
> java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 131006
> at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:609)
> at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:517)
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 131006
> at ucar.unidata.io.RandomAccessFile.read(RandomAccessFile.java:643)
> at ucar.unidata.io.RandomAccessFile.readShort(RandomAccessFile.java:1115)
> at ucar.unidata.io.RandomAccessFile.readShort(RandomAccessFile.java:1137)
> at ucar.nc2.iosp.IospHelper.readData(IospHelper.java:123)
> at ucar.nc2.iosp.IospHelper.readDataFill(IospHelper.java:79)
> at ucar.nc2.iosp.netcdf3.N3raf.readData(N3raf.java:63)
> at ucar.nc2.iosp.netcdf3.N3iosp.readData(N3iosp.java:484)
> at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:2020)
> at ucar.nc2.Variable.reallyRead(Variable.java:913)
> at ucar.nc2.Variable._read(Variable.java:898)
> at ucar.nc2.Variable.read(Variable.java:709)
> at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:557)
> at ucar.nc2.dataset.VariableDS._read(VariableDS.java:537)
> at ucar.nc2.SectionReader.reallyRead(SectionReader.java:75)
> at ucar.nc2.dataset.VariableDS._read(VariableDS.java:537)
> at ucar.nc2.Variable.read(Variable.java:709)
> at ucar.nc2.Variable.read(Variable.java:655)
> at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:606)
> at ucar.nc2.dt.grid.GeoGrid.readDataSlice(GeoGrid.java:517)
>
> Is there something I am missing?
>
> *STEVE HAWES* | PhD Candidate
> Coastal & Marine Ecosystems Group (CMEG) | School of Life & Environmental
> Sciences (SOLES) | Faculty of Science
>
> *THE UNIVERSITY OF SYDNEY*
> Rm 114, Edgeworth-David Bldg A11 | The University of Sydney | NSW | 2006
> *M* +61 439 881 324 <+61%20439%20881%20324>
> *E* *steven.hawes@xxxxxxxxxxxxx* <firstname.surname@xxxxxxxxxxxxx>  | *W* 
> *sydney.edu.au
> <http://sydney.edu.au>*
>
> Tweets @stevehawes
>
> *INSPIRED – the Campaign to support the University of Sydney*
> *sydney.edu.au/inspired* <http://inspired.sydney.edu.au/>
>
> CRICOS 00026A
> This email plus any attachments to it are confidential. Any unauthorised
> use is strictly prohibited. If you receive this email in error, please
> delete it and any attachments.
>
>
> Please think of our environment and only print this e-mail if necessary.
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>

• Follow-Ups:
  • Re: [netcdf-java] Multi-threading and gridsets - help
    • From: Steven Hawes

• References:
  • [netcdf-java] Multi-threading and gridsets - help
    • From: Steven Hawes