NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi John,
Yes, I thought it might be the scanning that takes the time. I had
assumed that the scanning takes place when you open a NetcdfDataset,
but I guess I'm wrong? Perhaps this implies that I need a
GridDatatypeCache as well as a NetcdfDatasetCache? (Caching is making
my head hurt, there are too many in my application already ;-)
Cheers, Jon
On Tue, Mar 18, 2008 at 4:22 PM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
> that should all work. the time may be in the scanning of 1000 files. Still, I
> have had other reports
> of extra slowness, I will investigate when I can.
>
>
>
> Jon Blower wrote:
> > Hi John,
> >
> > It's a JoinNew, because the data files do not contain a time axis.
> > Hence we use a <scan> element and the time axis values are deduced
> > from the file names using dateFormatMark. Our ncml file is attached.
> >
> > Yes, this is in our own application (ncWMS) not the TDS.
> >
> > Cheers, Jon
> >
> > On Tue, Mar 18, 2008 at 2:21 AM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
> >> HiJon:
> >>
> >> Are these JoinExisting or JoinNew ? How is the coordinate value
> specified? You are doing this in your own application, not in TDS?
> >>
> >>
> >> Jon Blower wrote:
> >> > Hi all,
> >> >
> >> > Using the Java NetCDF libraries, version 2.2.22, we have notice that
> >> > opening a GridDataset using TypedDatasetFactory.open() is slow when
> >> > the dataset in question is an NcML aggregation over a long timeseries
> >> > (around a thousand timesteps, with one timestep per file). The call
> >> > to TDF.open() takes around 3 seconds every time, which is a problem
> >> > for our application, where performance is important.
> >> >
> >> > We are using the NetcdfDatasetCache, which doesn't seem to help with
> >> > this problem. Is there something we can do to speed up the process of
> >> > creating a GridDataset? I don't think this was a problem for us in
> >> > previous library versions, where we used a different syntax
> >> > ("GridDataset gd = new GridDataset(nc);").
> >> >
> >> > I guess we could cache the GridDataset object in memory at the
> >> > application level, but is there a reason not to do this (e.g. if this
> >> > is a very large object)?
> >> >
> >> > Here's what we do using version 2.2.22:
> >> >
> >> > NetcdfDataset nc = NetcdfDataset nc =
> NetcdfDatasetCache.acquire(filename);
> >> > GridDataset gd =
> >> > (GridDataset)TypedDatasetFactory.open(DataType.GRID, nc, null, null);
> >> >
> >> > Thanks, Jon
> >> >
> >> _______________________________________________
> >> netcdf-java mailing list
> >> netcdf-java@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> >>
> >
> >
> >
>
--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: jdb@xxxxxxxxxxxxxxxxxxxx
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------
netcdf-java archives: