NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NetCDF Aggregation Exception



Thank you Rob!
for pointing out that the suffix cannot have a * in it.
Quite obvious now that you brought it to attention.
Removing it made some progress ~ the OPeNDAP service html
page now displays.  Still an issue going on with the time axis,
 in these files which I'm still investigating.

-Dan


Rob Cermak wrote the following on 7/21/2006 7:56 PM:
> Hi Dan,
> 
> I just figured out a little bit about aggrigation.  What do your
> files look like.  I think the problem may be with the
> dateFormatMark.
> 
> dateFormatMark="#yyyyMMdd"
> 
> suffix might need to be changed to ".nc"
> 
> Here is an example from our thredds system that worked.
> 
> First, this is what the netcdf files look like:
> 
> [cermak@daved data]$ ls -1 /space/data/GFS/*
> /space/data/GFS/gfs_20060720_00z.nc
> /space/data/GFS/gfs_20060720_06z.nc
> /space/data/GFS/gfs_20060720_12z.nc
> /space/data/GFS/gfs_20060720_18z.nc
> 
> This is what the aggrigation looks like:
>   <dataset name="GFS Aggregation" ID="GFSagg" urlPath="GFStmp">
>     <serviceName>robDODS</serviceName>
>     <netcdf
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>       <aggregation dimName="time" type="joinExisting">
>         <scan dateFormatMark="gfs_#yyyyMMdd_HH"
> location="/space/data/GFS" suffix=".nc" />
>       </aggregation>
>     </netcdf>
>   </dataset>
> 
> Note: if there is any fixed lettering before the #, it needs to be
> included so the scanner can pick up the filename.  Note my use of
> the suffix.
> 
> Hope this helps!
> 
> Rob
> 
> On Fri, July 21, 2006 2:28 pm, dan.swank wrote:
> 
>>THREDDS:
>>
>>I've adjusted the CDL in the NetCDF files of interest so that the
>>time
>>is the slowest varying dimension.  And I have started small (with 6
>>instead of 6000) files.
>>
>>Watching the logs I made sure there wer no errors/warnings
>>associated
>>with this dataset entry:
>>When I goto the OPeNDAP Service link on this page:
>>http://nomads.ncdc.noaa.gov:8085/thredds/testDatasets.html?dataset=test/BOWdailyagg2
>>The following Java exception appears in threddsServlet.log:
>>
>>============================================================================
>>  <dataset name="Blended Ocean Winds Test Daily Aggregation II"
>>      ID="test/BOWdailyagg2" urlPath="test/BOWdailyagg2"
>>dataType="Grid" >
>>    <access serviceName="allTest" urlPath="test/BOWdailyagg2"
>>        dataFormat="netcdf" />
>>
>>    <netcdf
>>xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>>     <aggregation dimName="time" type="joinExisting">
>>       <variableAgg name="wind" />
>>       <scan dateFormatMark="#yyyyMMdd"
>>location="/raid/seawind-test/"
>>suffix="*.nc" />
>>     </aggregation>
>>    </netcdf>
>>  </dataset>
>>============================================================================
>>2006-07-21T22:09:20.778 +0000 [    155709][      20] INFO  -
>>thredds.servlet.ServletUtil - Remote host: 192.153.129.21 - Request:
>>"GET /thredds/dodsC/test/BOWdailyagg2.html HTTP/1.1"
>>2006-07-21T22:09:20.780 +0000 [    155711][      20] ERROR -
>>dods.servlet.DODSServlet - DODSServlet.anyExceptionHandler
>>java.lang.ArrayIndexOutOfBoundsException: -1
>>        at java.util.ArrayList.get(ArrayList.java:326)
>>        at
>>ucar.nc2.ncml.Aggregation.getTypicalDataset(Aggregation.java:549)
>>        at
>>ucar.nc2.ncml.Aggregation.aggExistingDimension(Aggregation.java:558)
>>        at ucar.nc2.ncml.Aggregation.finish(Aggregation.java:361)
>>        at ucar.nc2.ncml.NcMLReader.readNetcdf(NcMLReader.java:353)
>>        at
>>thredds.servlet.DatasetHandler$NcmlFileFactory.open(DatasetHandler.java:146)
>>        at
>>ucar.nc2.NetcdfFileCache.acquire(NetcdfFileCache.java:181)
>>        at
>>thredds.servlet.DatasetHandler.getNcmlDataset(DatasetHandler.java:125)
>>        at
>>thredds.servlet.DatasetHandler.getNetcdfFile(DatasetHandler.java:57)
>>        at
>>dods.servers.netcdf.NcDODSServlet.getDataset(NcDODSServlet.java:355)
>>        at dods.servlet.DODSServlet.doGetHTML(DODSServlet.java:963)
>>        at dods.servlet.DODSServlet.doGet(DODSServlet.java:1463)
>>        at
>>dods.servers.netcdf.NcDODSServlet.doGet(NcDODSServlet.java:274)
>>        at
>>javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
>>        at
>>javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
>>        at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:284)
>>        at
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:204)
>>        at
>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:151)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>>        at
>>org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:245)
>>        at
>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:199)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:151)
>>        at
>>org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:509)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:149)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>>        at
>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:195)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:151)
>>        at
>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:164)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:149)
>>        at
>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:578)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:149)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>>        at
>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:156)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:151)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
>>        at
>>org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:972)
>>        at
>>org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:206)
>>        at
>>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
>>        at
>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:700)
>>        at
>>org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
>>        at
>>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
>>        at java.lang.Thread.run(Thread.java:534)
>>============================================================================
>>
>>What am I doing wrong?
>>Is this more likely a problem with the configuration file or with
>>the
>>data files?
>>
>>--
>>Dan Swank <address@hidden>
>>NOMADS Project:  Software & Data Management
>>Contractor - STG, Incorporated
>>Veach-Baley Federal Building
>>151 Patton Avenue
>>Asheville, NC 28801-5001
>>Phone: 828-271-4007
>>
>>===============================================================================
>>To unsubscribe thredds, visit:
>>http://www.unidata.ucar.edu/mailing-list-delete-form.html
>>===============================================================================
>>
> 
> 
> 

-- 
Dan Swank <address@hidden>
NOMADS Project:  Software & Data Management
Contractor - STG, Incorporated
Veach-Baley Federal Building
151 Patton Avenue
Asheville, NC 28801-5001
Phone: 828-271-4007

===============================================================================
To unsubscribe thredds, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
===============================================================================