NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: Use of http proxy?

Bummer. Found the real exception and the source - DODS libraries are using the commons http client, which requires explicit proxy setting at an application level (ie proxies must be a supported feature of DAP). But even when skirting DNS availability using hosts file, we go from this, 

java.net.UnknownHostException: www.marine.csiro.au
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at java.net.Socket.connect(Socket.java:469)
       at java.net.Socket.<init>(Socket.java:366)
       at java.net.Socket.<init>(Socket.java:239)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706) at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1321) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) 
       at opendap.dap.DConnect2.openConnection(DConnect2.java:218)

to this,
C:\Program Files\IDV_2.2>jre\bin\java -Dhttp.proxyHost=proxyhost -Dhttp.proxyPor 
t=3128 -Xmx512m -jar idv.jar
25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: I/O exception (java.net.ConnectException) caught when processing request: 
Connection timed out: connect
25/05/2007 14:52:15 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: Retrying request
25/05/2007 14:53:00 org.apache.commons.httpclient.HttpMethodDirector executeWith 
Retry
INFO: I/O exception (java.net.ConnectException) caught when processing request: 
Connection timed out: connect
Looking at the source below, it would indeed seem DAP does not support proxies. This is hard to believe - it's more effective than any EULA for curtailing commercial use. I'll take my grumbles to the DAP group I guess... :( 

opendap.dap.DConnect2.java:
private void openConnection(String urlString, Command command) throws IOException, DAP2Exception, ParseException { 

   initHttpClient();
   GetMethod method = new GetMethod(urlString);
   method.setFollowRedirects(true);

   if (acceptCompress)
method.setRequestHeader(new Header("Accept-Encoding", "deflate,gzip")); 

   // enable sessions
   if (allowSessions)
     method.setRequestHeader(new Header("X-Accept-Session", "true"));

   InputStream is = null;
   try {
     // Execute the method.

     int statusCode = _httpClient.executeMethod(method);
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^





Nick Bower wrote:
IDV seems to be having trouble with our http proxy (squid). I've set the following global parameters in runIDV.bat in the usual way; 

-Dhttp.proxyHost=proxyhost -Dhttp.proxyPort=3128
however IDV fails to retrieve any data in the Data Chooser. See traceback below. Could it have something to do with IDV insisting DODS URI's begin with dods:// perhaps? What is the recommended solution for using a http proxy?
By the way, I can navigate the server in the "Display" / web browsing tab. It's only the Data Chooser that fails. 

Thanks, Nick

An error has occurred:
There was an error loading the data:
Error creating data source:dods.grid with: http://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ Grid data source failed making data set: dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ Cant open dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ or as DODS dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ 
www.marine.csiro.au
java.io.FileNotFoundException: Cant open dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ or as DODS dods://www.marine.csiro.au/dods/nph-dods/dods-data/bl/SPINUP4/temp/ 
www.marine.csiro.au
   at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
   at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
   at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175)
at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171) 
   at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
at ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) at ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) at ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) at ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1446) 
   at ucar.unidata.idv.DefaultIdv.handleAction(DefaultIdv.java:112)
at ucar.unidata.idv.IntegratedDataViewer.handleAction(IntegratedDataViewer.java:1378) at ucar.unidata.idv.chooser.UrlChooser.loadURLInner(UrlChooser.java:250) 
   at ucar.unidata.idv.chooser.UrlChooser.loadURL(UrlChooser.java:222)
at ucar.unidata.idv.chooser.UrlChooser.doLoadInThread(UrlChooser.java:269) 
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
   at java.lang.reflect.Method.invoke(Method.java:585)
   at ucar.unidata.util.Misc$2.run(Misc.java:919)
   at ucar.unidata.util.Misc$3.run(Misc.java:947)


And another example from your own servers...
java.io.FileNotFoundException: Cant open dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc or as DODS dods://motherlode.ucar.edu:8080/thredds/dodsC/modelsNc/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20070524_1200.nc 
motherlode.ucar.edu
   at ucar.nc2.dataset.NetcdfDataset.openDODS(NetcdfDataset.java:380)
   at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:316)
   at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:175)
at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:203) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:183) at ucar.nc2.dataset.NetcdfDatasetCache.acquire(NetcdfDatasetCache.java:171) 
   at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:71)
at ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:726) at ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:743) at ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:335) at ucar.unidata.data.DataManager.createDataSource(DataManager.java:1186) at ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1698) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1617) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1595) at ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1576) at ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:610) at ucar.unidata.idv.chooser.ThreddsHandler.processNodes(ThreddsHandler.java:842) at ucar.unidata.idv.chooser.ThreddsHandler.processNode(ThreddsHandler.java:782) 
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
   at java.lang.reflect.Method.invoke(Method.java:585)
   at ucar.unidata.util.Misc$2.run(Misc.java:921)
   at ucar.unidata.util.Misc$3.run(Misc.java:947)