NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [netcdf-java] Fw: ERDDAP bug

yes, bob is correct. use opendap for remote access.

On 10/11/2012 11:30 AM, Brian Schlining wrote:
Bob's response is below.

--
Brian Schlining

Forwarded message:

*From:* Bob Simons <bob.simons@xxxxxxxx>
*To:* Brian Schlining <bschlining@xxxxxxxxx>
*Date:* Thursday, October 11, 2012 10:05:47 AM
*Subject:* Re: ERDDAP bug

I understand the issue that you are talking about. I would instead urge
you not to use this approach. This approach has the implicit assumption
that there is a static .nc file with the requested subset of the larger
dataset and that NetCDF-Java can read specific bytes from that file. See
http://www.unidata.ucar.edu/software/netcdf-java/reference/HTTPservice.html
That is just not a good way to think about that file. Please request a
.nc file from ERDDAP only if you want to download that .nc file.

For your purposes, please use NetCDF-Java to access an ERDDAP dataset as
an OPeNDAP dataset. This is what OPeNDAP was designed for and
NetCDF-Java supports this *very* well. This approach will work *much*
better, for a variety of reasons. Here is a code snippet:

NetcdfFile ncFile =
NetcdfDataset.openFile("http://beach.mbari.org:8180/erddap/griddap/erdRyanSST";,

null);
//note that there is no file extension at the end of the dataset's name
//Then you can do what you want with the ncFile object.

I will improve the documentation related to this. Thanks for pointing
out this weakness in the documentation.

I hope that clarifies the situation. If not, or if you have other
questions about ERDDAP, please let me know.

P.S. I am not on netcdf-java mailing list. Can you please post this
response on the netcdf-java mailing list? Thank you.


On 10/11/2012 9:31 AM, Brian Schlining wrote:
Hi Bob,

We're doing some work with ERDDAP and running into an issue using
NetCDF-Java to access files served by ERDDAP. I think I understand the
issue and know how to address it, so I'm passing the info onto you all
so it can be addressed. So here goes:

1) ERDDAP allows one to download a NetCDF file by building a link
appended with '.nc'. The link URL for the netcdf file would be something
like http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc. This
works great for downloading the files. However, it does NOT work with
the NetCDF-Java API; NetCDF-Java can normally read NetCDF files from
arbitrary non-opendap http urls.

2) The reason it fails is because NetCDF-Java needs to know the size of
the file being served. This requires that the HTTP response for a URL
like http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc to contain
a 'Content-Length' field. ERDDAP is not sending that … here's a response
header from ERDDAP (notice there's no 'Content-Length':

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Date: Thu, 11 Oct 2012 15:44:19 GMT
Last-Modified: Thu, 11 Oct 2012 15:44:19 GMT
xdods-server: dods/3.7
erddap-server: 1.38
Content-Disposition: attachment;filename=erdRyanSST_8571_f367_229e.nc
Content-Encoding:
Content-Type: application/x-download
Transfer-Encoding: chunked

3) Since ERDDAP is running on Tomcat, the only way I know of to set the
'Content-Length' is to explicitly call 'response.setBufferSize()' in the
servlet that returns the NetCDF file. Note that once the response size
goes beyond the bufferSize, Tomcat will fallback to 'Transfer-Encoding:
Chunked' (which we don't want). So make sure you're setting the buffer
size to the correct value.

Hope that helps!

p.s. I cc'd this to the netcdf-java mailing list in case I got something
wrong. Hopefully someone will correct me.

Cheers

-- B
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Brian Schlining
MBARI
Software Engineer, Research and Development
brian@xxxxxxxxx <mailto:brian@xxxxxxxxx>
(831) 775-1855

http://www.mbari.org/staff/brian

--
Sincerely,

Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
Phone: (831)658-3205
Fax: (831)648-8440
Email: bob.simons@xxxxxxxx <mailto:bob.simons@xxxxxxxx>

The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><



_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/




  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: