[netcdf-java] ERDDAP bug

• To: bob.simons@xxxxxxxx
• Subject: [netcdf-java] ERDDAP bug
• From: Brian Schlining <bschlining@xxxxxxxxx>
• Date: Thu, 11 Oct 2012 09:31:31 -0700

Hi Bob,  

We're doing some work with ERDDAP and running into an issue using NetCDF-Java 
to access files served by ERDDAP. I think I understand the issue and know how 
to address it, so I'm passing the info onto you all so it can be addressed. So 
here goes:

1) ERDDAP allows one to download a NetCDF file by building a link appended with 
'.nc'. The link URL for the netcdf file would be something like 
http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc. This works great for 
downloading the files. However, it does NOT work with the NetCDF-Java API; 
NetCDF-Java can normally read NetCDF files from arbitrary non-opendap http urls.

2) The reason it fails is because NetCDF-Java needs to know the size of the 
file being served. This requires that the HTTP response for a URL like 
http://beach.mbari.org:8180/erddap/griddap/erdRyanSST.nc to contain a 
'Content-Length' field. ERDDAP is not sending that … here's a response header 
from ERDDAP (notice there's no 'Content-Length':

  HTTP/1.1 200 OK
  Server: Apache-Coyote/1.1
  Date: Thu, 11 Oct 2012 15:44:19 GMT
  Last-Modified: Thu, 11 Oct 2012 15:44:19 GMT
  xdods-server: dods/3.7
  erddap-server: 1.38
  Content-Disposition: attachment;filename=erdRyanSST_8571_f367_229e.nc
  Content-Encoding:  
  Content-Type: application/x-download
  Transfer-Encoding: chunked


3) Since ERDDAP is running on Tomcat, the only way I know of to set the 
'Content-Length' is to explicitly call 'response.setBufferSize()' in the 
servlet that returns the NetCDF file. Note that once the response size goes 
beyond the bufferSize, Tomcat will fallback to 'Transfer-Encoding: Chunked' 
(which we don't want). So make sure you're setting the buffer size to the 
correct value.

Hope that helps!

p.s. I cc'd this to the netcdf-java mailing list in case I got something wrong. 
Hopefully someone will correct me.

Cheers

-- B
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Brian Schlining
MBARI
Software Engineer, Research and Development
brian@xxxxxxxxx (mailto:brian@xxxxxxxxx)
(831) 775-1855

http://www.mbari.org/staff/brian

• Follow-Ups:
  • Re: [netcdf-java] ERDDAP bug
    • From: John Caron
  • Re: [netcdf-java] ERDDAP bug
    • From: Tom Kunicki