NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Squid is a proxy server -- it supports byte-range requests, and can thus provide that service for a server than does not support byte-range requests. Configuration is a bit tricky -- you need to ensure that squid will download and cache the entire file for the initial partial requests, otherwise it will simply convert each partial request into a complete request and discard the rest. On Mon, Jun 10, 2013 at 3:12 PM, Ethan Davis <edavis@xxxxxxxxxxxxxxxx>wrote: > Hi Joe, > > HTTP 1.1 can support requests for byte-range [a] subsets of a resource. > This capability is optional so not all servers (or resources on a > server) need support it. If you look at the HTTP response headers you > get back from your [1] and [2] URLs you will see that [2] includes an > "Accept-Ranges: bytes" header whereas [1] does not. > > The netCDF-Java library can open remote datasets available over HTTP, > without downloading the entire file, if the server supports byte range > requests for that dataset. [However, depending on the access pattern, > this can be very inefficient as the client is doing the subsetting which > may require multiple HTTP requests. For OPeNDAP access, the subsetting > is done on the server and generally only requires one HTTP request.] > > Hope that helps, > > Ethan > > [a] https://en.wikipedia.org/wiki/Byte_serving > > On 6/10/2013 8:04 AM, H. Joe Lee wrote: > > Hi, John & Ethan! > > > > Thanks for the help! > > > > Now I figured it out that the "location" tag of NcML does not work > > with the on-demand NetCDF-3 file that is automatically generated by > > "fileout_netcdf" module by OPeNDAP server like [1]. If I save the > > NetCDF-3 file from [1] and put it in a regular HTTP server like [2], > > the NcML works fine. > > > > For comparison, the URL [3] is the DAP response that NCDump in > > ToolsUI can also recognize. Thus, I could make my NcML work by > > dropping .nc as Ethan suggested. Since [3] works fine with NcML, I > > have no reason to save and put it in a separate web server like [2]. > > > > However, I still don't know why OPeNDP fileout_netcdf module doesn't > > work with NcML "location". What's the key difference between [1] and > > [2]? It seems like an OPeNDAP sever configuration issue. I'm asking > > opendap-tech mailing list to get some help. > > > > Regards, > > > > > > [1] > http://eosdap.hdfgroup.uiuc.edu:8080/opendap/data/hdf4/foo2.hdf.gz.nc > > [2] http://hdfeos.org/foo2.hdf.gz.nc > > [3] dods://eosdap.hdfgroup.uiuc.edu:8080/opendap/data/hdf4/foo2.hdf.gz > > > > -- > > HDF: Software that Powers Science > > ============================ > http://eosdap.hdfgroup.uiuc.edu:8080/opendap/data/hdf4/foo2.hdf.gz.nc > > GET /opendap/data/hdf4/foo2.hdf.gz.nc HTTP/1.1 > Host: eosdap.hdfgroup.uiuc.edu:8080 > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 > Firefox/21.0 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-US,en;q=0.5 > Accept-Encoding: gzip, deflate > DNT: 1 > Connection: keep-alive > > HTTP/1.1 200 OK > Server: Apache-Coyote/1.1 > Last-Modified: Tue, 02 Oct 2012 02:58:53 GMT > Content-Disposition: attachment; filename="foo2.hdf.gz.nc" > XDODS-Server: dods/3.2 > XOPeNDAP-Server: bes/3.11.0, libdap/3.11.7, dap-server/ascii/4.1.2, > fileout_netcdf/1.1.3, hdf4_handler/3.10.1, hdf5_handler/2.1.2, > netcdf_handler/3.10.2, ncml_module/1.2.2, dap-server/usage/4.1.2, > dap-server/www/4.1.2 > XDAP: 3.2 > Content-Type: application/x-netcdf > Transfer-Encoding: chunked > Date: Mon, 10 Jun 2013 18:55:04 GMT > > ============================ > http://hdfeos.org/foo2.hdf.gz.nc > > GET /foo2.hdf.gz.nc HTTP/1.1 > Host: hdfeos.org > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 > Firefox/21.0 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-US,en;q=0.5 > Accept-Encoding: gzip, deflate > DNT: 1 > Connection: keep-alive > > HTTP/1.1 200 OK > Date: Mon, 10 Jun 2013 18:53:42 GMT > Server: Apache > Last-Modified: Mon, 10 Jun 2013 13:56:13 GMT > Etag: "a20b75-f0-4decd24c9da96" > Accept-Ranges: bytes > Content-Length: 240 > Keep-Alive: timeout=2, max=200 > Connection: Keep-Alive > Content-Type: application/x-netcdf > Content-Encoding: gzip > > > -- > Ethan Davis UCAR Unidata Program > edavis@xxxxxxxxxxxxxxxx http://www.unidata.ucar.edu > > -- > You received this message because you are subscribed to the Google Groups > "OPeNDAP Tech" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to opendap-tech+unsubscribe@xxxxxxxxxxx. > To post to this group, send email to opendap-tech@xxxxxxxxxxx. > Visit this group at > http://groups.google.com/a/opendap.org/group/opendap-tech/?hl=en. > For more options, visit > https://groups.google.com/a/opendap.org/groups/opt_out. > > > -- Dr. M. Benno Blumenthal benno@xxxxxxxxxxxxxxxx International Research Institute for climate and society The Earth Institute at Columbia University Lamont Campus, Palisades NY 10964-8000 (845) 680-4450
netcdf-java archives: