NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Correct, thanks Roy.Anything thats part of the file format is handled on the server. Standard HTTP deflate (zip) compression is used to send data across the wire, if client allows. the ncstream protocol is experimenting with more complicated schemes, but is not ready for general use.
Figuring out optimal chunking in netcdf4 is now the right game to play. It depends on the subset request, so you want to optimize your common requests.
the netcdf group would welcome user experience reports and timings. posting to this list lets us all mull over the issues.
thanks! John On 9/6/2012 6:18 PM, Roy Mendelssohn wrote:
Hi Ansley: I believe the answer is "yes", sort of. Remember, THREDDS unless you are using the netcdf subset service or another experimental service John C. is working on, does not deliver netcdf files, it delivers data through OPeNDAP, which does not have this compression as part of the description of the protocol. So the data must be uncompressed and translated to be sent. however, many OpeNDAP servers, and I believe this is true of TDS, if the client says it can receive zipped files, will zip the OPeNDAP stream and then that will be decompressed when it reaches the client. One question we (ERD) has been experimenting with is the read penalty with using netcdf4 files in conjunction with OPeNDAP, and what chunking/compression settings have the best tradeoff between performance and space I am probably off on some of this, but I believe it is basically correct. -Roy On Sep 6, 2012, at 5:04 PM, Ansley Manke wrote:Hi, Here's a question from a user of Ferret software, which is built with netcdf4 libraries. When opening and reading data from a THREDDS server, where the data is in netcdf-4 format, using compression, where does the decompression takes place: - at the THREDDS server and the uncompressed data are delivered over the web, - at the local machine, the compressed data are delivered over the web and are uncompressed at the local machine? -Ansley _______________________________________________ netcdfgroup mailing list netcdfgroup@xxxxxxxxxxxxxxxx For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. _______________________________________________ netcdfgroup mailing list netcdfgroup@xxxxxxxxxxxxxxxx For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
netcdfgroup archives: