NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [galeon] WCS CF-netCDF profile document

Roy,

this comment is very intriguing

'John Caron implied that at the GO- ESSP meeting that he had code that could 
stream netcdf-3, but that it would be unlikely that netcdf4  (or hdf5) could be 
streamed.'

am I correct in thing that hdf5 supports choosing the compression method, so I 
could in fact choose Jpeg2000 compression, well then the coverage encoding is 
netcdf or hdf, but the protocol (as specified by the mime type in the output 
formats) could be opendap, application/x+jpip+xml and the jpip extension will 
stream this coverage encoding directly.  However the caveat is that a coverage 
encoding for specifying a gmljp2 to hdf mapping would be required.  The JPIP 
server could stream the jpeg2000 codestream from an hdf file without 
modification but the payload would be meaningless without the metadata.

The protocol isn't so relevant, the coverage encoding is.

Norman



-----Original Message-----
From: galeon-bounces@xxxxxxxxxxxxxxxx on behalf of Roy Mendelssohn
Sent: Wed 9/24/2008 9:09 AM
To: Ben Domenico
Cc: galeon@xxxxxxxxxxxxxxxx; Steve Hankin; Jon Blower
Subject: Re: [galeon] WCS CF-netCDF profile document
This is an interesting question, and it is related to the larger question of whether it is better to transfer files as mime-types or better to have a binary protocol to transfer the information, in particular if we envision very large amounts of data being transferred. I know the OOI people are very concerned about this issue, in fact are pushing the idea of a modified OPeNDAP that uses the Advanced Messaging Queueing Protocol (AMQP see http://jira.amqp.org/confluence/display/AMQP/Advanced+Message+Queuing+Pro 
tocol)
as what gets put over the wire, a messaging protocol that is in heavy use in the financial industry to transfer real-time their transaction data (maybe we can get a $700 million bailout also!).
Seriously, one downside of netcdf delivered as a mime-type is that you can't stream the service - you have to hold the file in cache until it is complete and then send the result. OpeNDAP can be streamed, though not many OpeNDAP servers do that. John Caron implied that at the GO- ESSP meeting that he had code that could stream netcdf-3, but that it would be unlikely that netcdf4 (or hdf5) could be streamed. 

Interesting discussion.

-Roy