Re: [thredds] How are compressed netcdf4 files handled in TDS

• To: thredds@xxxxxxxxxxxxxxxx
• Subject: Re: [thredds] How are compressed netcdf4 files handled in TDS
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Mon, 25 Apr 2011 13:28:05 -0600

On 4/25/2011 11:30 AM, Roy Mendelssohn wrote:

Hi All:

We just converted one or our larger datasets  (larger in terms of the number of 
files that are aggregated) into compressed netCDF4. There is a substantial 
savings in storage, but we wanted to do a series of tests to see what hit in 
access time we would take, if any, wsince many of our users will make requests 
involving a lot of time periods.

In order to design these tests properly, we need to get a better understanding 
of how the TDS handles netcdf4 datasets that have compression.  Are the 
decompressed data cached, or more accurately cached any differently from data 
read from an uncompressed series of netcdf3 files, or since the decompression 
is handled automatically on the read, is everything handled the same after that?

We would also be interested other peoples experience with compressed netcdf4 
files in TDS, in particular when the extracts are not synoptic, but cover a lot 
of time periods in a region, or make a lot of very small calls to a large 
number of time periods  - such as we need to do for tagging data.

Thanks for any info,

-Roy

Hi Roy:

• Follow-Ups:
  • Re: [thredds] How are compressed netcdf4 files handled in TDS
    • From: Roy Mendelssohn

• References:
  • [thredds] How are compressed netcdf4 files handled in TDS
    • From: Roy Mendelssohn