Re: [thredds] NetCDF-4 DataReadingStrategy

• To: <rsignell@xxxxxxxx>
• Subject: Re: [thredds] NetCDF-4 DataReadingStrategy
• From: <Peter.Warren@xxxxxxxx>
• Date: Thu, 1 Sep 2016 01:04:02 +0000

Hi Rich,

Thanks for the response.

Assuming the Unidata guys are ok with the proposed solution, I'll setup a pull 
request for ncwms and edal-java.  

I notice in edal-java code there is a TODO stating that the default reading 
strategy chooser should take into account the size of the source grid.  I had a 
brief look at implementing this but it's non-trivial for complicated grids.

Cheers
Pete

-----Original Message-----
From: Signell, Richard [mailto:rsignell@xxxxxxxx] 
Sent: Thursday, 1 September 2016 12:07 AM
To: Warren, Peter (Mineral Resources, North Ryde) <Peter.Warren@xxxxxxxx>
Cc: THREDDS community <thredds@xxxxxxxxxxxxxxxx>
Subject: Re: [thredds] NetCDF-4 DataReadingStrategy

Peter,
The THREDDS Data Server code is now here:
https://github.com/Unidata/thredds/
so you could fork it and submit a pull request for this change if it hasn't 
already been handled in ncWMS2, which will be coming out with TDS 5.0, I think 
in October?

Having said this, I'm sure the Unidata guys will now chime in here to
correct me.   ;-)

-RIch

On Wed, Aug 31, 2016 at 1:56 AM,  <Peter.Warren@xxxxxxxx> wrote:
> Hi Thredds List,
>
>
>
> My colleagues and I (at CSIRO and Geoscience Australia) have been 
> using a patched version of thredds 4.2 for over 5 years to deliver 
> satellite earth observation data.  We offer the data in as many 
> formats as possible but our most used protocol is WMS for 
> visualisation as overlays on maps.  A the time (around 2011) we had to 
> patch thredds because we use relatively large (64gb)
> netCDF-4 source files which are too large to load into memory using 
> the default BOUNDING_BOX reading strategy.  Xiangtan Lin modified the 
> getOptimumDataReadingStrategy method in ncwms to make NetCDF-4 type 
> datasets default to SCANLINE reading strategy.  This reduced the 
> memory footprint sufficiently to get the wms layers working.  See discussion 
> here:
> http://www.unidata.ucar.edu/mailing_lists/archives/thredds/2011/msg003
> 28.html
>
>
>
> We are now consolidating this thredds instance into a large corporate 
> system and were hoping we could get this change included in the master 
> source so we can keep up with the current releases without having patch each 
> new one.
> Comments in the source seem to suggest that SCANLINE is the intended 
> default for NetCDF files but the existing check only catches versions prior 
> to 4.
>
>
>
> So, would it be possible to have this change made to the ncwms master? 
> I can make the changes and setup a GitHub pull request if that helps.
>
>
>
> Thanks
>
> Peter Warren
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are 
> recorded in the Unidata inquiry tracking system and made publicly 
> available through the web.  Users who post to any of the lists we 
> maintain are reminded to remove any personal information that they do 
> not want to be made public.
>
>
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598

• Follow-Ups:
  • Re: [thredds] NetCDF-4 DataReadingStrategy
    • From: Signell, Richard

• References:
  • [thredds] NetCDF-4 DataReadingStrategy
    • From: Peter.Warren
  • Re: [thredds] NetCDF-4 DataReadingStrategy
    • From: Signell, Richard