NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [thredds] Using HDF5's dynamic loading filter in THREDDS

  • To: Christian Ward-Garrison <cwardgar@xxxxxxxx>
  • Subject: Re: [thredds] Using HDF5's dynamic loading filter in THREDDS
  • From: Michael McDonald <mcdonald@xxxxxxxxxxxxx>
  • Date: Thu, 28 Apr 2016 12:33:50 -0400
Adam and Christian,

HYCOM.org just needed to build netcdf+hdf5 from source after
discovering a fatal bug in the latest RHEL6 hdf5 (from epel), which
was crashing our NCSS NetcdfSubsetService servers. The only part we
*added* to the NetCDF-4 C Library Loading reference
(http://bit.ly/1QD34bK) - in addition to "--enable-threadsafe" - was
"--enable-unsupported" in the build of hfd5.

1st remove the rpm build of netcdf+hdf5 from RHEL yum (if that's what
you are using. as we were).

yum remove netcdf netcdf-static netcdf-devel hdf5 hdf5-static hdf5-devel


Then download and build hfd5 1st and then netcdf 2nd from source.

cd /usr/local/src
wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.16.tar.gz
tar -zxf hdf5-1.8.16.tar.gz && cd hdf5-1.8.16
./configure --prefix=/usr --enable-threadsafe --enable-unsupported &&
make && make check && make install

ldconfig

cd /usr/local/src
wget https://github.com/Unidata/netcdf-c/archive/v4.4.0.tar.gz
tar -zxf v4.4.0.tar.gz && cd netcdf-c-4.4.0
./configure --prefix=/usr && make && make check && make install


After restarting tomcat the new netcdf bins/libs are working fine to
stage netcdf4 files to cache again for NCSS, and more importantly no
longer crashing tomcat in the process.

On Thu, Apr 28, 2016 at 2:55 AM, Christian Ward-Garrison
<cwardgar@xxxxxxxx> wrote:
> Hi Adam,
>
> THREDDS uses the NetCDF C library to write (but not read) NetCDF-4 files.
> This page [1] describes the procedure that THREDDS uses to locate the shared
> library on the system. Have you successfully built netcdf while linking
> against your custom hdf? Does your hdf require any changes to netcdf in
> order to take advantage of the new features?
>
> Cheers,
> Christian
>
> [1]
> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/netcdf4Clibrary.html
>
> On Wed, Apr 27, 2016 at 5:51 PM, Adam Steer <Adam.Steer@xxxxxxxxxx> wrote:
>>
>> Hi all
>>
>> I’ve got a request to enable HDF5’s dynamic loading filters capability in
>> a TDS. My colleagues have built a HDF5 C library which has dynamic loading
>> filters enabled, required because the standard HDF5 compression algorithm
>> isn’t working as well as some others they’ve tested and they want to replace
>> it. I can’t answer any more about the whys and wherefores of that - but the
>> task from my point of view is this:
>>
>> How can I get THREDDS to use a customised HDF5 library?
>>
>> As far as I can tell, I’d need to build THREDDS from source, with the new
>> HDF5 library on whatever system I’m using to build it.
>>
>> …but this strategy might be completely wrong. I’m uncertain about the
>> relationship between system NetCDF/HDF5 libraries and the Java-netCDF that
>> THREDDS uses.
>>
>> …which brings my second question: Would using a customised HDF5 library at
>> build time have the desired effect - being the ability to use HDFs dynamic
>> loading filters when interacting with netCDF 4 files?
>>
>> I hope my questions make sense - I’m feeling my way about THREDDS, NetCDF4
>> and HDF5 all at the same time.
>>
>> Thanks in advance
>>
>> Adam
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/



-- 
Michael McDonald
Florida State University



  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: