NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
I will take a look at that test in netcdf-fortran... I guess we all should have collaborated more and agreed to a common configure option name for the plugin path. :-( Now that the quantize and zstandard are in netcdf-c/netcdf-fortran, CCR does not have anything additional to offer at this time. (But BLOSC is coming for CCR...) WRT zstandard, here's a chart from a recent AGU presentation: Quantization and Next-Generation Zlib Compression for Fully Backward-Compatible, Faster, and More Effective Data Compression in NetCDF Files, https://www.researchgate.net/publication/357000984_Quantization_and_Next-Generation_Zlib_Compression_for_Fully_Backward-Compatible_Faster_and_More_Effective_Data_Compression_in_NetCDF_Files [image: image.png] On Tue, Aug 2, 2022 at 2:20 PM Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <matthew.thompson@xxxxxxxx> wrote: > Ed, > > > > Well, I suppose it was inevitable and I haven't made a massive change in > GEOS' build system in a while... > > > > I went back to my last attempt to build CCR (someday I'll get it > working!), and I think I need to add: > > > > 1. HDF5: --with-default-plugindir=$(HDF5_PLUGIN_PATH) > 2. netCDF-C: --with-plugin-dir=$(HDF5_PLUGIN_PATH) > 3. NCO: --with-hdf5-plugin-path=$(HDF5_PLUGIN_PATH) > > > > I suppose I should ask Charlie Zender about the last one...but it seems > right. > > > > Now for some reason I can't get the "make check" step for netCDF-fortran > to recognize HDF5_PLUGIN_PATH, so the zstandard tests fail...but I can run > them by hand. Not sure if it's on my end or I'm not passing things down > right to the check step? :shrug: > > > > Now onto testing it. Be prepared for questions like "What are good values > to set zstandard_level and quantizing and ...". (Well, first I'll go read > your papers and the tests in the code, but then I'll probably annoy you :D ) > > > > Matt > > > > -- > > Matt Thompson, SSAI, Ld Scientific Programmer/Analyst > > NASA GSFC, Global Modeling and Assimilation Office > > Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 > > Phone: 301-614-6712 Fax: 301-614-6246 > > *http://science.gsfc.nasa.gov/sed/bio/matthew.thompson > <http://science.gsfc.nasa.gov/sed/bio/matthew.thompson>* > > > > *From: *Ed Hartnett <edwardjameshartnett@xxxxxxxxx> > *Date: *Tuesday, August 2, 2022 at 1:03 PM > *To: *"Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" > <matthew.thompson@xxxxxxxx> > *Cc: *"netcdfgroup@xxxxxxxxxxxxxxxx" <netcdfgroup@xxxxxxxxxxxxxxxx> > *Subject: *[EXTERNAL] Re: [netcdfgroup] Can I build zstandard support in > netCDF with a static build? > > > > Howdy Matt! > > > > For zstandard, and other filter features, you must build and use shared > libraries. > > > > Also you need to build netcdf-c with --with-plugins=/some/path and then > set environment variable HDF5_PLUGIN_PATH to /some/path before you build > netcdf-fortran (or use any program with the zstandard filter). > > > > Ed > > > > On Tue, Aug 2, 2022 at 10:57 AM Thompson, Matt (GSFC-610.1)[SCIENCE > SYSTEMS AND APPLICATIONS INC] via netcdfgroup < > netcdfgroup@xxxxxxxxxxxxxxxx> wrote: > > All, > > > > I recently grabbed netCDF-C v4.9.0 and netCDF-Fortran v4.6.0 and thought, > heck, let's try to get zstd support as well! zstd built just fine and when > I built netCDF-C it seemed pretty happy: > > > > ZSTD Support: yes > > > > but then when I tried to build netCDF-Fortran: > > > > configure: WARNING: ------------------------------------------ > > configure: WARNING: libnetcdf was built with zstd support, but > HDF5_PLUGIN_PATH is not set. > > Either set the HDF5_PLUGIN_PATH environmental variable if zstandard > tests fail, > > or use --disable-zstandard-plugin when running configure. > > configure: WARNING: ------------------------------------------ > > > > Okay. New to me. I then went back to the netCDF-C build and saw: > > > > checking whether dynamically loaded plugins is enabled... yes > > configure: WARNING: --disable-shared => --disable-plugins > > ... > > checking whether and where we should install plugins... no > > ... > > Plugin Install Prefix: N.A. > > > > Seeing that warning reminded me I build netCDF-C and netCDF-Fortran as a > static library mainly for historical reasons (was static in the past and, > well, if it works, don't mess with it unless you need to!) > > > > So I suppose the question is: Can I use zstd with a static build of netCDF? > > > > Or is it time to bite the bullet and reconfigure my software to use shared > libraries... > > > > Matt > > > > -- > > Matt Thompson, SSAI, Ld Scientific Programmer/Analyst > > NASA GSFC, Global Modeling and Assimilation Office > > Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 > > Phone: 301-614-6712 Fax: 301-614-6246 > > *http://science.gsfc.nasa.gov/sed/bio/matthew.thompson > <http://science.gsfc.nasa.gov/sed/bio/matthew.thompson>* > > _______________________________________________ > NOTE: All exchanges posted to Unidata maintained email lists are > recorded in the Unidata inquiry tracking system and made publicly > available through the web. Users who post to any of the lists we > maintain are reminded to remove any personal information that they > do not want to be made public. > > > netcdfgroup mailing list > netcdfgroup@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > https://www.unidata.ucar.edu/mailing_lists/ > <https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.unidata.ucar.edu%2Fmailing_lists%2F&data=05%7C01%7Cmatthew.thompson%40nasa.gov%7Cc26dbcf169054901b36308da74a8fb15%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637950566361758136%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vLPKypJxoyONwV0DqbPbTFlxXTt7Ao9n0Ff7QBAAliU%3D&reserved=0> > >
netcdfgroup archives: