Overview
NEP v1.0.0 marks the first major release of the NetCDF Expansion Pack,
delivering high-performance compression for HDF5/NetCDF-4 files with two
complementary algorithms: LZ4 (optimized for speed) and BZIP2 (optimized
for compression ratio).
------------------------------
New FeaturesLZ4 Compression Support
- High-Performance Compression: LZ4 compression filter for HDF5/NetCDF-4
files
- Speed Optimized: 2-3x faster compression/decompression compared to
DEFLATE
- Lossless Compression: Guarantees data integrity with no data loss
- Transparent Integration: Works seamlessly with existing NetCDF-4
applications through HDF5 plugin system
- HPC Optimized: Designed for speed-critical workflows in
high-performance computing environments
BZIP2 Compression Support
- High-Ratio Compression: BZIP2 compression filter for HDF5/NetCDF-4
files
- Compression Optimized: Better compression ratios than DEFLATE for
archival storage
- Lossless Compression: Guarantees data integrity with no data loss
- Block-Sorting Algorithm: Particularly effective for repetitive
scientific data patterns
- Archival Optimized: Ideal for long-term storage where compression
ratio is prioritized over speed
------------------------------
InstallationPrerequisites
- NetCDF-C library (v4.9+)
- HDF5 library (v1.12+)
- LZ4 library
- BZIP2 library
- CMake (v3.9+) or Autotools
- Doxygen (optional, for building documentation)
Quick Start
CMake:
cmake -B build -DCMAKE_INSTALL_PREFIX=/usr/local
cmake --build build
cmake --install build
Autotools:
./configure --prefix=/usr/local
make
make install
------------------------------
Get the release here: https://github.com/Intelligent-Data-Design-Inc/NEP
