Re: [netcdfgroup] Quantize Support in netCDF-Fortran: Only Bit Groom possible?

• To: "Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <matthew.thompson@xxxxxxxx>, Edward Hartnett - NOAA Affiliate <edward.hartnett@xxxxxxxx>
• Subject: Re: [netcdfgroup] Quantize Support in netCDF-Fortran: Only Bit Groom possible?
• From: Charlie Zender <zender@xxxxxxx>
• Date: Mon, 7 Nov 2022 13:54:00 -0800

Hi Matt,

Your understanding of the quantization implementation
is correct on all counts. The actual # of bits per
digit is ln(10)/ln(2) ~ 3.32. And note that netCDF
implementation defines NSB as the number of
_explicitly stored bits_, which is one less than
the number of significant bits because the IEEE
format implicitly defines the first significant bit as 1.
Thus NSB <=23 not <=24.

Charlie
--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(

• Follow-Ups:
  • Re: [netcdfgroup] [EXTERNAL] Quantize Support in netCDF-Fortran: Only Bit Groom possible?
    • From: Baker, Lawrence M

• References:
  • Re: [netcdfgroup] Quantize Support in netCDF-Fortran: Only Bit Groom possible?
    • From: Charlie Zender
  • Re: [netcdfgroup] Quantize Support in netCDF-Fortran: Only Bit Groom possible?
    • From: Edward Hartnett - NOAA Affiliate
  • Re: [netcdfgroup] Quantize Support in netCDF-Fortran: Only Bit Groom possible?
    • From: Edward Hartnett - NOAA Affiliate
  • Re: [netcdfgroup] [EXTERNAL] Re: Quantize Support in netCDF-Fortran: Only Bit Groom possible?
    • From: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]