Re: question about float to int type conversion...

• To: netcdf-hdf@xxxxxxxxxxxxxxxx
• Subject: Re: question about float to int type conversion...
• From: Ed Hartnett <ed@xxxxxxxxxxxxxxxx>
• Date: Mon, 27 Jun 2005 09:32:11 -0600

Robb Matzke <matzke@xxxxxxxx> writes:

>>>>>> "Ed" == Ed Hartnett <ed@xxxxxxxxxxxxxxxx> writes:
> Ed>    float float_data_out2[DIM1_LEN] = {(float)INT_MIN - 1.0, 
> (float)INT_MIN - 1.0, (float)INT_MAX + 129.0};
>
> Hi Ed,
>   Looking at your output, you're on a machine where
> sizeof(float)==sizeof(int)==4. So the initialization of
> float_data_out2[] is not doing what you expect. The problem is that
> the significand of a float is less than 32 bits (because the sign and
> exponent take up some of the other bits). So the +1.0 and -1.0 have
> too small a magnitude to make any difference in the arithmetic. The
> loss of significant digits can be seen in your output: INT_MIN-1.0
> should be -2147483649, not -2147483648.
>
> Ed> So why doesn't it generate an overflow?
>
> Because by time HDF5 gets the data there is no overflow for your first
> two array values. Using `double' should fix the problem because it can
> represent all 32 bits of an `int'.
>
> -- 
> Robb Matzke L-159   | Ph:  812-967-6325  | I/O, I/O, it's off to disk we go,
> Lawrence Livermore  | Fax: 812-967-6326  | A gigabyte to read or write,
> National Laboratory | Hrs: 3am-11am PDT  | I/O, I/O.
>
>

Thanks very much - I should have seen that myself!

Ed
-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx

• References:
  • question about float to int type conversion...
    • From: Ed Hartnett
  • question about float to int type conversion...
    • From: Robb Matzke