Re: Problems with precision & ScalarMap

• To: Timo Thomas <tthomas@xxxxxxxxxxxxxxxxxxxxxxxx>
• Subject: Re: Problems with precision & ScalarMap
• From: Don Murray <dmurray@xxxxxxxxxxxxxxxx>
• Date: Wed, 23 May 2001 09:14:18 -0600

Hi Timo-

By default, the DefaultSet for the range values in a FlatField is a
FloatSet.  You can change the default set in 1 of two ways:

1) use the setDefaultSet() method in RealType:

    height = new RealType("height", SI.meter);
    height.setDefaultSet(new DoubleSet(height));

or 

2) use the rangeset parameter in the FlatField constructor:

    h_vals_ff = new FlatField( func_t_h, time_set, (CoordinateSystem[])
null, new Set[] { new DoubleSet(height) }, (Unit[]) null );

That will fix your rounding error.

However, the problem you are seeing with the long pause seems to be
related to computing the tick marks in the AxisScale.  I'm looking into
this and will post a fix when I have one.

Don Murray

Timo Thomas wrote:
> 
> Hi list,
> 
> the attached slightly modified P2_01 example doesn't work. There seems to be
> a problem with big data values. If I don't set the range of heightMap
> explicitly, the ScalarMap isn't initialized to the correct min/max data
> values (it rounds them somehow). If I set it, it takes minutes before a
> window is shown and there's only a point shown instead of a line.
> Maybe there's an unintended double to float conversion happening internally?
> 
> Timo
> --
> __.__________
>  |lmo |homas
> 
>   ------------------------------------------------------------------------
>                  Name: P2_01.java
>    P2_01.java    Type: text/x-java
>              Encoding: 7BIT

-- 
*************************************************************
Don Murray                               UCAR Unidata Program
dmurray@xxxxxxxxxxxxxxxx                        P.O. Box 3000
(303) 497-8628                              Boulder, CO 80307
http://www.unidata.ucar.edu/staff/donm
*************************************************************

• Follow-Ups:
  • Re: Problems with precision & ScalarMap
    • From: Don Murray

• References:
  • Problems with precision & ScalarMap
    • From: Timo Thomas