Re: Program to visualize layers

• To: visad-list@xxxxxxxxxxxxx
• Subject: Re: Program to visualize layers
• From: "William L. Hibbard" <WHIBBARD@xxxxxxxxxxxxx>
• Date: Fri, 12 Mar 1999 14:16 CST

Mohamad,
 
> Representing each layer as a Linear2DSet works fine for displaying
> surfaces and iso-contour lines.  I've developed programs similar to
> Ugo's for displaying geological layers and properties.  Hopefully I'll
> have some demos available soon.
 
Great, I look forward to seeing them.  I am always on the lookout
for web links from the VisAD page.
 
> For iso-surfaces and fancier volume operations (like slicing), we need
> to have a Linear3DSet.  This requires a regualr XYZ grid.  But we don't
> have that grid readily available when our only data are the layers given
> by Z values (and properties) on a requral xy grid.  To define the grid
> for Z axis, I setup a z axis to have as many values as I have z values
> on my layers, and defined a mapping of its index to the sorted layer z
> values.  In this way, I can define a regular Linear3DSet volume and set
> its elements using the layers data.  Ofcourse majority of this volume
> data will be missing and so its a good idea to initialize the volume
> elements to -1 first.
 
If your data have Z as a function of (X, Y) it would be nice to find
a way to solve your problem without using Fields with 3-D domains.
Given a Field with MathType ((X, Y) -> Z) you can display it as a
surface with the ScalarMaps X -> XAxis, Y -> YAxis, Z -> ZAxis.
You can also use ConstantMaps to ZAxisOffset in the addReference call
for each data object to create a set of stacked terrain surfaces.
However, I'm not sure if this is what you want.
 
> As I was happily persuing my dreams of setting up a 3D volume (given
> only layer data as above) using VisAD, I ran into memory and efficiency
> problems, ofcourse.  For my data, different layer z values easily add up
> to thousands.  Looking deeper into VisAD as to where and why, I learned
> that domains are repersented as arrays even for IntegerSets!  And so
> memroy is used up for going through index values (like 0, 1, 2, ...).
> Java's Iterators would be perfect for this.  Instead of using arrays for
> linear set domains, VisAD could use iterators that know the domain
> boundary and step values.  For  a 100x100x3000 grid this could make a
> big difference.  For the VisAD gurus (me not being one, so this could be
> all wrong:), I am talking about things like indexToValue() in
> Linear3DSet and other things.  So instead of:
>
>
>   /** convert an array of 1-D indices to an array of values in
> R^DomainDimension */
>   public float[][] indexToValue(int[] index)
>
> It could be:
>
>   public Iterator[] indexToValue(Iterator index)
>
> As for as I can tell, using customized iterators for representing and
> indexing through linear domains will simplify and reduce memory usage.
> Does this make sense?  I'd be interested to hear your thoughts on this
> and the volume issues.
 
Actually, LinearSets do not internally contain arrays of values.
However, they extend the indexToValue() method of Set and its
signature does require the return of an array, since other Set
subclasses include "curvey" GriddedSets and totally "messy"
IrregularSets that must be enumerated in an array rather than
computed by formula.  Also, an early assumption of VisAD was to
avoid any method calls once per data item (e.g., image pixel or
grid point), as are made with iterators.
 
So the storage of data objects is quite efficient, but the display
process can use lots of memory for temporary storage for arrays.
This also can happen when Fields with different domain Sets are
combined mathematically, such as subtracting images from
different satellite perspectives.
 
We are planning to add special case logic to the display process
to use less memory when MathTypes and ScalarMaps are simple and
Sets are LinearSets.  The general display logic that is currently
used has to be able to handle lots of situations: for example,
a user may define new spatial DisplayRealTypes and a new
CoordinateSystem relating them to XAix, YAxis & ZAxis, then
ScalarMap the components of a LinearSet to those new
'coordinates' - the resulting display X, Y and Z values will
not form a LinearSet and must be carried in an array.  Or the same
thing can happen with ScalarMaps to Display.Latitude,
Display.Longitude and Display.Radius.  Special display cases
can be defined for circumstances when this does not happen, to
save on memory.
 
In a sense, the Set is the iterator.  Our plans for special
cases are to avoid dumping arrays of values out of those
'iterators', so we will essentially be following your
suggestion.
 
If there is a way you can organize your data and displays that
does not require Fields with 3-D domains, that could help.  I'd
be happy to read a description of your data and of the displays
you want to generate, and see if we can find a way to reduce
memory use.
 
You have bit on the number one current problem in VisAD: large
temporary memory use in the display process.  However, we do
know how to address this problem, and it is only a matter of
finding the time to do it.  Since DataRenderer is extensible,
it is possible for anyone to write code suited to a particular
data organization and display mapping (although it is a bit
difficult).  We will make headway on this problem as soon as
we can.
 
Cheers,
Bill
 
----------------------------------------------------------
Bill Hibbard, SSEC, 1225 W. Dayton St., Madison, WI  53706
whibbard@xxxxxxxxxxxxx  608-263-4427  fax: 608-263-6738
http://www.ssec.wisc.edu/~billh/vis.html
 

• Follow-Ups:
  • Re: Program to visualize layers
    • From: Ugo Taddei