NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: IDV performance

Thank you very much for responding so quickly!  Yes, I will send the
data file in a few minutes.

Mike

Don Murray wrote:

Hi Michael-

I don't think you are doing something wrong.  It's just taking the
IDV a while to generate and render the glyphs.

Could you send your data file to support-idv@xxxxxxxxxxxxxxxx and
we can take a look at what's going on?

Thanks.

Don Murray

Michael Zeleznik wrote:

I have recently started working with the IDV, and have a question about
performance and memory usage.
Although new to the IDV, I have worked for many years with OpenDX (and IBM 
Data Explorer before it was open sourced).  I figure I must be doing
something terribly wrong with the IDV, since it runs so slowly, and uses so 
much memory, for a very simple visualization.

Thank you in advance for any help, or pointers to info, you might offer.

Here is my simple test visualization:

----------------------------------------
THE DATA

272 stations, each located at a (latitude, longitude).

Each station produces data consisting of 153 integer values, measured at
uniform time intervals, from a start time to a stop time.

The start time, stop time, and time interval are identical for all
stations.

I provide this data to IDV as an ASCII CSV file of the following format.
Now, I do realize that this format does not tell the IDV anything about the 
relationship among data elements, which it could probably utilize to be
more efficient, but this was the simplest way to initially get the data in 
(and the amount data is not large).

CSV FORMAT:

(index) -> (Time,Latitude,Longitude,Magnitude)
Time[fmt="yyyy-MM-dd HH:mm:ss"],Latitude[unit="deg"],Longitude[unit="deg"],Magnitude[unit="null"] 2006-11-15 11:33:16, 32.888901, -117.105103, 85 // Start of 1st station's data values 
2006-11-15 11:33:26, 32.888901, -117.105103, 55
[another 151 entries with the same Latitude and Longitude,
 at uniform time steps, with different Magnitudes]
2006-11-15 11:33:16, 48.719700, -122.707001, 802 // Start of 2nd station's data values 
2006-11-15 11:33:26, 48.719700, -122.707001, 580
[another 151 entries with the same Latitude and Longitude, at
the same uniform time steps as the first station, with different Magnitudes] 

... and so on for all 272 stations.

----------------------------------------
THE VISUALIZATION

I plot these 272 stations, each as a small glyph (filled circle) on a 2D
map. I have 3D Display unchecked in preferences.  The glyph color is
determined by a color map, mapping Magnitude into color. As I step through the time steps, the set of glyphs are colored according to the Magnitude at 
each station at that particular time.

----------------------------------------
THE PERFORMANCE

Running on Linux (Fedora Core 3), on a 3.2 GHz P4 with 1GB memory
  IDV 2.2
  java version "1.5.0_11" (the one that is packaged with IDV 2.2)
  java set to start with 750 MB.
It takes over 2 minutes to just create the first display of all stations, 
and uses 590 MB of memory.  If I use a simple square glyph instead of a
round one, it uses 400 MB. A simple + glyph uses 370 MB.

Once the display is created, stepping through the time steps is fast.

If I change the color map, it will take many more minutes to finish the
display, and will nearly double the memory usage.  It does not seem to
matter if I disable or enable caching.

Can anyone offer any suggestions on what to do?

Thanks again,
Mike

Michael Zeleznik
zeleznik@xxxxxxxxxxxxxxx
801-485-1106
  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the idvusers archives: