NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Don, Thanks for that. Don Murray wrote:
Hi James- Thanks for the test program. We find that images (radar and satellite) use a lot of memory that doesn't seem to get released. We've always figured it was something down in Java 3D (after some brief profiling). One of our next projects will be to nail this down more. Here are some observations: - when you call setSample on the FieldImpl, use the method with a copy signature and pass in copy=false. Otherwise a copy of your FlatField is made. Only do this if you are not changing the values underneath on the original FlatField for each timestep. Do you do this with your original app? Using that signature in the RadarMemoryTest really speeds up the test!
Right, it sure does. Looks like our Radar app uses copy=false in some places, but not everywhere. Will have a look at that.
- when I run this using copy=false, my max heap gets up to 89 mb and then stays there. I added in our ucar.unidata.util.MemoryMonitor widget to keep track of the memory usage and commented out it's call to run garbage collection so that wouldn't affect the results.
Yes, I see that too now, with copy=false.
- when you are running this test, what do you use for the heap size in your testing?
Was using the default (64MB). And with this setting, can get the copy=false case to produce an OutOfMemory exception too. Setting heap size to 128MB and it is ok.
- are you using the latest visad.jar (released last week) in your testing?
Have now tried the latest with same results (was using previous version earlier).
- Since you are calling disable/enableAction for every j, nothing really happens on the inner loop from the display side. So the memory use is really from the copying of the data on each set sample. Thus, the please wait only happens on each J instead of each i.
Right.
- If I add in a ScalarMap of index->Animation and use ImageRendererJ3D, my memory keeps climbing through the first few j iterations and levels off at 161 Mb max heap used, then slowly starts climbing and then finally levels off at about 200 Mb max heap. Since we use ImageRendererJ3D for our radar and satellite images, this explains some of our memory issues. Obviously, there's a serious leak in IRJ3D. But, it never goes beyond 200 even after 1500 j loops.
The radar app only uses the default renderer currently, so have been concentrating on that. We may want to move to the ImageRenderer at some point though, so interested in your comments on that. Also, our radar app definitely has animation so I'll need to put that into future tests, thanks.
And from your other message:
You're right. I was inadvertently covering up the window at various times during the testing.When I wrote this original message, I was not seeingthe memory spikes that James sees. My memory usage would level off at 89 mb. However, if I minimize the display window or have the window hidden by the readout window, the memory spikes up quickly. I'm not sure what to make of this. I suspect that Jame's variations in how many iterations it takes to run out of memory depends on how long the canvas is hidden during the tests.
Tom W, in answer to your question about "-verbose:gc" it looks like the gc is running when the window is hidden.
Cheers, James
This will be a useful program for finding where all the memory is being used, with or without IRJ3D. Unfortunately, we probably won't get a chance to look at this until late March given our schedule. Maybe someone at SSEC can look into it in the mean time. Cheers, Don James Kelly wrote:Hi All,We are having trouble with a memory leak in a real time radar application.The attached program (RadarMemoryLeak.java) also demonstrates the problem.The program starts up, reads a single radar image (from the "radar.dat" file attached), and creates an loop of 18 images from this one image (to simulate a 3 hour loop of 10 minute images). If you then hit the "enter key" (at the unix/dos prompt) the program then simulates new data arriving by: * resetting all the data (the 18 images) in the loop * calls setData on the data reference * loops 1000 times (the "i loop") * then an outer loop is called up to 10000 times (the "j loop") On all platforms (Linux, Windows and AIX) I have tested the program will eventually fail with an out of memory error.It occurs for both Java 1.4.2/Java3D 1.3.1 and Java 1.5/Java3D 1.4.0 beta.The number of loops required to crash the program is highly variable. I've seen crashes for values of j: * as small as 1 (ie 1000 calls to setData) * a lot around the 20 to 30 mark * a few in the hundreds * rarely more than 1000. So be patient if you are trying it out.Can anybody shed any light on what the cause of the memory leak might be?Many thanks, James Attached are: RadarMemoryLeak.java radar.dat To compile/run: download both files to the same directory, then: javac RadarMemoryLeak.java java RadarMemoryLeak Also are attached are text output from from a memory debugger: * start.txt: the state of the heap memory just after startup * end.txt: the state of the heap after the "out of memory" error
--James Kelly Meteorological Systems Section Bureau of Meteorology
PO Box 1289K Melbourne 3001, Australia Phone: 61-3-9669-4724 Fax: 61-3-9669-4128 Email: J.Kelly@xxxxxxxxxx
visad archives: