• View
• Edit
• History
• Print

MemoryLeaks

The nightly build opens about 660 demos, runs many of them and exports them all.

The problem is that we are running out of memory.

See Also

• Code Generation and Memory Leaks
• Dealing with memory leak in Kepler - a good tutorial about how to use jvisualvm

Summary

We use jvisualvm to connect to a running Ptolemy process and then view the heap. See Tracking down leaks using JVisualVM

Mac Users: To avoid leaks, you will want to make two edits to your ptolemy/gui/Top.java that are not checked in, see:

• MacOSXAdapter
• MacOSX CPlatformWindow

Analysis of the nightly build

Here are the scripts used to get the memory stats from the export run in the build

Here's how the commands were run

Here's the plot, the x axis is the model number, the y axis is the amount of memory. The red is from build 301, before the leak fixes, the blue is from build 317, after the fixes below. The spikes into negative territory are because the scripts are primitive and fooled by output of some of the runs.

From the above, we can see that memory use in the 301 run (in the red) is increasing over time. The garbage collector helps, but there are leaks.

My first idea was to make sure that opening a new model and closing it was not leaking memory.

Tracking down leaks using JVisualVM

Below are some random notes about tracking down memory leaks using JVisualVM.

The test case was

1. Invoke $PTII/bin/vergil
2. File -> New -> Graph Editor
3. Close the window
4. Then, in jvisualvm, click on the vergil process
5. In the Monitor tab, click on Perform GC and the Heap Dump
6. Click on Classes and then in the Class Name Filter at the bottom, enter ActorGraphFrame and hit return
7. The number of instances should be 0.

Non-zero number of ActorGraphFrame instances

There are a number of different ways that the ActorGraphFrame might avoid being garbage collected. What's happening is that something has a reference to the object and the garbage collector is not collecting it.

Details are below

Listeners have references to the object

Dealing with memory leak in Kepler covers this. Basically, the dispose() methods need to call methods in ptolemy.actor.gui.MemoryCleaner.

_findInLibraryyEntryBox

I added the following to BasicGraphFrame.dispose()

BasicGraphFrame$MoveToFrontAction

To remove the reference to BasicGraphFrame$MoveToFrontAction, I added the following to BasicGraphFrame.dispose()

BasicGraphFrame$HierarchyTreeCellRenderer

BasicGraphFrame$HierarchyTreeCellRenderer is an inner class that had a reference to BasicGraphFrame.

To fix this, I added the following to BasicGraphFrame.dispose()

The _treeView.setUI(null); was the key statement.

MacOSXAdapter

MacOSXAdapter and JPopupMenu were appearing in the list of references.

It turns out that we are using some deprecated methods. My temporary workaround was to temporarily comment out Top._macInitializer():

We should implement a more permanent fix here.

MacOSX CPlatformWindow

Under MacOSX, CPlatformWindow had references that prevented ActorGraphFrame from being GC'd.

To replicate

1. Get a heap dump as above and search for instances of ActorGraphFrame
2. In instances pane, right click and select "Show Nearest GC Root"
   
3. Then, in the References pane, note that CPlatformWindow has a reference
   

It turns out that under MacOSX, there are leaks. http://stackoverflow.com/questions/19781877/mac-os-java-7-jdialog-dispose-memory-leak describes the problem with JDialogs. I adapted that code and added to ptolemy.gui.Top.dispose():

Unfortunately, the above code is Mac-specific and compiling it produces a warning:

  Top.java:382: warning: ComponentPeer is internal proprietary API and may be removed in a future release
        java.awt.peer.ComponentPeer peer = getPeer();

So, that code is left commented out.

Linux BltSubRegionBufferStrategy

Unfortunately, under Linux, we get a different set of leaks.

If we use jvisualvm under RHEL with Java 1.8.0_51, do the procedure above and then "Show nearest GC Roots", we get

Searching for CurrentFocusCycleRoot leak yielded http://oracle.developer-works.com/article/5359339/How+to+realy+unregister+listeners+%28or+how+%22great%22+swing+leaks+memory%29, which says:

So, I modified

That seems to have done it.

Manager

To replicate:

1. $PTII/bin/vergil
2. Open a model, run it and close the window
3. In jvisualvm, find the vergil process and create a head dump:
   1. Monitor tab
      1. Click on Perform GC
      2. Click on Heap Dump
   2. In the heapdump tab:
      1. Enter Manager in the search window at the bottom.
4. Note that once instance is present. Right click on the instance and select "Show Nearest GC Root"
5. Note that the leak is in _readerRecords in Workspace

(Note that this run was done with the contents of Manager._shutdownHooks() commented out, see below.)

The Manager has a thread, which is an inner class.

While in that thread, when we want to plot, we initialize an Effigy. We end up trying to get read access on the directory Entity, that then adds the Manager thread to the Workspace._readerRecords hashmap. Note that this is a different Workspace than the workspace of the model.

The problem is that when we want to set Manager._thread to null, there is no easy way to find all of the Workspaces and then remove the reference from each _readerRecord.

Edward suggested that the thing to do is to change _readerRecord to a WeakHashMap. This solved the problem.

Manager and shutdownHooks

It turns out that the shutdown hook code in Manager leaks.

If we comment out the contents of Manager._registerShutdownHook(), and we have the above change to _readerRecords, then we have no leaks.

To replicate:

1. $PTII/bin/vergil
2. Open a model, run it and close the window
3. In jvisualvm, find the vergil process and create a head dump:
   1. Monitor tab
      1. Click on Perform GC
      2. Click on Heap Dump
   2. In the heapdump tab:
      1. Enter Manager in the search window at the bottom.
4. Note that once instance is present. Right click on the instance and select "Show Nearest GC Root"
5. Note that the leak is in ApplicationShutdownHooks:

If we edit Manager, comment out the body of _registerShutdownHook() and rerun the above test, then there are no instances of Manager.

So, what we need to do is remove the shutdown hook when we destroy the manager.

The solution here was to change _registerShutdownHook() so that it sets a private variable to the value of the thread and then passes that value to Runtime.addShutdownHook(). Then when we are setting _thread to null, we also call Runtime.removeShutdownHook().

Edward suggested:

Checking for leaks in ExportModelJUnitTest

The overview is that we run ptjacl within Eclipse using a short version of models.txt. Eclipse is set up to break on exit so that we can then use jvisualvm to check for leaks.

The reason we use ptjacl is because it is not opening up Vergil so we don't end up debugging vergil, instead we are closer to debugging how ant runs things. An alternative would be to invoke ant from within Eclipse and debug that. With ptjacl, we need to use the debugger so that we can break on exit() because JUnit invokes exit()

Details:

1. Edit $PTII/ptolemy/configs/doc/models.txt and remove most of the models. A good number of models to run is roughly 8. Later, to get this file back, remove it and do a svn update on it.
2. In Eclipse, create a debug configuration that will invoke tcl.lang.Shell
3. In Eclipse, search for System in the JRE and then set a break point at the start of the exit() method. We need to do this because JUnit calls exit(). There is probably a way around this, but this works.
4. Start up the debugging session and enter:
   set cmdArgs [java::new {java.lang.String[]} 1 {ptolemy.vergil.basic.export.test.junit.ExportModelJUnitTest}]
   java::call org.junit.runner.JUnitCore main $cmdArgs
   

   [$[Get Code]]
   What the above does is create an array of Strings and then passes it to the main(String[]) method of JUnitCore
5. The models will be opened and export
6. When the models are done, use jvisualvm to look for the largest items on the heap.

Below is the sample run of 8 models

• Double clicking on the Plot instance and then right clicking and selecting "Show Nearest GC Root" indicates that the defaultUncaughtExceptionHandler is involved:

The solution to this was modify MessageHandler so that _statusHandler is a WeakReference:

• It also looks like we are leaking Configuration objects, there is one per model. Below is the output of "Show Nearest GC Root":

Calling _controller.setConfiguration(null); in ActorGraphFrame.dispose() helped, but then _configurations in Configuration.java was holding a reference.

The fix for that is in ConfigurationApplication.closeModelWithoutSavingOrExiting

• If we look at the 40 largest objects, we see that SCRModel is also present?

Another thing is to add -XX:+HeapDumpOnOutOfMemoryError to the command line so that we dump the heap when we run out of memory. To do this, I edited $PTII/build.xml.in:

Manager again: _treeViewModel

After a few weeks, opening vergil, running the Noise model and closing the Noise model showed a leak again:

_treeViewModel seems to be the culprit.

Here, the fix was to add the following to BasicGraphFrame.dispose():

_container in CompositeActor

One of the changes above was to have CompsiteActor._setContainer() call

However, this means that running the Zigbee demo throws a NPE because the container is null:

If we place a breakpoint, then what is happening is that MoMLParser.parse() is failing to handle the LinkApplication and setContainer(null) is being called:

The xml that is causing the error is <deleteProperty name="_senderDestLine"/>.

In theory, we could fix that XML, but as this would happen elsewhere, so a fix is needed

HierarchyListeners

See https://projects.ecoinformatics.org/ecoinfo/issues/7190

Below is the output of JVisualVM

NamedObj defines _hierarchyListeners

It seems like when we clone, it is not sufficient to set _hierarchyListeners to null in NamedObj, we also need to look at the container and remove the listeners. setContainer() should do this for us.