Possible Proof of Memory Leaking in P3D

General Discussion
21

I’m trying the memory leak track ideal, but not sure how to start it.

I put track-memory-usage 1 in my config file, but what do I have to type when starting the app?

ppython -i myapp

or

ppython myapp -i

and should control C be pressed over the game window?

=================================7:24 AM

Figured out how to use that method, but I don’t understand it well enough to benefit from it.

Once again, my memory will go down when I destroy an instance and up when I create one, but it will not go down the same everytime.

Eventually, Memory creeps up little by little. I’ve looked everywhere this time and I don’t see a single reference left behind anywhere.

My friend wasn’t kidding when he said beware of Panda3D’s memory dumping method.

22

I was reading up on Python’s garbage collection module and it states that Python will not only use reference counting, but it will also run a garbage collection, which is suppose to find all things that become trapped in memory and can not be freed.

I thought Panda3D data can not be freed once it becomes trapped…?

I setup the garbage list to print all things collected but always got a list of Zero.

import gc
gc.set_debug(gc.DEBUG_LEAK);

#and I had this print out
print "Garb: "+str(gc.garbage);

I even tried invoking the collection manually, but can’t say that it made any difference.

gc.collect();

I’m starting to draw a conclusion that Panda3D will eventually raise in memory no matter what. I can see my memory going down on a dump and up on a load, but eventually it’ll go down less, which means it’ll slowly allocate more memory.

Sometimes memory stands still on a load and sometimes after creeping up, it’ll drop down a lot, but never at the all time low it once was originally. I understand that P3D will hold on to memory, but that’s holding on too tightly.

I’m only working on partical effects and creating and destroying instances. I’ve only been using Panda3D for a couple of months, so I don’t think I’ll give it up that fast.

I’m just worried that, if a user of an app I make with P3D runs it long enough and instances are created and destroyed over a few hours time, those little gains in memory will soon flood out memory and crash the app.

I wish there was a way to print out all references pointing to a particular instances when it’s created and then a final print out of references once it’s destroyed.

Although, if I have any references stopping a cleanup then I’ll be damned.

I’ve deleted every object,

actor,

attribute,

attribute pointing to tasks,

stopped all tasks,

collision nodes,

collision Handler attributes,

the attribute in base (base.cTrave, base.eTrave, etc),

any textures stored as attributes,

I’ve cleaned out any lists within an instance that contained a pointer to another created instance (by looping the list and deleting every sub instance and then deleting the list itself),

and I haven’t forgotten to call that self.ignoreAll first.

Local/unbound variables delete themsleves (the ones without the self parameter)

Did I leave out anything?

23

It certainly sounds like you’re being thorough in your object cleanup, probably much too thorough. In general, you shouldn’t need to be as conscious of this as you are; the intended design of Python is that you shouldn’t need to think about memory management at all, and to a large extent this is true.

I think you’re worrying a bit too much about the possibility of a leak; you can’t really tell much simply by examining the application memory, because both Panda and Python are likely to hold onto memory for caching purposes, which will happen seemingly at random. But just because memory doesn’t drop down to its original starting point doesn’t mean that there is a leak. Neither Panda nor Python will return all of their memory back to the operating system when they are not using it; instead, they will hold on to some of that memory and reuse it for some operation in the future, which is more efficient.

If you do suspect a leak, you can prove it by watching objects accumulate using MemoryUsage.showCurrentTypes() across an operation that should return to its starting point (but even then you have to be careful that you’re not just watching a cache fill up).

Here’s a few Config.prc variables you can set to disable some of the caches within Panda. Of course this will impact your performance, but it will help to avoid getting distracted by caches when you are analyzing the scene with MemoryUsage. These are the major application-visible caches; some low-level caches cannot be disabled.

transform-cache 0
state-cache 0
geom-cache-size 0
released-vbuffer-cache-size 0
released-ibuffer-cache-size 0

You can also see memory usage within PStats, which is a bit more accurate than watching the application memory in the task manager, especially with track-memory-usage in effect. The memory usage reported in PStats is a more precise measurement of the memory actually in active use within Panda, and to a lesser extent, Python. If you see this memory grow steadily over a long period of time, especially with the above caches disabled, you might have found a leak.

David

24

I’m hoping you can help me with this one, drwr.

I think I tracked down the origin of the memory leak. What I did was start stripping away on code, tasks, attributes (self variables), etc…and then check the memory to see if it reached a stopping point.

Yes it did! I found one clear problem that was causing it (can’t believe I missed that).

The second problem isn’t so easy to pin-point. My memory increase almost stopped when I changed the base.cTrav (which is defined in showbase) to self.cTrav (which is just attribute holding the Trav value/pointer).

What I did next was get rid of the collision solids, but kept the collision Queues. I figured the queues were not the issue, but maybe the solids were not being cleaned/deleted correctly.

That did the trick! My memory stopped growing, even with identical instances constantly creating and destroying, which is the way it should be.

Now that creates questions for me. Drwr, you mention that base.cTrav (defining through base) was just a method for auto triggering the collision trav for an instance/object.

It’s obvious calling del on the base.cTrav was not working correctly, or defining under base for object instances is not a good idea, which brings me to this question -

Is there a way to trigger a collision check manually using a self variable instead of creating an attribute through base? In other words:

#Auto way
base.e_beTraV;

#none auto way
self.e_beTraV;

How can I use that “none auto way” to trigger collision checking?

Moving on…

The collision solids.

Is there a certain way to remove collision solids? What I’ve been doing was this:

del self.ExposedLJointBone;
self.ObjMask1.removeNode(); del self.ObjMask1;
del self.ObjMask1ColH;
self.ObjMask2.removeNode(); del self.ObjMask2;
del self.ObjMask2ColH;
self.ObjMask3.removeNode(); del self.ObjMask3;
del self.ObjMask3ColH;

Then the cleanup and deletion of the Actor which held the collision solids (attached to joints) follows.

For what I can tell, my memory issues has to do with the Collision Travs and those Collision Solids.

Additionally, I like the idea of have a different Trav with every instance, but I’m going to try a different approach. That is, using one Trav for the test app (if I can get away with that). I’m going to have to move some code around, but I think it’s possible.

25

I don’t really understand everything you are saying, but I should clarify that there’s no magic with base.cTrav. It’s just that there is a task that ShowBase automatically starts that calls base.cTrav.traverse() every frame (you can see this task if you inspect ShowBase.py), so whatever traverser you assign to base.cTrav gets automatically traversed.

If you don’t assign a traverser to base.cTrav, then you have to call traverse() on it yourself. That’s all that base.cTrav means.

I certainly don’t know anything in the collision system that might be a source of a memory leak. Can you reproduce this leak with a simple program?

Here you are again being much more thorough than is actually necessary. obj.removeNode() would be sufficient for removing an “into” node. For a “from” node, that is, something that you have explicitly added to a CollisionTraverser, you should probably remove it first.

As I keep trying to explain, it’s really not a good idea for performance reasons to have more than one CollisionTraverser. Having multiple traversers means that all of them have to do some of the same work redundantly, but if you have only one traverser that handles all of your collisions, it can share much of this work.

David

26

Hallelujah!!!

Lol

I don’t mean to post in rows, but just want to let everyone viewing this thread know that I believe I’ve solved the problem.

First off, I don’t believe there’s an issue with P3D and defining multiple Travs. The problem was the lack of knowledge of the consequences behind defining multiple Travs. It turns out that giving every instance of an object a random Trav of it’s own is a “no no.”

Why is this a “no no?” I can only theorize that collision solids passed into the Trav do not get cleaned up correctly.

What I did was define a Trav Class and pass collision data from the instances to the Trav Class. I already had a running task calling the base.cTrav (which was defined in my Trav Class).

The task that called for the collision checking is the task the calls all movement updates first. One thing I noticed about using P3D is the fact, where code executes in the program makes a huge difference on the outcome.

I noticed this when movement was not in sync. I figured the movement of objects appeared to be out of sync because of when everything got updated, especially the camera movement.

So one task will call a movement up date for all instances and the camera, then that same task calls for a collision check on the scene. This approach keeps movement in sync without that poor visual effect you get when the camera is out of sync with object movement.

Additionally, what I did was create a removeCollider method in my Trav Class. I then made sure each instance called the removeCollider function of the Trav Class, passing its own data as arguments.

Following that removeCollider call, I deleted the solids and queues as usual.

End of Memory Leak!

(I just pray that it really is the end of it this time, and for all time. This engine is making my back hurt.)

Thanks for your time, drwr.