Tighter ODE Integration

Hi all,

The guys at Disney have done an awesome job in integrating ODE into Panda3D, and I’m very thankful for that.
Currently, however, the system requires you to manage the body<>nodePath connections yourself and reposition the nodepaths based on the body position every frame. It’s no big deal, but it could be easier.
Based on many community requests, I have an idea (for 1.7) to integrate it even tighter by making an OdeNode class, which would inherit from PandaNode and OdeBody. It would override setTransform and getTransform to use the nettransform on the ode body as well. Also, it would manage destruction of the underlying dBodyID object correctly.
So, this is just a tiny extra layer, but it saves people from a lot of effort.

We could even take it one step further. Whenever a new body is created or a child/parent is assigned, it will loop through the scenegraph backward and forward to see if it has any (no matter how distant) parent or child OdeNodes. If so, it will connect those using an OdeJoint. The user will be able to get a handle to the joint himself, to set its settings.
When the OdeNode is destructed, it would simply grab the child and parent OdeNodes and connect those.
(However, would this particular idea be too heavy? For every manipulation operation, it would need to traverse the SG.)

Even one step further: we could make a RagdollActor class, which inserts an OdeNode for every bone, and sets the joint settings according to the way the bones are arranged. The user would just need to call find() per bone to set the max & min angles.

How do you guys think about these ideas?

pro-rsoft

I like, especially that ragdoll idea:-) You will have to consider how weights are distributed and which collision solids to use though, i.e. imagine someone with a big cyborg arm - you will want to make it real heavy and use a specific collision solid there. Some default ability and then the ability to replace the default collision solids/weights would be nice. Not sure what you mean by the automatic joint creation though.

However, a further suggestion to improve integration - right now creating collision geometry from panda geometry is a bit basic, specifically you can’t give it a tree of meshes, only a flattened mesh. The ability to give ODE a tree of meshes and have it construct a trimesh for each mesh and an ode space for each level of the hierarchy would be very convenient. Reason I ask is right now Chicken will kick out an octree mesh… and then I can’t use it with ODE - I have to flatten it first, losing all optimisation potential. Also, the ability to do this with it only copying over geometry that is marked for panda’s collision system would be nice, as you could switch between systems very easily that way. (Throw render at the relevant function, get back a space… go.) I know the above can be implemented in python, and indeed I have been intending to do so at some point, but this is one of those things that I think makes sense to have in the core engine.

Additionally, and this might just be my imagination, but geometry marked for collision for Panda’s system didn’t seem to work when passed to ODE’s trimesh - didn’t seem to get stored within. Moment I removed that it was for (panda) collision it started working. I didn’t look at it closely though - I might be mistaken as there was other stuff going on at the same time which might of been responsible.

Well, you can always change the settings later. Like:

cyborg = RagdollActor("models/modelWithHeavyArm")
panda.find("**/+OdeNode/Arm").setMass(yourOdeMass)

I could even add a defaultMass argument to the RagdollActor constructor. Maybe we can even use tags to store the mass for a node - you could then store them in the .egg file.

Well, the problem is that ODE is not scene-graph based. Your suggestion would require creating an OdeNode for every GeomNode. It’s possible, but it needs a bit thought how that would be handled.

That’s weird. Looking at the ODE source, it just seems to grab all Geoms regardless of any collide masks set.

That sounds fine - I was just worried about access being hidden away. Tags would also be awesome.

ODE does support hierarchies… its just not obvious how. (This is my understanding - I might have got some details wrong as I am yet to experiment with this as my test stuff isn’t large enough to need it.) Essentially the model typically given is that you have a bunch of objects inside a space and then you do collisions between objects in the space. But you can also put spaces inside a space, and all objects in the parent space will then collide with the child space. In fact, when you do this it does exactly what panda does - quickly prunes stuff if their axis oriented bounding boxes don’t overlap. Simple ODE spaces are to all intents and purposes equivalent to nodes in Panda in other words, at least from a collision point of view.

How about I write some code to do it and post it back here? That will give me an opportunity to test it out and see what happens - I need it anyway, so might as well.

Must of been something else that was breaking it then - as I said I didn’t spend much time on it, it just didn’t matter than much.

Hmm. I think if I made OdeNode be an OdeSpace, that would certainly complicate stuff a lot more. I think I personally prefer the non-hierarchical way by passing the net transform to ODE.
Also, I don’t think collisions between sibling OdeSpace’s are allowed then, are they?

I wasn’t suggesting making OdeNode a child of OdeSpace - I think doing that the way you stated makes more sense as you can then choose your ode space as a hash space or octree space to get the needed speed up.

When dealing with large meshes, i.e. an entire level however the hash space/octree space can do nothing to accelerate (There are acceleration structures in ODE’s trimesh, but there not as effective.), so it makes sense to octree the mesh and put it into ode as an octree using spaces… and that is why I would like that feature. And my understanding is that collisions do happen between siblings, that kind of being the point. (Or more accurately you have a callback that recursively does the collisions, of which ode defines a default to do precisely that.) There are other ways of doing it of course - you could just chop up the mesh and store all parts flat, relying on the hashing or ode octreeing to accelerate it, but an explicit tree has a balancing advantage, and is easier from a management point of view.

Hmm, maybe for later then. For that, we’d need to redesign a part since Panda’s ODE layer isn’t designed for a large number of spaces.
I’d rather pursue my original idea.

I’ve been given this a bit more thought; what you say does make sense. We could store a dSpaceID on the OdeNode object. That would hide the interface from the user while still maintaining the performance benefit you get from it.
People could then call a makeSpace method (or so) on the root node which would return an OdeSpace that wraps this node’s dSpaceID.
In the other thread you mentioned that it worked like a scene graph this way - but are the transforms still global, or do they become local to the space’s transform?

I have to admit I’m not sure if the transformations are global or not - I’ve just been looking at ODE’s manual and I can’t find any clear evidence either way, but then I didn’t look too long. http://opende.sourceforge.net/wiki/index.php/Manual_(All)#Spaces describes them as ‘implacable’, which would imply one coordinate system though, and I would certainly implement a physics system with only one coordinate system as it makes the code a lot neater. Might have to try it and see what happens.

P.S. Panda ODE doesn’t currently expose the convex or heightfield object types, which I guess are new since the original integration - they could be useful, especially if heightfield was integrated tightly with Pandas existing terrain capabilities. (For the convex object initialise with a mesh, creating a plane for each polygon - would then be identical to a trimesh in usage except only for convex meshes. I imagine they are a lot more stable than trimeshes, and so could be good when you otherwise want to do a trimesh-trimesh collision, which I’ve found to be mostly unusable. Documentation on them is rather lacking though.)

Hmm. I’ve been playing around with PyODE, but I wasn’t able to do setPosition on a space (“Cannot set a position on non-placeable geoms”). Also, I wasn’t able to attach a space to a geom (“Cannot convert ode.GeomBox to ode.SpaceBase”).
So, clearly, there is no scene graph look-alike in ODE.

We could still do this approach by secretly creating a dSpaceID in the background of each OdeNode and any subgeoms would be added to this space. That would be just a simple addition to my original idea, though - we would still have to do the scene graph placement ourselves.

The point here isn’t really to duplicate the scene graph, though you can, at least structurally - what matters is the ability to optimise collisions by being able to quickly throw away large number of objects, by having a single test against a space within another space - same way a scene graph structure optimises graphics rendering by throwing away large chunks of the scene, if constructed well. Really, once the space inherits from a geom it should all start working correctly, as long as the interface code hasn’t made any bad assumptions. (Not likelly, but ho-hum.) My mention of copying the Panda scene graph really comes down to one thing - panda already has lots of support for its scene graph, so the ability to duplicate chunks of it into ODE is the path of least resistance for both implementing and using this. i.e. use Pandas pre-existing code to construct, then give it to ODE, rather than duplicating all the features Panda already has.

But I see no need for a panda node to have an ode space number as that information is only needed temporarily whilst constructing an ODE representation of something panda already has a representation of. I’m not even sure an ODE aware derivative of a panda node would need that information, as you don’t need the space number to get a body’s location and update the nodes position accordingly - just the body number.

As I see it getting that inheritance working is ‘all’ that is really required - everything else just follows and is mostly convenience code, which would admittedly be quite convenient. Especially the body aware panda node. Plus there is adding those extra collision primitives, though that is independent from the rest and could be done at any time.

Yes, but that is a different story, not related to this thread.

Whilst you can certainly hack a solution within the current system when the hierarchy problem is solved that solution would logically use it, that being the more elegant solution, and hence need recoding, mostly from scratch. I would of thought getting it right now and saving the effort of implementing a to-be-deleted temporary hack would be preferable. That is my 2 pennies at any rate.

drwr, what do you think about the idea in my first post?

I think there might be some technical problems in getting an OdeNode to work the way the user will expect. Overridding set_transform and get_transform will only work for transform changes that are made directly on the node itself; but if the user changes a transform on a parent node, he will certainly expect that change to propagate into the Ode space as well.

Also, similar problems with automatic reparenting detection. You can get a callback when the node itself is reparented to a new node, but not when some ancestor’s parenting is changed.

I think the only realistic way to map Panda’s scene graph into Ode space is to have a traverser that runs every frame, similar to Panda’s cull and collision traversers, and which rebuilds the Ode space according to what the current state of the world is. This might require some clever caching mechanisms to make this operation less expensive (but it might not–it’s worth a try naively first, to see how bad it will be).

David

Hmm, I might have had a wrong idea how panda’s scene graph worked - I always thought functions like set_transform propagated down. You’re right, it would become impossible (or very hacky) to do this without dedicating Panda’s entire scene graph to ODE.

The idea to make a global class that synchronizes the ode world and panda scene graph is not a bad idea. We would still need the OdeNode class because it’s going to be a little bit tricky to merge the two transforms (one from ode, one from the scene graph) otherwise, but I think that would only make things easier.
So, we would create a global OdeTraverser or OdeSceneGraphManager (or so) class, that gets instantiated in ShowBase. (possibly only when people have set a certain config var, to avoid performance overhead for users who don’t want ode.)
This class could also represent the OdeWorld to hold the bodies.
The OdeNode class would still inherit from OdeBody (Is that a good idea, making a reference-counted class inherit from a class like OdeBody? Otherwise we could just store a dBodyID and re-implement the functions - since OdeNode would entirely handle the construction and destruction of the dBodyID, I guess that would be okay now)
It would also store the net transform from last frame and a dJointID connecting the node to it’s parent (the connecting will be done by the global class).
The traverser would be ran in a task, that loops through the OdeNodes, checks their net transform. If it has changed, it will override the position on the dBodyID - if it didn’t change, the dBodyID position will win.
It will also manage the joints that connect the bodies by looking at its nearest parent that is OdeNode.

We could store an array with OdeNodes on the global ode traverser so it won’t actually have to traverse, at least not every frame.

We would then still keep the possibility for a RagdollActor class.

How about that idea?

Sounds like you’re on the track of something good. No doubt there will be implementation issues yet to be discovered, but from a high level, this all sounds good. :slight_smile:

David

Me again, and I’m complaining about OdeSpace not being a child of OdeBody. Again. However, I just wanted to share one more consequence of this design fault, a basically fatal one. I really can’t emphasis enough how much I think fixing this problem should be top of the improvements list.

Simple scenario: you want to use OdeRayGeom to cast rays and find out what they hit.

You can’t collide the ray with the scene directly by calling OdeUtil.Collide(…), because of the bad inheritance, even though that is the correct way to do it. (And supported by every other Ode interface in existence.)

You think that you can add the ray to the scene and use the callback to detect the collisions. Problem is the objects now collide with the ray, so the moment you point a ray at them they try and move to the end of the ray (It has finite length.). Which is to say, they accelerate so fast there half way to infinity before the collision system next runs, and you never see them again. I’ve wasted around 6 hours of my time figuring this out, when a single function call to do this exists in normal Ode.

The only way to do it right now (That I can think of - I’ld love to hear another solution.) is brute force - test the ray against every object in the scene in one big slow loop. Which is to say that the OdeRayGeom is useless - it simply can’t perform its intended function.

The first person shooter I was writing, so I can have a go at making some AI and to demo how it could be done in Panda, is dead in the water until this is fixed. The brute force solution is simply not good enough with the 50+ objects I have, and causes a noticeable frame rate glitch. Only other solutions are to flip to PyODE, or move to Pandas physics system, both of which would involve re-writing so much I might as well start again. Or a bunch of even more insane tricks that mostly come down to re-writing sizeable chunks of Ode myself.

The path of least resistance would be to fix this problem myself - the question is will you accept the patch? Its going to be big, complicated and will probably mess with half of the current ODE code. Plus I am really not a fan of creating it and having it stored for 6+ months whilst we wait for 1.7.

We accept patches.

Plus I am really not a fan of creating it and having it stored for 6+ months whilst we wait for 1.7.  

If you going to make your game you can compile your own panda and distribute it with your game.

Also you can try pyODE.

I’ve heard it, considered it, agreed with it, didn’t forget it, and it’s still in the back of my mind. And as I said the first time, I’ll get back to you about it.

What’s the function called?

If it would make you happy, I could duplicate that function for the next bugfix release, but accepting an OdeSpace. Maybe even add a function that casts between OdeSpace and OdeGeom.

I’m afraid that’s not up to me.