[Solved] Mixing C++/Python, accessing the engine in parallel

C++ coding using Panda3D
1

Hello,
First of all, I’m discovering Panda3D with pleasure, and I think I get the right engine to work with, after having played a bit with Ogre and Irrlicht.

I’m thinking of mixing Python and native code for performance reasons. I want to port to Panda an existing C#/DirectX project where thousands of particles are continually colliding. The particle system has to be optimized as much as possible and I want to code it in C++.

Different methods :

  • The particles are created and rendered with Python. When performing the collision, a C++ function is called and operated on the array (ctypes passing an array or with a custom Python module). How do I pass the right pointer ? getParticlesDict ?

  • The whole project is written in C++, and if I want to use python, I have to load and execute the scripts by myself

  • I extend Panda3D (and then have to recompile parts of the source) adding a new class derived from ParticleSystem, and use it like a classical particle system from the python code

The best for me would be to directly access the engine created in the Python context, from a C++ thread or set of classes (say, in a DLL).
I’d prefer avoiding huge transfers of data and being able to upload the vertex/index data to the video card without passing by the Python context.

2

Not quite sure what question you’re asking, but any of these seems like a viable possibility. Though if your primary intention is to develop a particle system, that sounds like something that is best done in C++. Python is most suited for writing the top-level control code, for instance the code to control the game logic.

David

3

Okay, that’s what I think so. But my question is, what is the best way to mix C++ and Python, when :

  • there’s a lot of shared memory between C++ and Python (ie : something that you can’t pass in argument to a function, like 3000 vertices)
  • it has to deal with the rendering engine

As I said, I’d prefer avoiding huge transfers of data and being able to upload the vertex/index data to the video card without passing by the Python context.

4

Hello
I am interested in mixing Python and C++ too, and I have started some small projects wrapping existing C++ libraries for Panda3D.

So far I have done this by creating C++ extensions (by hand, that is without SWIG or boost) that can be imported like standard Python modules. This is similar to the third method in your original post, but without re-compiling Panda3D:
https://discourse.panda3d.org/viewtopic.php?t=2340
https://discourse.panda3d.org/viewtopic.php?t=2265

If you have particular problems or want to share tricks you have found out, I am hight interested :slight_smile:
enn0x

PS: Panda3D objects you create in Python, for example a NodePath instance, are C++ objects with a Python “shell” around them. You can pass these objects from Python to C++ and “unpack” the original C++ pointers to these objects, and then work with them from C++. I don’t know if this make any sense for your project, but creating a GeomVertexData in python (which contains the 3000 vertices) and then passing it to C++ code is possible (I didn’t try with GeomVertexData so far).

5

Okay, and the “unpack” is done with a simple cast like described in this post : panda3d.etc.cmu.edu/phpbb2/viewt … =5762#5762

That means once the reference to the device is present in the C++ side, all the Vertex data is transmitted without even knowing the main script is in Python.

That’s exactly what I need. Thank you.
You solved my problem, anyway I’m open to discuss Python / C++ mixing. Maybe we could add some info in the Wiki or FAQ. Actually, I found on others forums rumors about Panda3D like “It’s a python engine, not designed to be programmed with C++”. Perhaps this comes from the lack of C++ examples in the Wiki.

6

I’m trying to extend the Panda engine with a custom module. I managed to create, build, import the module, and to call some dummy function.
The problem appears when i want to return a Panda3D object that was created in my C++ code ; in this case a GeomNode instance.

With this topic : panda3d.etc.cmu.edu/phpbb2/viewt … ight=dtool i learned how to pass an object from Python to C++.

Should i try to create myself a Dtool_PyInstDef struct (and set _ptr_to_object to the pointer of my GeomNode instance) and then cast it to (PyObject *) or should i use the Py_BuildValue() (python.org/doc/1.5.2p2/ext/buildValue.html) with the “O” param ? But then i don’t understand how python will know that it’s a GeomNode object.

Of course, this doesn’t work :

	Dtool_PyInstDef geomNodeInstance;
	[...]
	geomNodeInstance._ptr_to_object=&geomNode;
	geomNodeInstance._memory_rules=false;
	geomNodeInstance._signiture=PY_PANDA_SIGNITURE;
	return (PyObject *)&geomNodeInstance;
7

This seems to work for me:

IMPORT_THIS struct Dtool_PyTypedObject Dtool_GeomNode;

PT(GeomNode) node = new GeomNode( "foo" );

...

node->ref( );
return DTool_CreatePyInstanceTyped( (void *) node.p( ), Dtool_GeomNode, true, ( node.p( ) )->as_typed_object( )->get_type_index( ) );

enn0x

8

One easy answer is to avoid the need to return a synthesized Python wrapper altogether, by defining your function to receive a PandaNode as an input parameter. Then your function can simply attach the new GeomNode to the node it is given, thereby making it available to the caller, without having to return anything!

But, it is possible to synthesize a Python wrapper. Use something like this:

#include "py_panda.h"

IMPORT_THIS struct Dtool_PyTypedObject Dtool_GeomNode;
...
PT(GeomNode) geom_node = new GeomNode("node");
...
return DTool_CreatePyInstanceTyped((void *)geom_node.p(), Dtool_GeomNode, true, false, geom_node->get_type_index());

The fourth parameter to DTool_CreatePyInstanceTyped, “false” in the above example, is a brand new parameter that only exists with the bleeding-edge CVS version of Panda. If you have a version of Panda that you downloaded from the website, you should omit this parameter (it indicates whether the pointer should be considered const or not).

Note that the third parameter, “true” in the above, is the “memory_rules” parameter. It should be true to indicate that Python should assume ownership of the memory for this object. It should always be true for an object that inherits from ReferenceCount, such as GeomNode.

In fact, there are some special rules about ReferenceCount-derived objects, which may not be obvious from reading the C++ code. These objects should generally not be created locally; they should always be allocated from the heap (i.e. with the “new” operator). And you should store them immediately in a PT(ObjectType) pointer, which will increment the reference count automatically; otherwise, you risk the object being inadvertently deleted while you’re working with it. That is to say, you should always create a GeomNode with code like this:

PT(GeomNode) geom_node = new GeomNode("node");

And not like this:

GeomNode *geom_node = new GeomNode("node");

and never like this:

GeomNode geom_node("node");

This requirement is due to the assumptions made by the ReferenceCount base class. In particular, the reference count is initially zero (which causes problems in the second case, above) and it will try to delete itself if the reference count is incremented and then returns to zero (which can have disastrous consequences in the third case, above).

Finally, you should never explicitly delete a ReferenceCount-derived object with the “delete” operator. It will delete itself when the last reference count goes away, so if you delete it explicitly, it will cause a double-delete.

David

9

Okay thank you, it seems to work (well, my code still doesn’t work but the simple call of my function doesn’t segfault so it’s much better).

Anyway, I understood the need of creating a PointerTo by browsing the source (the compiler was complaining of a ref count problem).

Perhaps I didn’t search properly but I think it’s a lack of the C++ documentation that I’ll add in the Wiki when everything will work.

Thanks for the help !

10

I would like to ask for help on the same topic.

I want to pass a matrix from C++ to Python. The following code seems to work, but somehow I have a bad feeling about reference counts and memory leaks:

    LMatrix4f* _mat;
    _mat = new LMatrix4f( 0.0f, .... );
    return DTool_CreatePyInstance( _mat, Dtool_LMatrix4f, true );

Is there another way to do this?
PT(LMatrix4f) doesn’t work, or I am just too stupid to do it right.
enn0x

11

This seems right to me.

LMatrix4f doesn’t inherit from ReferenceCount, so you can’t use PT(LMatrix4f). And it’s not a TypedObject, so CreatePyInstanceTyped would be wrong. But CreatePyInstance is correct, and the “true” value for the third parameter tells Python that you are passing ownership of the pointer to it, so that Python will call delete when its last reference goes away.

David

12

Unfortunately the links that should describe the “unpacking” of Python wrappers to original C++ pointers are broken. Since I struggled a bit to find how to perform this “simple cast”, I thought it would be useful to post the steps here.

You need to use the function DTOOL_Call_GetPointerThisClass from “py_panda.h” (C:\Panda3D-1.6.2\include).

It will require the corresponding Dtool_PyTypedObject of the object you’re trying to cast, which you can get this way:
IMPORT_THIS struct Dtool_PyTypedObject Dtool_CLASSNAME;
Ex : IMPORT_THIS struct Dtool_PyTypedObject Dtool_GeomTriangles;

The call should look like this:

void test_panda_arg_from_python(PyObject obj)
{
int param = 0; // used for error reporting only
std::string fn_name; // used for error reporting only
GeomTriangles pGeomTriangles = (GeomTriangles*) DTOOL_Call_GetPointerThisClass(obj, &Dtool_GeomTriangles, param, fn_name, false, NULL, false);
pGeomTriangles->get_vertex(0);
}

If you’re using a compiler that needs explicit additional libraries inclusion (like bjam if you’re using Boost.Python like me), then the necessary lib is libp3dtoolconfig.lib (in C:\Panda3D-1.6.2\lib).

Hope this might save some time for someone.
-David