Creating geometry in a separate thread

[federicocasares]

Hi everyone. I am kinda new to Panda3D and so far I love it, but I have got stuck when trying to create geometry in a separate thread. I’m making a game that creates a lot of meshes on the fly as the user is playing, using GeomVertexData and GeomVertexWriter.

Some of these on the fly creations can take up to 1/5th of a second, so it’s not acceptable for the game to be stopped during that time, it becomes too noticeable. So I thought it might be a good idea to put that in a separate thread (I’m already doing that for networking and works perfectly). So far, I haven’t been able to do it. Things I have tried so far:

1. Creating the GeomVertexData and GeomVertexWriter(s) objects in the main thread and then writing data to them in a different thread. This fails because GeomVertexWriter knows who created it and doesn’t allow other threads to alter it.
2. Creating the GeomVertexData and GeomVertexWriter(s) objects in the separate thread, populating the vertex data and then attaching the node for rendering. This generates a deadlock.
3. Creating the GeomVertexData and GeomVertexWriter(s) objects in the separate thread, populating the vertex data and then attaching the node for rendering in the main thread. This also generates a deadlock.

I have tried with Python’s thread and threading libraries as well as those provided in direct.stdpy. So far everything leads to crashes and deadlocks. Any thoughts? Can someone provide an example of geometry creation in a separate thread?

By the way, I’m doing these tests on a 64 bit Arch Linux.

Thanks a lot in advance,
Federico.

[Tober]

If you can’t fix the deadlock, perhaps you could make new geometry in a panda3d task that keeps track of the time it’s taking and returns when you’re running out in that frame. It will probably take longer to make the geometry, but the main loop will be given control between the returns so the screen will still update.

[cslos77]

Here is an example of a simple triangle created in a separate thread. It should pop up on the screen in 4 seconds or so after the program starts. To do this you need to create your vdata and geom in the main thread then send them to the second thread where you then create the vertex writers and primitives. When you’ve finished populating the geom “send” it back to the main thread where it can then be associated with a node and attached to render (or wherever.)

Note: this doesn’t work with the multiprocessing module (for multicore threading) even though most things you can do in threads in Python you can also do in multiprocessing, but there seems to be a fundamental block on passing panda geoms between processes.

import direct.directbase.DirectStart
from direct.showbase.DirectObject import DirectObject
from panda3d.core import GeomVertexFormat, GeomVertexData, GeomVertexWriter
from panda3d.core import GeomTriangles, Geom, GeomNode, NodePath
from threading import Thread
from multiprocessing import Pipe

DELAY = 4  # Delay in seconds until triangle pops up on screen.

def build_triangle(geom, vdata, child):
    """Build triangle in second thread."""
    # Create vertex writers and fill with data.
    vertices = GeomVertexWriter(vdata, "vertex")
    colors = GeomVertexWriter(vdata, "color")
    vertices.addData3f(0, 10, 10)
    vertices.addData3f(-10, 10, 0)
    vertices.addData3f(10, 10, 0)
    colors.addData4f(.9, 0, 0, 1)
    colors.addData4f(.9, 0, 0, 1)
    colors.addData4f(.9, 0, 0, 1)
    # Create primitive triangle and populate.
    triangles = GeomTriangles(Geom.UHStatic)
    triangles.addVertices(0, 1, 2)
    triangles.closePrimitive()
    # Add primitive to geom and return it to main thread.
    geom.addPrimitive(triangles)
    child.send(geom)
    
def render_triangle(thread_2):
    """Render triangle in main thread (after delay)."""
    # Get geom back from second thread.
    geom = parent.recv()
    thread_2.join()
    # Add geom to node and attach to render.
    node = GeomNode("geom")
    node.addGeom(geom)
    node_path = NodePath(node)
    node_path.reparentTo(render)


# Create vdata and geom in main thread.
vformat = GeomVertexFormat.getV3c4()
vdata = GeomVertexData("Data", vformat, Geom.UHStatic)
geom = Geom(vdata)

# Send them to second thread.
parent, child = Pipe()
thread_2 = Thread(target=build_triangle, args=(geom, vdata, child))
thread_2.start()

# Set up the task manager to trigger render_triangle after DELAY seconds.
taskMgr.doMethodLater(DELAY, render_triangle, "tri", extraArgs=[thread_2])
import sys

base.disableMouse()
base.camera.setPos(0,-200, 0)
base.camera.lookAt(render)
base.accept("escape", sys.exit)
run()

[federicocasares]

@Tober

If you can’t fix the deadlock, perhaps you could make new geometry in a panda3d task that keeps track of the time it’s taking and returns when you’re running out in that frame. It will probably take longer to make the geometry, but the main loop will be given control between the returns so the screen will still update.

I hadn’t thought of that, Tober, thanks for the suggestion. I hope I can do it with “real” threads instead in order to make the code cleaner and the performance higher, but yours is a valid approach as well. Thanks!

[federicocasares]

@cslos77

Here is an example of a simple triangle created in a separate thread. It should pop up on the screen in 4 seconds or so after the program starts. To do this you need to create your vdata and geom in the main thread then send them to the second thread where you then create the vertex writers and primitives. When you’ve finished populating the geom “send” it back to the main thread where it can then be associated with a node and attached to render (or wherever.)

I’ll give that a try and let you know, cslos77. Thanks a lot for the detailed example and the explanation!

[federicocasares]

@cslos77

Here is an example of a simple triangle created in a separate thread. It should pop up on the screen in 4 seconds or so after the program starts. To do this you need to create your vdata and geom in the main thread then send them to the second thread where you then create the vertex writers and primitives. When you’ve finished populating the geom “send” it back to the main thread where it can then be associated with a node and attached to render (or wherever.)

Note: this doesn’t work with the multiprocessing module (for multicore threading) even though most things you can do in threads in Python you can also do in multiprocessing, but there seems to be a fundamental block on passing panda geoms between processes.

I implemented your solution (creating vdata and geom in the main thread, populate it somewhere else and then come back to insert geom) using multiprocessing and it worked perfectly. Thanks a lot!