I’m working on a large virtual-world project. Check screenshot here. The terrain is generated using a slightly modified marching cubes algorithm with the terrain divided into 32x32x32 chunks (size to be optimized later).
I’ve tried a few different approaches to generating new terrain on the fly. Building a whole chunk at once is decently quick, but causes framerate hiccups. Spreading the job out over time causes no hiccups, but takes forever to build a single chunk. Every frame I do a simple search for missing chunks within a short distance of the player. When a chunk is found that needs loaded, the loading process is started and the search exits. It doesn’t search again until the new chunk is done loading so there’s only ever one chunk loading at a time.
Is this a case that would benefit enough from multi-threading to make it worth compiling panda with full threading support? I imagine it is because I would have this loading process always running on the second core (which would give me fast loading without slowing down the game.)
Are there instructions somewhere on compiling panda3d from source? I haven’t found any yet but I also haven’t downloaded the source.
Can anyone suggest a different option I haven’t thought of yet? Could I run another thread without any panda code without compiling?
But it sounds like panda3d threads don’t have this problem because they’re not implemented in python?
Anyway, I’ve successfully built panda for real threading (thanks to Chrys for his helpful links) so now I’m closer to my goal. From what I’ve got working so far it loads in the data much faster than before and doesn’t cause framerate issues.
The best I’ve got so far is using an asynchronous task to generate the necessary data, then when it’s done I synchronously build the Geom from that data and stick it in the scene graph. This synchronous part is the only thing I still need to resolve right now. However I try and put this code into the task it locks up the main thread. It should be obvious that this is my first go at multi-threading so I really don’t even know what to look out for. What kind of operations should I be doing to make this process safe?
“Synchronous” means it runs in the main thread, so of course it will lock up the main thread for whatever period of time it takes to generate the Geom. If that period of time is very small, that’s not a problem; but if it’s noticeable, then you get a noticeable hitch in the rendering.
So your two approaches to eliminate a noticeable hitch are:
(a) reduce the amount of time required to generate the Geom, or
(b) do this part of the task asynchronously as well.
It is certainly an option to generate a Geom in a sub-thread. Just parent it into the scene graph in the main thread.
Loading in the task then creating the Geom in the main thread worked, but stuttered as you said.
Creating the Geom in the task and then attaching it to the scene graph in the main thread locked up the program.
Just tried something different: changed the main update task to asynchronous. Now both threads keep going like they should (they keep writing stuff to the console) but the program itself is still unresponsive.
I was just handling the objects directly from both threads, but I can see that’s a bad idea. Did some more reading and found that queues seem to be used commonly for this sort of thing, so I tried. Still deadlocking and it seems to keep trying to load the same chunk all the time. So here we go:
from direct.stdpy import threading
from direct.showbase.PythonUtil import Queue
loadQueue = Queue()
doneQueue = Queue()
# just loop forever looking for stuff to load
if not loadQueue.isEmpty():
chunk = loadQueue.pop()
chunk.readDensityMap() # this and fillDensityMap() get the data used to create the geometry
chunk.generateGeometry() # this actually creates the Geom
#end class EChunkLoader
# this is inside the map class
def update(self, x, y, z):
self.playerX = x # player position
self.playerY = y
self.playerZ = z
if not doneQueue.isEmpty():
doneQueue.pop().finishLoad() # puts the Geom on the scene graph
start = int(self.chunkDimensions / 2)
end = self.chunkDimensions - start
startz = int(self.chunkZDimensions / 2)
endz = self.chunkZDimensions - startz
for x in range(-start, end):
for y in range(-start, end):
for z in range(-startz, endz):
chunkX = int(self.playerX / self.chunkSize) + x
chunkY = int(self.playerY / self.chunkSize) + y
chunkZ = int(self.playerZ / self.chunkDepth) + z
key = str(chunkX) + '-' + str(chunkY) + '-' + str(chunkZ)
if key not in self.chunks:
self.chunks[key] = EMapChunk(self.npMapRoot, self.loadingChunkX, self.loadingChunkY, self.loadingChunkZ, self.chunkSize, self.chunkDepth)