I am using Panda3D to make a small simulation for a machine learning project. I mostly have it working at this point, but I have hit a minor issue and would like some advice from some experienced Panda users.
One of the requirements of my system is that it needs to record data from the sim. That is easily done with python; not a problem. The problem is what to do with it.
The data grows rather quickly, and at some point it is too much and will need to be dumped to a file.
File access/writing is slow enough to throw off the simulation while it is occurring.
I would like to be able to kick the data out to a file, and reset the DATA dictionary, but do so without impacting the simulation. It is ok to pause during that time.
My initial thought is to handle the file i/o in a separate thread. Does anyone know what kind of impact this would have on the game loop or the ODE step?
Sounds like a fine solution. Threading should solve your problem acceptably. Of course, using threading can complicate your code; you have to be careful to avoid race conditions and deadlocks.
In Panda, there are some additional considerations with using threading. Your best bet is probably to use Panda’s threading constructs, rather than Python’s (which means you should import direct.stdpy.threading instead of threading); and you should use Panda’s file object rather than Python’s to avoid blocking your other threads (which means you should input direct.stdpy.file).
from direct.stdpy import threading
from direct.stdpy.file import open
(You can use Python native threading if you want, but if you do you have to be absolutely sure never to make any Panda calls from the sub-thread. It’s usually easiest to use Panda’s constructs.)
Hey thanks. I’m sure it would have taken forever to track down a bug due to using the wrong thread module. I didn’t even suspect that Panda had its own. That little bit of info is probably the difference between days and few hours of work.
Fortunately, I do have varied and strange coding experience, and am familiar with deadlocks, race conditions and threading in general. I appreciate the advice just the same, though:D