I’ve got a custom controller I want to use with my Panda3D game. I have pySerial installed as a plugin to Panda3D.
I have task set up to run every 16ms using delayTime and task.again (but have also tested with much slower times with the same problem, even letting it run every 3 seconds).
The problem is, even if I strip the task out so it only looks as follows:
There is a massive 1-2 second freeze on the game’s window halting rendering as this task is called. So as above I have the delayTime to 3 seconds, (much slower than could be useful for a controller) and it visibly freezes every 3 seconds.
Each line is only about 100 characters long and this is all is contained in the “line” variable when I print its contents so there is some really bad overhead somewhere either in Panda’s handling of pySerial or in pySerial itself.
Has anyone had any problems like this with pySerial at all? If so, is there another serial library I could use?
You probably wanted to say that you have installed the Python-module “pySerial” inside the site-packages directory of the Python distribution which comes with Panda3D. It is in no way related to Panda3D.
Panda3D does not handle pySerial at all. It is your code which handles pySerial.
The pySerial method “readline” is I/O bound and blocks. You should never call blocking I/O operations from inside a main loop, no matter if GUI, Game or whatever! This quote is from the pySerial documentation, and makes it very clear:
pySerial has excellent documentation and tutorial. Look at the wxPython tutorial in particular. There you will see how you can poll pySerial with timeout and without blocking the rest of the application. I would choose a smaller timeout though, since 500ms seems too large in my opinion. 200ms, or even better 100ms.
No idea what happened but the problem has solved itself for the time being, I will put it on its own thread on its own task chain though to be sure with global variables to be used inside the main loop. Obviously using locks around the writing.
I don’t want to sound rude, but I think you are moving in the wrong direction. Python threads are not OS level threads - they can’t interrupt operations carried out e. g. within a C++ extension. This means that the pySerial readline method still can block if there is not a full line of data available at the time of calling “readline”, and your application freezes until data becomes available.
Problems like this usually come back at a later time if not understood and solved immediately. I recommend one of these approaches:
(1) Use the Python “multiprocessing” module, which effectively spawns a second process where you can execute the blocking operations and then store e.g. the lines read in a queue.
(2) Do it like in the pySerial wxPython tutorial. Call read(1) with a SHORT timeout, and then check if data is available and read the data available. append data read to a buffer. Here you have to split the data read into lines yourself, but this approach is more easy to implement for someone not experienced with threading/ipc/…
I just looked at the tutorial and am not sure why it is any different for my purposes.
I know exactly when data is being sent from my device (every 16ms) so when I do readline() every 16ms in pySerial I know I will get data I want. There will never be no data when I call readline() as I call every 16ms.
I need it to be every 16ms because my game runs at 60fps and so that’s the length of a frame.
This worked when my project was in C#, I used to be on about 100-200fps with my scene with the serial being read in every 16ms.
I just don’t understand why Panda3d/pySerial completely dies and if I run it as a task.cont it runs at 25fps or if I run it as a task.again with task.delayTime = 0.016 it is every slower at 16fps.
Is there something panda3D does strange with task management that’d make task.again slower than task.cont.
Now this is strange. Are you sure you did the same in C# as you told us you did in Python? A single thread, and no tinkiering with ReadTimeout?
Assuming you just have called ReadLine, and the method returned with a single lines of data. Let’s call this time t0. Now
you let the engine handle the scene rendering. Let’s assume your C# engine is lightning fast, so it only needs 1ms. We have t1=+1ms now, and we are back at reading from the serial interface. You call ReadlLine again, but there is no data here. Let’s assume your device send the data after 16ms (this is what you claim). So ReadLine blocks for 15ms until it returns the next line at t=+16ms. Back to rendering and so on… Your framerate is 60, at max!
Python thread are not OS-level threads (like in C#). But you can use the multithreading module, which offers the same API as Python threading. But I suggested this already.
Oh, and is it possible that your framerate is 60 when you comment out the calls to readline (e. g. replace it with a fixed line from your device)?
If I comment out readline() then yes it is at 60fps.
Sorry I wrote the C# version a long time ago and forgot that yes the reading in is on its own thread.
Sorry I completely misunderstood/misread what you said about the threading in python and so will have a go with the python threading library. Im just confused about if I create my own taskchain with numThreads=1 in Panda3D and put my serialReadIn task which calls readline() on this taskchain, isn’t that the same as using python’s threading library and setting up the thread myself? Other than panda3d has no direct knowledge of the thread/task?
Thanks so much for the help so far enn0x and sorry for wasting your time with my misreading.
You probably have turned screen synchronisation on. This means that half of the time Panda3D is waiting, half of the time the pyserial. It’s just a guess, but I think you would end up with around 30fps.
Well, what I wanted to say is that Python threading won’t help you here. That’s all. Either poll pyserial with a short timeout (1 character) and only if there is data available read a full line, or read from pyserial in a separate process.
i don’t know if that is of any help in your case.
for me i worked around a simmillar problem by polling the data from the serial device. so every time i wanted new data i had to send a request-byte to it.
that allowed me to set up the code, send a byte, render a frame (and during that time let the device reply). and by the time rendering was finished and the new frame started, i could read the complete data that was waiting for me in the buffer with read() or readline().
if nothing helps, use a separate process, and pass the data over sockets or pipes.
