Taks/threads problems

Scripting Issues
1

Hi, I am coding a server using panda3d and I am facing one problem when dealing with the tasks/threads.

Right now I have the program divided in two parts, one that is an infinite loop that checks if there is a new connection to accept (if there is, it accepts it) and a then sleeps for 0.05 seconds.

The other part is triggered when enough accepted connections are reached, at this point the program should spawn a new thread to handle a hosted game. The old thread keeps accepting connections.

I am currently using this infinite loop and sleep code because if I use tasks in the normal way the program will consume all my CPU, even if it is doing absolutely nothing.

My problem is in the new thread spawn part, I tried using actual threads, but this causes the program to crash when it tries to load the actors, here is the error message:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/share/panda3d/direct/stdpy/threading2.py", line 472, in __bootstrap
    self.run()
  File "/usr/share/panda3d/direct/stdpy/threading2.py", line 452, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/fernando/Documents/projects/PythonArenaServer/instance.py", line 69, in manageHostedGame
    self.createAndSendLoginAndPlayerInfo()
  File "/home/fernando/Documents/projects/PythonArenaServer/instance.py", line 100, in createAndSendLoginAndPlayerInfo
    loadedPlayer = Player(id, x, y, z, scale, playerClass, connection, playerCollisionTravesser, side, maxHp, name)
  File "/home/fernando/Documents/projects/Arena/worldCharacters.py", line 137, in __init__
    annimationDict
  File "/usr/share/panda3d/direct/actor/Actor.py", line 277, in __init__
    self.loadModel(models, copy = copy, okMissing = okMissing)
  File "/usr/share/panda3d/direct/actor/Actor.py", line 1889, in loadModel
    model = loader.loadModel(modelPath, loaderOptions = loaderOptions, okMissing = okMissing)
  File "/usr/share/panda3d/direct/showbase/Loader.py", line 159, in loadModel
    node = self.loader.loadSync(Filename(modelPath), loaderOptions)
AttributeError: Loader instance has no attribute 'loader'

I also tried using task chains, I used this code:

		taskMgr.setupTaskChain('IstanceChain'+str(taskChainId), numThreads = 1, tickClock = False)
		taskMgr.add(
						self.manageHostedGame, 
						'Hosted game'+str(taskChainId), 
						taskChain = 'IstanceChain'+str(taskChainId)
		)

The task chain code seems to be ignored, as the program executes it, but the function I added in the taskMgr.add is not executed at all (I tried placing a print in the first line of the function and it is never called).

Some extra info:
I am using ubuntu 10.04 64 bits and panda 7.2, the program is windowless (since it is a server), but it does load models so that it can process collisions. Loading models works fine with threading but loading actors doesn’t.

Any help is very well appreciated.

2

There’s nothing wrong with your program taking up all your CPU - most games do, in fact. It’s there, so why not use it?

The threaded version of your program probably didn’t work because Panda 1.7.2 (IIRC) isn’t thread-safe unless you compile it yourself. This is also the reason the task chain-based version of your program didn’t work: Since Panda doesn’t use real threads for its task system, it never gets the chance to execute taskMgr.step() - which runs a single iteration of Panda’s task scheduler, and if called would have executed your task chain - because it was in your infinite loop.

To fix this problem, I recommend using a full task-based solution, with no infinite loops. Your code will be simpler, and it will be a lot more efficient. The 100% CPU usage is nothing to worry about; again, it’s there - why not use it if no other program is?

3

The problem of using 100% CPU is that this is the server side, not the game =p

Hmm, as far as I could understand from the Manual, taks are not run in a multi-thread based environment, but task chains are, I may be wrong tought.

Since the tasks are called once per frame, isn’t there a simple way to force the system to run in 20 fps rate? I believe this would simulate my sleep system.

Also, I will check if I can’t force the step =)

4

Are you running anything else on the server? :stuck_out_tongue:
Also, task chains are not run in a separate thread, so they aren’t concurrent, either. To get around this, you can call taskMgr.step() to force an iteration of the task manager. Another solution would be to force the framerate to 20 FPS, like you suggested, but imposing your sleep system on unrelated parts of your server seems like overkill to me.

5

Hmm, seems like you are right. Strange, the way the manual specifies it would give the impression that it would spawn a new thread. Still I used the step method, the function was called and then stopped again (waiting for a new step() call).

The server does nothing but checking for new connections, but using the normal task system, it will check for connection a load of times per second, consuming the whole CPU. In a normal server application, this connection checking would be blocking, but I assume that this wouldn’t work with panda as well… I will try forcing the frame rate to 20. I tried once with no success, maybe I missed something.

6

There seems to be a bit of confusion in this thread.

When you create a task chain with numThreads = 1, you have indeed created a threaded task chain, and the tasks that you add to that chain will be run in a separate thread. However, these are Panda’s “simple” threads (in 1.7.2 and before), not true threads, so there are some rules that you have to follow. One rule is that you should not call time.sleep(), you must call Thread.sleep() instead. Another rule is that you must not use Python’s blocking networking interfaces, but you may use Panda’s blocking networking interfaces if you like.

Still, threads are not the best solution for this problem. I really, really recommend not using threads at all, unless you already have a lot of experience using threads. Writing correctly threaded code is really, really hard to do and takes years of practice.

Instead of using threads, write everything with the normal task system. Just be sure not to do anything that takes a long time in any one task.

All of this has nothing to do with the 100% CPU problem. If you don’t want your process to consume 100% CPU, just call base.setSleep(0.01).

David

7

Did you mean setSleep(0.001) instead of 0.01? Otherwise you’ll get a somewhat significant performance reduction.

8

Thanks a lot for the clarification.

I do have some experience with threads and concurrency, but I am mostly used to the posix implementation of it, which is quite different. In my particular case, each thread handles its own data, so there is no need to worry about race conditions or concurrent memory access.

Anyway, I used the globalClock to set the framerate to 20 and I am using the normal task system now. The process is now consuming 21% of a core but is pretty stable on this 21%, which is enough for my needs right now.

I think I will wait the 1.8 release to revisit this part of the program, unless the performance becomes a problem.

Thanks again.