[SOLVED] Strange affinity side effect with multiprocessing.

General Discussion
1

Hi Everyone,

I’m having a strange problem with the multiprocessing package and panda. Importing panda modules causes multiprocessing to only use one of my cores.

I’ve created an example test case. It uses an infinite loop to ping the cores so you’ll have to manually kill the python processes.

So if I run the following I can see in my task manager that both cores are at 100%.

#from pandac.PandaModules import Point2
from multiprocessing import Pool

def dummyWorker(a):
    while True:
        continue
            
class DummyTester(object):
    def __init__(self):
        self.pool = Pool(2)
        
    def go(self):
        result = self.pool.map_async(dummyWorker, range(2))
        result.get()
        
if __name__ == "__main__":
    DummyTester().go()

But if I uncomment out that one line there it only uses one of my cores.

This is with a fresh download of panda 1.7.0 on windows xp, intel core2 duo.

-Greg

2

Have you tried importing Panda after multiprocessing?

I had some problems as well, but eventually I got it to work.

panda3d.org/forums/viewtopic … 4fbd0882d9

3

On some very old windows-es threads get different OS times on different cores panda3d sets affinity to the first core all the time.

set

auto-single-cpu-affinity #f

before importing direct start.

4

Thanks for the replies.

I tried setting the config variable but that didn’t seem to work. And searching the code for ‘affinity’ didn’t seem to indicate it was being used. But that did point me to the TrueClock class which I was able to use to set the affinity manually.

So adding the following after my panda imports appears to have fixed my problem.

from panda3d.core import TrueClock
TrueClock.getGlobalPtr().setCpuAffinity(0xFFFFFFFF)

Note that setCpuAffinity gets called for every process I start up.

-Greg

5

Hmm, can you check the setting of your config variables:

print ConfigVariableInt('client-cpu-affinity-mask')
print ConfigVariableBool('auto-cpu-affinity')
print ConfigVariableInt('client-cpu-affinity')
print ConfigVariableBool('lock-to-one-cpu')

These values should be -1, false, -1, and false, respectively, to avoid setting any initial affinity mask. If they are not these values, where are they coming from? If they are these values, I don’t understand where the initial affinity mask is being set.

David

6

Hi David,

lock-to-one-cpu is True for me. Setting that to false in the prc file fixes the issue and now I don’t have to set the affinity manually anymore. Thanks!

Could this be the culprit, in trueClock.cxx:169?

  ConfigVariableBool lock_to_one_cpu
    ("lock-to-one-cpu", false,
     PRC_DESC("Set this to true if you want the entire process to use one "
              "CPU, even on multi-core and multi-CPU workstations. This is "
              "mainly a hack to solve a bug in which QueryPerformanceCounter "
              "returns inconsistent results on multi-core machines. "));

  if (lock_to_one_cpu) {
    set_cpu_affinity(0x01);
  }

My c++ is rusty so I’m not sure if the compiler will use some sort of operator overload to cast ‘lock_to_one_cpu’ to an int or bool. If not maybe a get member needs to be called?

-Greg

7

No, that’s the way bool config variables work in C++. It must be explicitly set True in some config file. It should tell you which file this is when you print the config variable using:

print ConfigVariableBool('lock-to-one-cpu')

What does it say? Maybe we’re leaving this set in the default Config.prc that ships with Panda.

David

8 
>>> print ConfigVariableBool('lock-to-one-cpu')
ConfigVariable lock-to-one-cpu:
  lock-to-one-cpu #f  (from /c/Panda3D-1.7.0/etc/Config.prc)
  lock-to-one-cpu #t  (from /c/Panda3D-1.7.0/etc/Confauto.prc)
  lock-to-one-cpu 0  (default value)

Looks like it’s the Confauto.prc file.

-Greg