Issue with multiprocessing.Connection when running as a p3d

Scripting Issues
1

Hello,

I’ve been working on a project with Panda3d for many months now, and I’m very happy with it! That I haven’t had a need to ask anything until now is a credit to how good the documentation is and how easy working with panda is.

That said, I’ve encountered a problem. :smiley:

I’m using multiprocessing to handle network requests in their own process. This works fine in Panda3d 1.8 when running directly with ppython.

Unfortunately, if I package it into a p3d it then fails. No matter how I run it in p3d the result is the same: browser plugin, panda3d executable, or an executable created with pdeploy standalone.

Both the working and non-working case are with panda 1.8 on win32.

The error occurs the moment I create a multiprocessing.Pipe: the specific line triggering it in my case is:

  self.connectionPipe,rppipe = Pipe()

Which works as desired when run without going through a p3d file.

The traceback it creates is:

Traceback (most recent call last):
  File "C:\buildslave\release_rtdist_win32\build\panda3d\built_cmu\direct\p3d\AppRunner.py", line 638, in run
  File "C:\buildslave\release_rtdist_win32\build\panda3d\built_cmu\direct\task\Task.py", line 502, in run
  File "C:\buildslave\release_rtdist_win32\build\panda3d\built_cmu\direct\task\Task.py", line 460, in step
  <creating of the pipe for my network process>
  File "c:\python27\lib\multiprocessing\__init__.py", line 107, in Pipe
  File "c:\python27\lib\multiprocessing\connection.py", line 222, in Pipe
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect
:TaskManager: TaskManager.destroy()

It appears that something about how a p3d is run confuses multiprocessing when it tries to create a temporary file to use as a windows named pipe. Normally multiprocess creates it based on os.getpid() and an itertools counter. Does anything unusual happen that might interfere with one or both of these when running a p3d?

Thank you,
-Rick

2

Hmm, I’m not entirely sure what Python’s multiprocessing module is doing under the hood, but if it’s trying to spawn another instance of Python I do find it easy to believe that this might have problems in the p3d environment. This is because we’re running embedded Python, from a custom runtime environment; it’s not at all a straightforward Python process, and a child process might have to do special things to launch correctly.

I haven’t looked into this at all before. It’s possible that there’s just a simple thing that’s getting in the way and which is easily fixed; but it also might be that we will have to create a custom multiprocessing module in order to simulate Python’s standard behavior.

David

3

Normally, multiprocessing uses fork(), but since Windows doesn’t have it, it’ll try to find the location of the Python executable in order to spawn another copy of Python. Since we embed Python, that doesn’t work for us.

4

Thanks for the responses!

For now I’m working around the issue by not using multiprocessing when base.appRunner is around, which isn’t ideal but is fine for my purposes for the foreseeable future.

I’ll come back to it in a while and take a closer look, and at that point I’ll follow up here with what I find (and possibly a way to get it to work, if I find one) in case anyone trying it in the future can benefit.

-Rick