using packp3d to pack py+pyd+dll

Hi,

currently im stucked while trying to pack my current project ( dl.dropbox.com/u/6841319/reloaded.zip ) into a p3d file. Im using following commands:

packp3d -o sf.p3d -d C:/SF/reloaded -m start.py -n dll -n pyd

panda3d.exe sf.win32.p3d

The outputs of them are:

If i remove -e pyd or -e dll the size of the p3d changes so they should be really included. If I remove the pyd file it cant find the module sf_loader so it reacts correctly. If I remove the dll the output does not change so I guess it is unabled to load the dll correctly. I already tried to rename the dll but the behaviour did not change. Changing -e to -n does not work too.
Is there a way to check if the dll is correctly packed into the p3d? If I add the dll to the System32 Folder the application seems to find the dll but gets stucked (no window is opened, cpu-cycles are still consumed).

P.S: the last issue was due to “if name == ‘main’:”

Usually I submit

multify -x -C folderName -f fileName.p3d

in order to extract files from p3d (and so I can check which files are included).

I think you will need to have MSVS Express installed in order to properly package up a p3d file that includes dll’s and pyd’s. This is because MSVS includes tools like dumpbin.exe and mt.exe which are necessary to determine all of the dependent dll’s required by each dll or pyd that you include.

It’s also a little troubling that you have a file named sf_loader.py as well as one named sf_loader.pyd (even though they are in different directories). I’m not sure that the p3d system can handle this well. Can you try changing the name of one of them to ensure that they are kept different from each other? It’s probably a good idea to do this anyway to prevent confusion in your own code. Commonly, pyd files that implement some low-level functionality are named with a leading underscore: _sf_loader.pyd.

Note, of course, that including pyd’s and dll’s will constrain your p3d file to be Windows-only; it will no longer be platform-independent. Of course you knew this already. :slight_smile:

edit: also note that you can easily inspect the contents of a p3d file (which is just a Panda3D multifile, after all) without unpacking it with the command:

multify -tvf myfile.p3d

David

@yaio thx, the dll is inside but it cannot be found. Maybe due to the fact that the packp3d.exe is defining the startup dir, not the path of the p3d-file?

@drwr Well I have MSVS installed (because I developed the sf_loader.pyd). Furthermore I only see a sf_loader.pyd, no sf_loader.py at all :open_mouth: .
Im aware of the dependency-problem ofc, but the c++ code itself is easy to port.
Thanks for the -tvf hint :slight_smile:

If I workaround the dll problem by putting the boost dll in the system32 folder (which is not really a general solution!), I can start the app but for any reason it raise this:

Why does it search a file in a path which does not exist? Are python26 and C:\panda3d-1.7.0\built_cmu\ virtual paths, added by the packp3d? (file is a variable name in this case! but it seems like it wants to resolve it as class/module)

OK, but perhaps the MSVS tools are not on your path, because you can see it complaining about not being able to find “dumpbin” or “mt”. Try running packp3d from a MSVS command shell, or explicitly add the MSVS bin directory (which contains these tools) to your PATH.

There is no sf_loader.py in your source tree? Then something is getting confused, very interesting. That might be due to the “-n pyd”, which really shouldn’t be necessary. It should work properly without this, as long as the sf_loader.pyd file can be located by the packp3d tool.

Try replacing this line:

parser = importer.DesReader("TXT/DEEPTEXT.DES") 

with:

parser = importer.DesReader(base.appRunner.multifileRoot + "/TXT/DEEPTEXT.DES") 

This is necessary because when you read files from the p3d file, they are not found in the current directory, but instead in the directory named by base.appRunner.multifileRoot. (We will change this in the 1.7.1 release.) Also, make sure that the case is exactly right; the p3d file system is case-sensitive.

The error messages are confusing you a bit here. ptyhon26 and built_cmu are not real paths at all, not even virtual paths; they’re just names that are leftover from a previous build (Python records the full pathnames from when the source was compiled, and reports them again later, even when those pathnames are no longer relevant). Also, file is indeed a variable name at one level of the stack trace, and then a module name in a lower level, but that’s just a detail; in fact it is working as intended but the file you are trying to load is indeed not located within the current directory.

David

I’m not sure I understand the question (excuse me, but my english is very bad), so excuse me if I don’t get the point (and ignore my message). Windows searches dlls into well-defined places, and obviously it doesn’t look into p3d files. I don’t know if Panda’s developers now are “exposing” dll files contained into p3d files to Windows (maybe not, because when you placed the files into System32 the application can find them). If this is not the case, when you deploy your application (e.g. when your users will launch your installer), you can put needed dlls into “right” places (exactly what you did manually when you put the files cited into System32).

Yeah, actually, the runtime system does automatically unpack any dll files found into a temporary folder and puts that folder on the PATH. So it’s supposed to work. The problem is if all of the dependent dll’s don’t get automatically detected, which will happen if dumpbin.exe is not found on the path (as appears to be the case in this instance).

David

Wow David, this is so cool! I had been mislead by this specific example: if he manually placed libs it works (so I thought that dependencies were already in PATH), and so I thought that if Panda made analogous actions (placing libs in a right place) dependencies should be resolved. But surely I’m missing something. Anyway, the good news for nox_firegalaxy is that my post was useless. :slight_smile:

@drwr If I use packp3d within the msvs console it finds the tools and packs the files inside correctly. But if I remove boost from system32 it misses again boost.
Maybe it ignores the PATH variable for any reason although msdn.microsoft.com/en-us/library/ms682586(VS.85.aspx claims that the PATH variable is included in the search :confused:

TO the loading issue:
If I use absolute paths I can surpass this exception but run into another:

If I use the ‘base.appRunner.multifileRoot + “/TXT/DEEPTEXT.DES”’ it cannot load the file again :confused:
Im a bit helpless… :blush:

Include the encodings package by specifying -r morepy on the packp3d command-line. Include files with a .des extension by specifying -e des.

@rdb -r morepy works perfectly, thanks! Seems like I can remove the ‘-p C:/panda3d-1.7.0/python/lib’ for the ConfigParser too :slight_smile:

I have to load the DES (and many other) files from extern. No way to add them.
‘print base.appRunner.multifileRoot’ puts /MT out btw.

EDIT:
I got in trouble with the -r audio issue (with openAL I get no sound but with FMod getSound on a UserDataAudio-instance returns only ‘None’) and the “with open” (missing exit in the file.py) bug.
Is the openAL issue solved too and ow likely is it that 1.7.1 is released this month?

Does it give you any error messages at packing time about not being able to locate the dll? I guess you can work around this by copying the boost dll into your own project directory before packing it.

I don’t know what you mean here. Why can’t you add the des files, exactly?

Actually, it’s “/mf”, but yes, that’s what it’s supposed to print. I say again that case is important: /MF is very different from /mf. So when you attempt to open base.appRunner.multifileRoot + “/TXT/DEEPTEXT.DES”, are you sure that your file is really named “TXT/DEEPTEXT.DES” and not “txt/deeptext.des”?

I haven’t heard about a resolution for the OpenAL issue, though this is the first time I’ve heard about a problem with FMod. Are you sure you have the filename right on your sound file?

David

No there are no output beside “Generating sf.win32.p3d” and the dll is packed into the p3d (checked with multify).

Well I cannot add them because on the one hand they are protected by a copyright and on the other hand some of them are saved games, local configs etc.

I beg your pardon. Its /mf of course, but the path (“TXT/DEEPTEXT.DES”) is correct (checked with “with open”).

I load the sound by a UserDataAudio instance. With OpenAL it works (started from python-console) but with FMod audiomgr.getSound returns None. Additional I get the output: “FMOD audio manager does not support MovieAudio sources”

What kind of audio file are you loading? What is the output of multify -tf on your p3d file?

David

errrrr, they are called .MVI(not the apple stuff)/.PCM/.PCL/.PCR. Had to write my own importer (sf_loader) for them. Related Code which works for openAL and without p3d (self._video.sounddata.data is a buffer):

sound = UserDataAudio(self._video.audio_samplerate, self._video.audio_num_channels)
            sound.append(self._video.sounddata.data)
            sound.done()
            print sound, len(sound)
            self.audioMgr.getSound(sound).play()

So, I don’t see either TXT/DEEPTEXT.DES, or any of your audio files, in your p3d file. Where are you loading these files from?

David

Again: I load them from extern (they are part of the game which i cannot add to the p3d because of legal reasons). Currently I load them using os.chdir(“C:/SF/”). For the python “open” I use "os.getcwd() + “/” + " which works but i dislike the fact that I have a fix workingfolder.
I hope there is a way to get the real(!) path there the p3d is located.
The audio is loaded else an exception would be raised.

Ah, so that’s what you meant when you said you “load them from extern”. My apologies, I didn’t understand that earlier.

A UserDataAudio is used to interface with the MovieAudio system, it is not a direct interface into Panda’s audio layer. As it happens, the MovieAudio system has not yet been integrated with FMod, which explains why it is failing in that case.

In addition to os.chdir(“c:/sf/”), try using vfs.chdir(’/c/sf/’). This is necessary because in a p3d file, the Python file operations are redirected to use Panda’s vfs instead, so it’s necessary to tell the vfs about the new working directory.

David

Yeah the vfs thinks seems to work, thanks :slight_smile:

So the remaining issues:
-OpenAL/FMod
-“open” in file.py
-the boost dll stuff

Only the last one is not really fixed. Currently I work around by adding the dll to the system32 dir but maybe one of you can invastigate the reasons?

dl.dropbox.com/u/6841319/reloaded.zip <- source + dll + pyd ofc it will fail starting because of the missing gamefiles but the dll problem occurs first, so the missing gamefiles should not interfere.

Is there already a release date for a new pandaversion?

I don’t know about this, but I think there have been some holdups. It has been challenging to get releases out recently as CMU has been going through some changes. Things should hopefully settle down by the fall.

David