running package and .pyd

Hi,

I’m back with a new incomprehensible issue.

I’m on the amazing packaging panda3d’s functionality.
The ppackage tools seems to run into a perfect way. With multify I can see that all my included files are in the package.
The only one warning given by ppackage is :

:Packager(warning): Unable to determine dependencies from /c/.../cmmclient.pyd

cmmclient.pyd is a binded c++ dll for python with the interrogate tool.
Also, I payed attention to use mt.exe and dumpbin.exe from MSVC as asked by ppackage.

After that I try to run my freshly generated p3d. But the following error comes up :

import cmmclient
  File "VFSImporter", line 127, in load_module
  File "VFSImporter", line 228, in _import_extension_module
ImportError: DLL load failed: The specified module could not be found.
Failure on startup.

It works fine when I run it without packaging and I have reduced my python code to the only one line ‘import cmmclient’… I have also tryed to force PYTHONPATH, PATH and others (because of my lack of understanding) to look at many place to find my ‘.pyd’ wthout success…

Also it was working on linux some times ago so I really don’t understand what’s going on !

I don’t know what to try know… any ideas ??

vash

Could you run this command, and show us the output?

dumpbin /dependents cmmclient.pyd

Ok the command give me an error that says it could not find mspdb100.dll
I have added the path to this dll. The warning on ppackage has gone !
But the final issue is still the same.

Here is the result of the dumpbin command :

Dump of file build/bin/Windows/Release/cmmclient.pyd


File Type: DLL

  Image has the following dependencies:

    cmm.core.dll
    cmm.producer.dll
    cmm.graph.dll
    cmm.taskgraph.dll
    cmm.resource.dll
    cmm.dem.dll
    cmm.detection.dll
    cmm.house.dll
    cmm.graphcontroller.dll
    cmm.ortho.dll
    cmm.util.dll
    cmm.calibration.dll
    libp3dtoolconfig.dll
    libpandaexpress.dll
    libp3dtool.dll
    python26.dll
    MSVCP100.dll
    MSVCR100.dll
    KERNEL32.dll

  Summary

       16000 .data
       43000 .rdata
       1C000 .reloc
        1000 .rsrc
       B1000 .text

Also the pyd’s linked .dll were added to the package. That’s a good news, but it don’t solve the import error…

mmm, also I using Visual 2010… So it may cause a conflict between MSVCP100.dll and MSVCP90.dll downloaded from panda3d (because I know from the manual that panda3d were built with VC9). What do you think ?

I’am currently trying to build my entire project under VC9 2008 express (not very convenient because of many external libs dependencies !).
Know I’m asking : why a VC10 compiled version of panda3d does not exists ? It’s very restrictive that panda3d impose his own building environment (no cygwin or other visual compiler… and also a specific version of ubuntu for Linux…).
But this topic is maybe in the pipe for the next release ? :slight_smile:

OK I have now compiled all under VC9.
The problem is still the same.

I have split my project in order to see which linked dll cause the ErrorImport on my ‘.pyd’.

It seems that the error raised when pthreadvc2.dll is linked.
Also, I suspect that panda3d use pthread too (I found pthread.h files and some others .impl that lead me to this conclusion). I may be the cause of a conflict… ?

If someone can give me any clues on this hairy building issue, I would be very grateful.

Unfortunately, we don’t currently support embedding a compiled DLL inside a p3d file. You can, however, embed it with a generic Panda3D “Package”, which you host for download on your own server, and which your p3d file can reference.

This is a bit more complicated then simply building everything inside a p3d file, but it’s not really that bad, and it does have some advantages.

David

It’s exactly what I have made. My .pyd is in a panda3d package referenced by a p3d file.

Again, I suspect that the problem comes from the panda3d compiling environment that could be different from mine.
I think that pthread version used when compiling panda3d is different that my pthread version, so that make things incompatible…

I don’t know if you understand what I mean… ?

Is someone know the exact version of pthread that is used for building panda3d ? Then I could use that version to compile my dll.
Here is a link to the downloadable pthread lib for win32, maybe it’s one of this one ?
ftp://sourceware.org/pub/pthreads-win32

I don’t think we use pthreads on Windows…

There is some “#include <pthread.h>” and pthread functions in panda3d sources (atomicAdjustPosixImpl, mutexPosixImple, …) or maybe it’s only for Linux/Mac ??

I’d think that’s only for Linux, Mac OS X and FreeBSD.

Ok so what’s going on with pthread ?
Why when it’s linked to my pyd I’ve got this ugly ImportError ?? It’s very strange.

Ok… I was wrong. Pthread is not the problem but a part of the code that depends on it.

After a dichotomous search (comment/uncomment half of the code to find the line that generate the error), I finaly found that the error raise when a class is derivated from the panda3d ReferenceCount class.

I try this :

class _COREAPI_ Object
{
}

It works !

class _COREAPI_ Object : public ReferenceCount
{
}

Generate an ImportError (as usual, only when packaged !!)

Now I’m asking : What the HELL ?!
Notice that there is NO published member functions added, NO call to any constructors… only an python import to the .pyd

I’ve dug depper, and I tried to inherite from the class Buffer that itself inherite from ReferenceCount :

class _COREAPI_ Object : public Buffer
{
}

It works !! (??)

And I’ve got the same problem with PointerTo, PointerToArray…

Is there a flag or somethings to compile with those classes ??

Which version of Panda3D are you linking against? I’m not sure if the runtime distribution build is ABI compatible with the SDK builds, which is probably causing seemingly arbitrary issues on your end.

One big thing that’s different between the Runtime build and the SDK build is the NDEBUG macro. The SDK is built with OPTIMIZE=3, while the Runtime is made with the OPTIMIZE=4 setting.

There are some subtle differences between OPTIMIZE 3 and OPTIMIZE 4, and one major difference: NDEBUG. To run successfully with an OPTIMIZE 4 build, you need to have NDEBUG enabled in your project settings, which is the opposite advice for running against the normal SDK build.

Try setting NDEBUG when compiling your pyd. You do need to continue to use VC9.

David

@rbd : I’m linking to panda3d-1.7.2

@drwr : I have tryed the NDEBUG flag on VC9… not better.

I don’t know what sort of magic there is inside, but it’s powerfull ! :s
Now I’m looking for good old tricks to pass through this… but for each obstacle passed another is facing !
It’s crazy… I’m probably missing somethings somewhere.

Try copying the Panda3D DLL’s out of the p3d installation directory (which is c:\Users<your name>\AppData\Local\Panda3D\hosts… and somewhere under there). If there’s any doubt, you can find the precise name in the log file (in AppData\Local\Panda3D\log).

Copy these DLL’s to another directory on your hard disk, and run your application interactive from that directory. You should get the same ImportError. With this, you can more easily research precisely what is going wrong with the import.

David

I found it !! Finally !

We were right, it was a flag usage things.
I have built my dll with “#undef DO_MEMORY_USAGE” instead of “#define DO_MEMORY_USAGE”. And that’s working :slight_smile:

To found this, I have created a small exe that use the Windows LoadLibrary function to open each of my dll… when my core dll were linked to the dll downloaded from runtime.panda3d.org I get the a real explicit error on a missed function. It was one in MemoryUsage…

In the future I’m thinking of having my own panda3d compiled and packaged version, host from my side. It should give me a perfect compatibily and control over those flags and building environment.

Thanks again for your help.

vash

Great, I’m glad you found it, and it’s good to know about those flags. Now that you say it, I remember that I should have known that DO_MEMORY_USAGE is also undefined in the OPTIMIZE 4 case. :frowning:

While this is indeed a viable option, I’d encourage you to use the standard Panda distribution if at all possible. Now that you know the proper compilation settings this should be easy to do, and doing so will be friendlier to your users, who may have already downloaded the standard Panda3D from something else and won’t have to download another version needlessly.

David