Debugging Panda under Win platform?

C++ coding using Panda3D
21

Let me clear up a couple of points of confusion:

(1) You don’t actually need to have a “Debug” build in order to step-debug and set breakpoints. You can do this perfectly well with a “Release” build. The only problem is that in a “Release” build the code will have been compiler-optimized, and so the assembly code won’t precisely match the C++ code, and you get some weird behaviors from time to time. Also, watch variables are frequently reported incorrectly.

(2) If you want to debug with a true “Debug” build, you need to have Panda and Python both built in “Debug” mode. By Python and Panda convention, the dll’s also must be named _d.dll; if makepanda doesn’t do this, it’s a bug, and it means makepanda can’t produce a working Debug build. You’ll have to use the ppremake system to make a Debug build of Panda, or fix the makepanda bug.

(3) Making a “Debug” build of Python should be easy. Get the MSVS project file, and pick “Debug” from the build menu. Click “Build Project”. This is all I had to do last time I build this. It will produce a program called python_d.exe, which you have to run (instead of python.exe) to use your debug Panda build.

David

22

Okay, I’ll fix it. Does it only apply to the dlls, or also to the .lib files? And to the .exe files (e.g. pview_d.exe)?

Would this also need to be changed on linux and OSX platforms? Currently, the RPM release builder builds everything in debug mode, then strips the debug symbols and saves it to a separate file - so we can quickly provide a not-so-large panda3d-dbg package containing just the debug symbols.
We would lose this benefit if we made it _d.so as well there (although it’s not such a big deal if our -dbg packages become a little bigger).

23

No, this is Windows-only. The reason Python has adopted this funny naming convention on Windows (and we have followed the convention) is that on Windows, Debug and Release builds are not generally cross-compatible; they use completely different heap calls. This means that you can’t link a Debug program with a Release dll, and vice-versa (unless you design a very narrow API that doesn’t involve passing ownership of heap objects across the DLL call). Furthermore, if you do attempt to mix-and-match programs like this, it appears to work but it just crashes mysteriously, usually in malloc() or free().

So, on Windows, Python names everything _d in Debug mode to prevent accidentally linking the wrong-heap program. It is necessary to apply the same name to .lib as well as .dll files. Python also applies the name to the .exe files, though we haven’t gone that far with Panda; we still name our .exe files without the _d.

David

24

That definitely appears to be the case with makepanda under Windows.

I didn’t even know what ppremake was; I assumed it was part of the automatic build process and didn’t realize it was an alternate method for compiling Panda using the Cygwin dev toolset.

I missed that INSTALL-PP document.

Okay, I did that. But what about all of the libraries that Panda links against, such as python25.lib and the rest that exist in win-python/libs? What about TCL/TK and the other dependencies? Is none of that necessary and I can just use the bare-bones Python executable?

I mean, I haven’t just tried to take my bare-bones python_d.exe and run it against the --optimize 1 compiled Panda library. I just assumed that wouldn’t work for the reason you already mentioned; You need to either go all-or-nothing when it comes to mixing debug and release library stubs. (ie: No using a release .lib to load a chunk of debug DLL code, due to heap manager differences, different MSCRTs, etc.) So having Panda3d link against the default python25.lib found in win-python wouldn’t work when python_d.exe was invoked and it loaded python25_d.dll, etc.

Or am I just making this much more harder than it really needs to be? In other words, the actual linked interface in your Python version doesn’t matter too much because Panda3d doesn’t actually use it? (Or it’s used in a manner that doesn’t matter if a debug DLL is loaded with a release-compiled library.) I know Panda3d as a part of makepanda will link programs to python25.lib in the win-python/libs directory, so I just assumed it was an issue.

I guess my real confusion comes from the fact that makepanda.py is a bit opaque and monolithic for me. It’s not as easy and poking around in makefiles to see exactly what is being pulled in from where. I also do not know anything about using Python as a developer, so I am not sure what sort of work is needed to fold it into a project. It’s a bit of a philosophy difference, I guess, since I do know that Python packages like to compile from .py files. I’m more familiar with a traditional UNIX-y environment (make + build tools) or Windows (usually, MSVC-based projects), or a mix of the two (like ppremake, UNIX make with MSVC compiler/linker).

As for the Windows/(practically every other OS) differences, I wasn’t aware that you could mix and match a debug .so with a release .a so easily (or vice versa), so I suppose this indeed must have been confusing to people who primarily work in those other OSes.

25

I don’t see how you are doing this, David.

I dropped python_d.exe and python25_d.dll into my debug\python directory and invoked my program from python_d.exe. It complained about not being able to load the first Panda library it encounters, which is libpandaexpress:

Traceback (most recent call last):
  File "C:\Panda3D\games\projshadows\src\projshadows.py", line 1, in <module>
    from pandac.PandaModules import loadPrcFileData
  File "C:\Panda3D\latest\debug\pandac\PandaModules.py", line 1, in <module>
    from libpandaexpressModules import *
  File "C:\Panda3D\latest\debug\pandac\libpandaexpressModules.py", line 1, in <module>
    from extension_native_helpers import *
  File "C:\Panda3D\latest\debug\pandac\extension_native_helpers.py", line 54, in <module>
    raise ImportError, message
ImportError: Cannot find libpandaexpress_d.dll
[7685 refs]

So I renamed libpandaexpress.dll to libpandaexpress_d.dll (considering our discussion above), and now stuck at:

Traceback (most recent call last):
  File "C:\Panda3D\games\projshadows\src\projshadows.py", line 1, in <module>
    from pandac.PandaModules import loadPrcFileData
  File "C:\Panda3D\latest\debug\pandac\PandaModules.py", line 1, in <module>
    from libpandaexpressModules import *
  File "C:\Panda3D\latest\debug\pandac\libpandaexpressModules.py", line 1, in <module>
    from extension_native_helpers import *
  File "C:\Panda3D\latest\debug\pandac\extension_native_helpers.py", line 81, in <module>
    Dtool_PreloadDLL("libpandaexpress")
  File "C:\Panda3D\latest\debug\pandac\extension_native_helpers.py", line 79, in Dtool_PreloadDLL
    imp.load_dynamic(module, pathname)
ImportError: Module use of python25.dll conflicts with this version of Python.
[7689 refs]

I downloaded and built from the 2.5.2 archive found on Python.org.

Edit: Checking dependencies, I see that libpandaexpress is linked against PYTHON25.DLL, so I guess I definitely need the python25_d.lib stub to make this work. That means digging through makepanda to find out where to convince it to link against python25_d.lib instead of python25.lib.

There is definitely more than just dropping python_d.exe in and going with this.

26

Right, you can’t mix-and-match Debug and Release builds. That means you can’t run python_d.exe (a Debug build) against libpandaexpress.dll (a Release build), even if you rename the DLL.

But you can use python.exe against libpandaexpress.dll, and run this in the MSVS debugger. If you have the pdb’s from the Panda build, you can even step through and set breakpoints and such.

Alternatively, since you do have a python_d.exe, you can build your own version of Panda as a Debug build, producing libpandaexpress_d.dll et al, and you can then run this in the MSVS debugger as well.

Either route will work fine, though the Debug build will behave a little bit more predictably in the debugger.

David

27

Appearently, I don’t understand how to generate a debug build of Panda.

I am using the MSVC makepanda solution for Debug, which runs:

–everything --optimize 1 --outputdir debug

But this isn’t generating libraries such as libpandawhatever_d.dll and it’s trying to link against the non-debug Python stub files.

So what am I doing wrong?

Update: These ARE release dlls, not debugs. They’re linked against things like MSVCR90.DLL. Grrrrrrr. They just had the .pdb symbols generated for them, which is why I could see code via attaching to the non-debug Python.

28

Er, the win-python we are shipping contains a hacked pyconfig.h which always loads python26.lib (or 25 on non-latest-cvs) even when it’s debug mode.
If you have a debug build of Python, replace the pyconfig.h with the version from your builld.

EDIT: I’ve compiled my own debug build of Python - I’ll send it to you later, after I fixed up makepanda.

29

I’ve just fixed makepanda, a lot of stuff was wrong with it. It indeed used the release version of the standard library in optimize 1 option. It now also adds the _d suffix to .dll, .pyd, .mll, .dlo, .dli and .dle files on Windows.

I’ve checked the changes into CVS. Here’s the Python debug build:
152.10.254.215/~pro-rsoft/win-python-dbg.zip
Extract it into thirdparty. I’ve altered makepanda to use “thirdparty/win-python-dbg” when in optimize mode 1 or 2.

I’m running into linker errors on libpandaode and the plugin, so compile with --no-ode --no-tinyxml. (Unresolved external symbol __imp___invalid_parameter_noinfo, I haven’t found the cause of it yet)
I didn’t test running it yet (as compile did not complete yet here), so let me know if anything doesn’t work right.

30

Cool, I will try out these changes. I had already started to modify makepanda (such as adding /MDd flags to compile with debug runtime libraries), but I trust your ability to poke around in this far more than mine! So I just updated from the CVS.

I checked your binaries for the debug of Python – They don’t look any different than what I had come up with, save you have a modified pyconfig.h that correctly pulls in the debug stub library. So I am guessing we don’t need all of the other libraries, just the python25(_d).lib stub.

I missed the pyconfig.h. You had pointed me to pyconfig.h before but I didn’t actually search for the library in there. It never occurred to me that the actual lib file would be added to the build chain via a header file. I just thought you meant it compiled in some kind of flag that was later picked up by makepanda which would add in python25.lib. I was beginning to think I was crazy because I couldn’t find it in anywhere in makepanda/makepandacore. :stuck_out_tongue:

31

jGenPyCode.py is failing.

1>Generating dependencies...
1>thirdparty/win-python-dbg/python_d.exe direct\src\ffi\jGenPyCode.py -r libpandaexpress_d libpanda_d libpandaphysics_d libpandafx_d libp3direct_d libpandaskel_d libpandaegg_d
1>Importing code library: libpandaexpress_d
1>Traceback (most recent call last):
1>  File "direct\src\ffi\jGenPyCode.py", line 94, in <module>
1>    DoGenPyCode.run()
1>  File "C:\Panda3D\latest\debug\direct\..\..\direct\src\ffi\DoGenPyCode.py", line 298, in run
1>    generateNativeWrappers()
1>  File "C:\Panda3D\latest\debug\direct\..\..\direct\src\ffi\DoGenPyCode.py", line 258, in generateNativeWrappers
1>    Dtool_PreloadDLL(moduleName)
1>  File "C:\Panda3D\latest\debug\direct\..\..\direct\src\extensions_native\extension_native_helpers.py", line 77, in Dtool_PreloadDLL
1>    raise ImportError, message
1>ImportError: DLL loader cannot find libpandaexpress_d.

jGenPyCode.py is being passed the DLLs named whatever_d.dll from makepanda.py in RunGenPyCode, but the extension_native_helpers.py has been set up to examine its calling Python. (If it detects its been started with python_d.exe, it automatically appends _d to the incoming DLL.)

I’m not sure how to cleanly resolve this, as the target list needs those DLLs to contain the _d extension during the build chain, but not at the dependency generation.

So I guess just strip off the _d if it finds it at that point?

Update, here is my modified, utterly-untested RunGenPyCode:

def RunGenPyCode(target, inputs, opts):
    if (PkgSkip("PYTHON") != 0): return
    
    cmdstr = SDK["PYTHONEXEC"] + " " + os.path.join("direct", "src", "ffi", "jGenPyCode.py")
    if (GENMAN): cmdstr += " -d"
    cmdstr += " -r"
    for i in inputs:        
        if (GetOrigExt(i)==".dll"):
            basedll = os.path.basename(os.path.splitext(i)[0].replace(GetOutputDir()+"/lib/",""))
            # strip out the _d in case it is present
            dstrip = basedll.partition ("_d")
            basedll = dstrip[0] + dstrip[2]
            cmdstr += " " + basedll
    
    oscmd(cmdstr)

PS: Success! That generated a successful debug build. I can debug successfully now in MSVC, set break points, and I get an accurate stack/variable watch due to matching runtimes. I can finally start doing some serious debugging!

32

Awesome! I’ve just applied a fix for this to CVS.

33

About __imp___invalid_parameter_noinfo: cowboyprogramming.com/2007/02/22 … rid-of-it/

34

I know this thread has been inactive for awhile, but after some wrangling, the latest version of the CVS source built in Debug mode for me after I disabled a bunch of things. Also it would only build inside MS VS2008. Not from a command prompt. Hers what I used

cd .. & makepanda\makepanda.bat --everything --optimize 1 --no-dx9 --no-dx8 --no-openal --no-openssl --no-fmodex --no-ffmpeg --no-ode --outputdir debug

Also, is there any way of posting a debug version of the thirdparty libs on the downloads page? Most importantly a debug build of python to make things a bit easier for those of us editing the source? I had to use an old link I happened upon, because trying to build debug python from that CVS tree is a bit of a nightmare. Just thought I’d offer that up. Any comments?

~andrew

35

yes. 1.7.2 is much better. this is what i had to do:

grab win-python-dbg from user rdb from here: http://152.10.254.215/~pro-rsoft/win-python-dbg.zip
copy the unzipped win-python-dbg to c:\Panda3D-1.7.2\source\thirdparty\

open solution makepanda\makepanda.sln
for some reason this wouldn’t open for me directly. so i made a shortcut with the target set to:
“C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe” C:\Panda3D-1.7.2\source\makepanda\makepanda.sln
choose the Debug solution configuration.

right click on makepanda project. choose properties. open Configuration Properties. choose NMake. set Build Command Line to:
cd … & makepanda\makepanda --everything --optimize 1 --outputdir debug --verbose --no-gl --no-dx8 --no-fmodex --no-ode
also in the properties dialog. click on Debugging.
change Command to:
C:\Panda3D-1.7.2\source\debug\python\ppython_d.exeand Command Arguments to:
-i Tut-Asteroids.pyand Working Directory to:
C:\Panda3D-1.7.2\source\samples\Asteroids
since i’m building a dx9 only version, edit makepanda\config.in. change:
load-display pandagl
#load-display pandadx9to:
#load-display pandagl
load-display pandadx9

hit f5 and play asteroids.

36

Hey RDB, any chance you’ve got a prebuilt win-python-dbg for python2.7 to use with Panda 1.8 ??

37

Sorry, no. You’ll need to grab the Python 2.7 source from the Python download page and compile it using VS 2008 in debug mode. It should be relatively straightforward.