Build corrupted with optimize=4 on Windows?

Compiling or Editing the Panda Source
1

I have encountered a weird behaviour with my Windows build of Panda3D SDK. If I compile with --optimize=4 panda3d seems broken. For example, the content of MAIN_DIR env variable changes :

With the following program :

from panda3d.core import ExecutionEnvironment

main_dir = ExecutionEnvironment.getEnvironmentVariable("MAIN_DIR")
print("MAIN", main_dir)

from direct.showbase.ShowBase import ShowBase

main_dir = ExecutionEnvironment.getEnvironmentVariable("MAIN_DIR")
print("MAIN", main_dir)

The output with optimize=3 is :

c:\msys64\home\ld\cosmonium>c:\panda3d-1.10.7-x64\python\python.exe test_main.py
MAIN C:\msys64\home\ld\cosmonium
MAIN C:\msys64\home\ld\cosmonium

But with optimize=4 :

c:\msys64\home\ld\cosmonium>c:\panda3d-1.10.7-x64\python\python.exe test_main.py
MAIN C:\msys64\home\ld\cosmonium
MAIN C:\

I’m buiding Panda3D using VS 2015 and the following command (using Python 3.7 from the thirdparty bundle):

makepanda/makepanda.py \
--everything \
 --msvc-version=14 \
 --windows-sdk=7.1 \
 --no-eigen \
--nocolor \
--use-sse2 \
--threads=4 \
--optimize=4 \
--installer \
--lzma

Has anyone encoutered something like that ?

2

Could it be some kind of cross-build pollution? Could you try the optimized build from here and let me know whether it has the same problem?

For the record, targeting the Windows 7.1 SDK with Python 3.7 is a little pointless, because Python 3.7 requires Vista anyway. However, it should still work in principle (I think we do it too for the official 1.10 builds).

3

Sadly the problem is still there:

c:\msys64\home\laure\cosmonium>c:\panda3d-1.10.8-x64\python\python.exe test_main.py
MAIN C:\msys64\home\laure\cosmonium
MAIN C:\
Panda version: 1.10.8 (eea0abe4886011827e42b8accc6372a64a0ee997) by cmu (Dec 21 2020 16:02:11)

It’s probably a conflict with something installed on my machine (but what ?) I will try and uninstall everything I can

4

I’m getting the same issue. I’ll investigate.

5

FWIW I also tried with an older build I’m sure was working and now it shows the same problem. So it might be related to a recent Windows update.

6

While I build Panda, are there any other ways in which the build appears to be corrupted, or is the MAIN_DIR issue the only thing?

It’s good that you found this issue now, since I was about to release 1.10.8.

7

Actually I had issue making a new dist for my app, It immediately crashed as it could not import some symbol while loading panda3d.core (but the symbol name wasn’t present in the log). So there might be another problems lurking behind that one…

8

I just built Panda from source with --optimize 4 and it doesn’t have this issue. However, I notice I built with the Windows 8.1 SDK. I’ll rebuild with 7.1. I’ll also make sure I’m using the same thirdparty packages.

9

I confirm, with normal SDK and wheel, the bundled app works fine. However when switching to the 1.10.8 optimized SDK and wheel from above the bundle crashes immediately :

Traceback (most recent call last):
  File "__main__", line 38, in 
  File "importlib._bootstrap", line 983, in _find_and_load
  File "importlib._bootstrap", line 967, in _find_and_load_unlocked
  File "importlib._bootstrap", line 677, in _load_unlocked
  File "importlib._bootstrap", line 819, in exec_module
  File "cosmonium.cosmonium", line 23, in 
  File "importlib._bootstrap", line 983, in _find_and_load
  File "importlib._bootstrap", line 967, in _find_and_load_unlocked
  File "importlib._bootstrap", line 677, in _load_unlocked
  File "importlib._bootstrap", line 819, in exec_module
  File "direct.showbase.ShowBase", line 40, in 
  File "importlib._bootstrap", line 983, in _find_and_load
  File "importlib._bootstrap", line 967, in _find_and_load_unlocked
  File "importlib._bootstrap", line 677, in _load_unlocked
  File "importlib._bootstrap", line 819, in exec_module
  File "panda3d.core", line 1, in 
  File "imp", line 342, in load_dynamic
  File "importlib._bootstrap", line 696, in _load
  File "importlib._bootstrap", line 670, in _load_unlocked
  File "importlib._bootstrap", line 583, in module_from_spec
  File "importlib._bootstrap_external", line 1043, in create_module
  File "importlib._bootstrap", line 219, in _call_with_frames_removed
ImportError: DLL load failed:

This is with the pure Python version of the app.

10

Sorry, I forgot to add that I can not reproduce that problem with the asteroid sample however.

11

Well, I reproduced the MAIN_DIR issue. Continuing to investigate.

So, cosmonium fails to import panda3d.core, but other applications don’t? Could it be a stale cache in the build directory? If not, could you find out whether hoisting the panda3d.core import before your other imports works?

12

It looks like import panda3d.direct is what ruins MAIN_DIR. For some reason, it thinks that MAIN_DIR wasn’t set, and tries to compute and set it again, but for some bizarre reason, the second time, filename.make_absolute() fails.

This may be some kind of strange linking issue between the various DLLs.

13

I tried that, but it didn’t change anything. So, I removed everything I could until the build worked again and slowly added everything back… And now it works fine with the opt build with everything included again (and I removed the build directory between each try) !
I will try again later with a fresh setup to be on the safe side

14

All evidence points to a compiler bug regarding the MAIN_DIR issue. Apparently random changes can get the bug to trigger or not to trigger.

Let me see if I can make the issue reliably disappear with a particular random change. And maybe we need to be using a more recent compiler for the public builds.

15

Well, I fixed the bug that made Panda set the MAIN_DIR more than once; since it was setting it correctly the first time, maybe this is enough to fix it? I’m a little afraid that this problem might rear its ugly head again in some other location.

New wheels are sprouting up here, mind trying to build with that?

I’ll wait a day or two before releasing 1.10.8 so that you have time to test and report any more issues.

Thanks!

16

So far no more issue :slight_smile:

I tested with the optimized SDK and wheel; and both the source and the build_dists bundle work fine.
I also rebuild my custom version of Panda3D and again no more issue encountered.

I hope this is the last time we see it, compiler bugs like that one are a nightmare to find and work around… I think I will look at switching to VS 2017 in the future

17

All right, thanks for your testing!