Setuptools and NumPy 2

[sTorro]

Hi all,

I have to say that I am really enjoying development with Panda3D, it is a great engine. Thanks for all the work that has gone into it.

Excuse my ignorance; this is the first time I’ve used setuptools
I am trying to generate an .exe for my project using setuptools, but the resulting executable crashes immediately on launch, as shown in the attached log file (at the end of the post). The issue seems to be related to NumPy (2.3.5) and its native libraries. I have tried including NumPy in several different ways, and with different Python/NumPy versions, but it still crashes on startup.

I also tried using the following code in setup.py to include the NumPy libraries, based on a suggestion I found in this forum (although it referred to an older NumPy version), but it does not help or seem to have any effect:

'package_data_dirs': {
    'numpy': [('numpy.libs/*', '', {'PKG_DATA_MAKE_EXECUTABLE'})],
},

If I remove all NumPy-related code from my project, the .exe is generated and runs correctly, but the models appear rotated, and I suspect this is caused by panda3d-gltf. The model animations do not play at all. I am using .glb files for my 3D models. Is it mandatory to convert them to .bam files, or should the distributable be able to load .glb files directly?

Does anyone have any idea what might be going on? I am honestly a bit lost, and any help would be greatly appreciated.

Also, this working .exe forces me to place my assets folder inside another folder called “Lib”. Why does this happen? I would prefer to have the assets folder next to the .exe file. Is this configurable?

I have also managed to build the project using PyInstaller, it correctly includes NumPy in the build, but it shows the same issues with model rotations and animations (is panda3d-gltf not being included?). I would prefer to continue using setuptools if possible to build the distributable.

In my project I am using the following libraries (latest versions), and I am running Python 3.13.7:

panda3d
#types-panda3d # this provides type hints for panda3d, it is not needed for release builds
panda3d-gltf
pydantic
json5
imgui-bundle
pyperclip
numba # will install numpy 2.3.5

And also the latest version of setuptools.

My setup.py looks like this at the moment; it still contains some commented-out sections from previous attempts:

from setuptools import find_packages, setup

setup(
    name='Whispers',
    # packages=find_packages(include=['p3dimgui', 'p3dimgui.*']),
    # packages=find_packages(include=['src', 'src.*']), 
    # setup_requires=["numpy"],
    # install_requires=["numpy"],
    options={
        'build_apps': {
            'gui_apps': {
                'whispers': 'run_game.py',
            },

            'log_filename': 'whispers.log',
            'log_append': False,

            'platforms': [
                # 'manylinux1_x86_64', 
                # 'macosx_10_6_x86_64',
                'win_amd64'
            ],

            # Specify which files are included with the distribution
            'include_patterns': [
                'assets/**',
                'src/default_config.json5',
            ],

            'rename_paths': {
                'src/default_config.json5': 'default_config.json5',
            },

            # 'include_modules': [
            #     # 'panda3d',
            #     # 'imgui-bundle',
            #     # 'pyperclip',
            #     # 'pydantic',
            #     # 'json5',
            #     'numpy',
            #     'numpy.core',
            #     # 'numba',
            # ],
            'include_modules': [
                'panda3d-gltf',
            ],

            'plugins': [
                'pandagl',
                'p3openal_audio',
                'p3assimp',
            ],

            'use_optimized_wheels': False,
            'prefer_discrete_gpu': True,
            # 'bam_model_extensions': ['.glb'],

            # does not seem to work properly with numpy
            # 'package_data_dirs': {
            #     'numpy': [('numpy.libs/*', '', {'PKG_DATA_MAKE_EXECUTABLE'})],
            # },
        }
    }
)

Everything works perfectly when I run the game from the IDE; only the generated .exe has issues.

These logs contain the latest versions of the dependencies.
Build (python setup.py build_apps) log: Build log - Pastes.io
Game execution log: Game execution log - Pastes.io

Thanks for your time.

[Trent]

so i have requirements.txt

panda3d==1.10.15
numpy==2.1.0

options = {
    'build_apps': {
        'requirements_path': 'requirements.txt',
        'include_modules': ['direct.particles', 'numpy'],

i recommend deleting the build folder so it generate fresh
these are current versions i know compililing to exe works with

here my how my full setup looks like

from setuptools import setup

setup(
    name="fruitlessfields",
    options = {
        'build_apps': {
            'requirements_path': 'requirements.txt',
            'include_modules': ['direct.particles', 'numpy'],
            'exclude_patterns': [
                '**/*.tmp',
                '**/*.bak',
            ],
            'include_patterns': [
                '**/*.vert',
                '**/*.frag',
                '**/*.cg',
                '**/*.egg',
                '**/*.json',
                '**/*.wav',
                '**/*.png',
                '**/*.bam',
                '**/*.ttf',
                '**/*.ptf',
                '**/licenses/*',
                '**/README.md',
                '**/data/icon.ico',
            ],
            'gui_apps': {
                'fruitlessfields': 'main.py',
            },
            'log_filename': '$USER_APPDATA/fruitlessfields/output.log',
            'log_append': False,
            'plugins': [
                'pandagl',
                'p3openal_audio',
            ],
            'use_optimized_wheels': True,

        }
    }
)

[rdb]

What happens when you add an explicit import to numpy._core._exceptions to force this module to be picked up - or add it to include_modules?

[sTorro]

After days of struggling, I finally managed to get this working with a few hacks. Thank you for your answers; they have been helpful. With NumPy 2.1.0 it works. The build log shows some warnings, but I assume they can be ignored. The version of NumPy I was using before does not work at all, so I will stick with 2.1.0.

The next issue I encountered is related to Numba. Below are the setup.py file, the post_build.py script and the hack that must be executed before importing Numba for the first time. Perhaps someone can suggest a better solution.

This is only valid for Windows builds. I plan to deploy on Linux, so I will need to find a better solution, but it works for now.

I have one more question regarding model file extensions. I assume the official way to handle 3D models is to use .bam files, correct? Does that mean the deployed executable must use .bam files? I am using .glb files during development, and I do not mind converting them to .bam, but I want to be sure.

Also, what is the recommended Python version to use with Panda3D? I am currently using Python 3.13.

Thanks in advance for any help.

setup.py:

from setuptools import setup

setup(
    name='MyGame',
    version="0.0.1",
    
    options={
        "build_apps": {
            "gui_apps": {
                'mygame': 'run_game.py',
            },
            
            "platforms": [
                "win_amd64",
                # 'manylinux1_x86_64', 
                # 'macosx_10_6_x86_64',
            ],
            
            # Include patterns - ensure data files are included
            'include_patterns': [
                'assets/**',
                'src/default_config.json5',
            ],
            'rename_paths': {
                'src/default_config.json5': 'default_config.json5',
            },
            
            # Modules to include
            "include_modules": [
                "numpy",
                "numpy.*",
                "imgui_bundle",
                "imgui_bundle.*",
                
                # Numba and its dependencies
                "numba",
                "numba.*",
                "numba.core.config",

                "llvmlite",
                "llvmlite.*",

                "llvmlite.binding",
                "llvmlite.binding.ffi",
                "llvmlite.binding.dylib",
                
                # Other dependencies
                "munch",
                "glfw",
                "pyperclip",
            ],
            
            # Plugins needed
            "plugins": [
                "pandagl",
                "p3openal_audio",
            ],
            
            # Log settings
            'log_filename': 'whispers.log',
            'log_append': False,

            # 'use_optimized_wheels': False,
            'prefer_discrete_gpu': True,
            'bam_model_extensions': ['.glb'], # convert .glb to .bam
        },
    },
    
    # Required packages
    install_requires=[
        "panda3d>=1.10.14",
        "numpy==2.1.0",
        "imgui_bundle",
        "pyperclip",
        "json5",
        "pydantic",
        "numba",
    ],
)   

print("\n" + "=" * 70)
print("BUILD COMPLETE!")
print("=" * 70)
print("IMPORTANT: Run 'python post_build.py' to fix DLL locations.")
print("=" * 70 + "\n")

post_build.py:

import os
import shutil


def copy_dll_to_lib_path(build_dir: str, dll_name: str, lib_subpath: str) -> bool:
    """Copy a DLL from root to the expected Lib/ subdirectory."""
    src_dll = os.path.join(build_dir, dll_name)
    target_dir = os.path.join(build_dir, "Lib", lib_subpath)
    target_dll = os.path.join(target_dir, dll_name)

    if not os.path.exists(src_dll):
        # Try to find it in the venv
        venv_path = os.path.join(
            os.path.dirname(os.path.abspath(__file__)),
            ".venv",
            "Lib",
            "site-packages",
            lib_subpath,
            dll_name,
        )
        if os.path.exists(venv_path):
            src_dll = venv_path
        else:
            return False

    if os.path.exists(target_dll):
        print(f"    [SKIP] Already exists: {lib_subpath}/{dll_name}")
        return True

    os.makedirs(target_dir, exist_ok=True)
    shutil.copy2(src_dll, target_dll)
    print(f"    [OK] Copied {dll_name} to Lib/{lib_subpath}/")
    return True


def fix_build_directory(build_dir: str) -> bool:
    """Fix the build directory by copying DLLs to expected locations."""
    if not os.path.exists(build_dir):
        print(f"  [SKIP] Directory not found: {build_dir}")
        return False

    print(f"  Fixing DLLs in {build_dir}")

    # Fix imgui_bundle (glfw3.dll)
    copy_dll_to_lib_path(build_dir, "glfw3.dll", "imgui_bundle")

    # Also copy to ROOT directory (where VC++ runtime DLLs are located)
    # This is critical because llvmlite.dll depends on MSVCP140.dll etc.
    venv_dll = os.path.join(
        os.path.dirname(os.path.abspath(__file__)),
        ".venv",
        "Lib",
        "site-packages",
        "llvmlite",
        "binding",
        "llvmlite.dll",
    )
    root_dll = os.path.join(build_dir, "llvmlite.dll")
    if not os.path.exists(root_dll) and os.path.exists(venv_dll):
        shutil.copy2(venv_dll, root_dll)
        print(f"    [OK] Copied llvmlite.dll to ROOT directory")
    elif os.path.exists(root_dll):
        print(f"    [SKIP] llvmlite.dll already exists in ROOT")

    # Copy OpenMP runtime (vcomp140.dll) - Critical for Numba
    vcomp_sys = r"C:\Windows\System32\vcomp140.dll"
    vcomp_target = os.path.join(build_dir, "vcomp140.dll")

    if os.path.exists(vcomp_sys):
        if not os.path.exists(vcomp_target):
            try:
                shutil.copy2(vcomp_sys, vcomp_target)
                print(f"    [OK] Copied vcomp140.dll to ROOT directory")
            except Exception as e:
                print(f"    [ERROR] Failed to copy vcomp140.dll: {e}")
        else:
            print(f"    [SKIP] vcomp140.dll already exists in ROOT")
    else:
        print(f"    [WARNING] vcomp140.dll not found in System32! Numba may fail.")

    # Copy llvmlite vendored dependencies (from llvmlite.libs)
    # llvmlite wheels often bundle a hashed msvcp140.dll in .libs folder
    venv_site_packages = os.path.join(
        os.path.dirname(os.path.abspath(__file__)), ".venv", "Lib", "site-packages"
    )
    llvmlite_libs = os.path.join(venv_site_packages, "llvmlite.libs")

    if os.path.exists(llvmlite_libs):
        print(f"  Processing llvmlite.libs from {llvmlite_libs}")
        for filename in os.listdir(llvmlite_libs):
            if filename.lower().endswith(".dll"):
                src = os.path.join(llvmlite_libs, filename)
                dst = os.path.join(build_dir, filename)
                if not os.path.exists(dst):
                    shutil.copy2(src, dst)
                    print(f"    [OK] Copied vendored DLL: {filename}")
                else:
                    print(f"    [SKIP] Vendored DLL already exists: {filename}")
    else:
        print(f"    [WARNING] llvmlite.libs folder not found at {llvmlite_libs}")

    return True


def main():
    print("\n" + "=" * 60)
    print("Post-Build Fix for DLL Locations")
    print("=" * 60)

    script_dir = os.path.dirname(os.path.abspath(__file__))

    # Fix build directory
    print("\n[1] Fixing build/win_amd64/...")
    build_dir = os.path.join(script_dir, "build", "win_amd64")
    fix_build_directory(build_dir)

    print("\n" + "=" * 60)
    print("Post-build fix complete!")
    print("=" * 60 + "\n")


if __name__ == "__main__":
    main()

The hack, execute it before importing numba:

import os
import sys
import logging

def _patch_llvmlite_for_frozen():
    """
    Patch llvmlite to load DLL directly from the executable directory
    instead of using importlib.resources which fails in frozen apps.
    """
    try:
        frozen = getattr(sys, 'frozen', False) or not hasattr(sys.modules.get('__main__', None), '__file__')
    except:
        frozen = True
        
    if not frozen:
        return

    logging.debug("--- Patching llvmlite for frozen app ---")
    
    # Find the DLL
    exe_dir = os.path.dirname(sys.executable)
    
    # Check root first (where we put it in post_build)
    dll_path = os.path.join(exe_dir, "llvmlite.dll")
    if not os.path.exists(dll_path):
        dll_path = os.path.join(exe_dir, "Lib", "llvmlite", "binding", "llvmlite.dll")
    
    if os.path.exists(dll_path):
        logging.debug(f"Found llvmlite.dll at: {dll_path}")
        
        # Add to DLL search path (Critical for dependencies like MSVCP140.dll)
        if hasattr(os, 'add_dll_directory'):
            try:
                os.add_dll_directory(os.path.dirname(dll_path))
                os.add_dll_directory(exe_dir)
                logging.debug("Added DLL directories to search path")
            except Exception as e:
                logging.warning(f"Warning: Failed to add DLL directory: {e}")
                
        # Patch importlib.resources.as_file
        import importlib.resources as ir
        from contextlib import contextmanager
        from pathlib import Path
        
        # Save original if needed
        if not hasattr(ir, '_original_as_file'):
            ir._original_as_file = ir.as_file # type: ignore
        original_as_file = ir._original_as_file # type: ignore
        
        @contextmanager
        def patched_as_file(traversable):
            # Check if this look like a request for llvmlite.dll
            is_match = False
            try:
                name = getattr(traversable, 'name', str(traversable))
                if 'llvmlite.dll' in name or 'llvmlite.dll' in str(traversable):
                    is_match = True
                    logging.debug(f"DEBUG: Intercepting as_file for: {name}")
            except Exception as e:
                logging.debug(f"DEBUG: Exception checking traversable: {e}")
                
            if is_match:
                yield Path(dll_path)
            else:
                with original_as_file(traversable) as path:
                    yield path
                    
        ir.as_file = patched_as_file
        logging.debug("Patched importlib.resources.as_file successfully")
    else:
        logging.warning("WARNING: llvmlite.dll not found on disk!")

_patch_llvmlite_for_frozen()

requirements.txt:

panda3d
numpy==2.1.0

# panda3d-imgui
imgui_bundle
pyperclip

json5
pydantic

numba

[rdb]

Yeah, packaging things that depend on .dll files and other non-Python extensions can be a pain. Once in a while we add specific hacks to build_apps for known packages (like numpy) but they keep changing upstream and it’s hard to keep up. Glad you managed to figure it out.

@sTorro

The next issue I encountered is related to Numba. Below are the setup.py file, the post_build.py script and the hack that must be executed before importing Numba for the first time. Perhaps someone can suggest a better solution.

The recommended way to copy additional data files is using package_data_dirs, for example:

'package_data_dirs': {
    'llvmlite': [
        # Copy llvmlite dll to root
        ('llvmlite/binding/*.dll', '', {}),
    ],
    'imgui_bundle': [
        # Copy glfw3 dll to imgui_bundle directory
        ('imgui_bundle/glfw3.dll', 'imgui_bundle', {}),
    ],
},

The .libs directory shouldn’t need to be explicitly handled if you use the built-in copy mechanism since it is normally already scanned for dependencies.

@sTorro

I assume the official way to handle 3D models is to use .bam files, correct? Does that mean the deployed executable must use .bam files? I am using .glb files during development, and I do not mind converting them to .bam, but I want to be sure.

Yes, this is recommended because .bam files are faster to load and then don’t require bundling a loader plug-in. You’ve already discovered bam_model_extensions which is the recommended way to auto-convert .glb files to .bam during build.

@sTorro

Also, what is the recommended Python version to use with Panda3D? I am currently using Python 3.13.

Any supported version. Python 3.13 is fine. Keep in mind that if you still wish to support Windows 7, however, you need to use Python 3.8, although it is now officially EOL.

[sTorro]

Thanks for the clarifications, rdb. I’ll take a look at package_data_dirs; with a bit of luck I might even be able to drop post_build.py.

I’ll need to spend more time on this issue in the future. Maybe I can eventually remove the hack around importing numba — I’m really not a fan of it, but it works for now.

Thanks for everything, best regards!