Packp3d import error: missing modules

D_LaMa · May 7, 2018, 9:08pm

Hello everyone! I´m having some trouble trying to create a .p3d application out of my project. I run the following packp3d command:

packp3d -o TruckGame.p3d -d C:\Users\Dani\Desktop\GIT_Masterarbeit -r morepy -e xlsx -e dds

And I get a warning message:

There are some missing modules: ['matplotlib.mlab', 'matplotlib.pyplot', 'numpy', 'xlrd']

I do need and use these modules in my code, and that is the reasi´on why I can’t open the resulting .p3d file (import error). So far I have tried three alternatives:

Adding ‘-r morepy’ command (as you can see above) with nothing else, with the given warning as result.
Adding ‘-p modules_path’, with exactly the same output.
Adding ‘-r morepy -r matplotlib -r numpy -r xlrd’, where apparently these modules are not found (warning below):

:Packager(warning): Unknown package matplotlib, version “”
:Packager(warning): Unknown package numpy, version “”
:Packager(warning): Unknown package xlrd, version “”
:Packager(warning): Cannot build package TruckGame due to missing dependencies: matplotlib, numpy, xlrd

I am using Panda3D 1.10, and I already had Python on my computer (v 2.7 and 3.6) through Anaconda 3. The python version I use for this project is 3.6. I would appreciate any help on this topic.
Thank you in advance!
Dani

rdb · May 7, 2018, 9:25pm

If you are using Panda3D 1.10 and Python 3.6, then packp3d may be inadequate for your needs. I would suggest that you try out the new deploy-ng system instead.

You can install a deploy-ng-enabled version of Panda3D using pip:

pip install --pre --extra-index-url https://archive.panda3d.org/branches/deploy-ng panda3d

To see an example of how to set it up for your project, check out this version of the Asteroids sample program.

Or, consult the corresponding manual page (I am not 100% sure how up-to-date it is):
https://www.panda3d.org/manual/index.php/Using_Setuptools

It has not been extensively tested on thirdparty packages like numpy and matplotlib, but @Moguri or I will be happy to help with any issues you run into.

D_LaMa · May 8, 2018, 8:01am

Thank you for the quick answer!
I ran the pip installation command for a deploy-ng-enabled panda3D and got the following output:

Looking in indexes: https://pypi.org/simple, https://archive.panda3d.org/branches/deploy-ng
Requirement already satisfied: panda3d in c:\users\dani\anaconda3\lib\site-packages (1.10.0.dev1182)

I’m not sure if the installation is complete, when I try ‘python.exe -m pip list’, the listed Panda3D version is ‘1.10.0.dev1182’ (no mention to ‘deploy-ng’ as I saw in other topics in this forum).
I have tried to build my project (after creating the requirements and setup files) using ‘setup tools’ as follows:

C:\Users\Dani>python setup.py bdist_apps
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'bdist_apps'

It fails also to build the ‘asteroid’ project:

python C:\Panda3D-1.9.4-x64\samples\asteroids build_apps
python.exe: can't find '__main__' module in 'C:\\Panda3D-1.9.4-x64\\samples\\asteroids'

Any recommendation?
Thanks in advance,

Dani

rdb · May 8, 2018, 8:45am

You’ll have to run it from a build of Python that does not already have Panda3D installed, or use a virtualenv if you want a better way to install Python libraries without interfering with any other versions of them.

D_LaMa · May 9, 2018, 8:21am

I create a new Python environment (with Anaconda) and installed the deploy-ng-enabled Panda3D and all the other needed libraries. My project works without problems in this new environment. Now, I’ve tried to build it again, getting the following assert error:

AssertionError: Requirements.txt path does not exist: ./requirements.txt

This happens as well with the “asteroid” example. I have tried it without the ‘requirements_path’ in the ‘setup.py’ file (default) as well as with the path explicitly written, but the error is the same for both cases.
Any idea?

Thanks in advance!

Dani

rdb · May 9, 2018, 9:07am

Did you cd to the directory containing requirements.txt in the terminal before running the command? Could you show which exact command you ran and the exact output?

D_LaMa · May 9, 2018, 9:31am

Ok, thanks! it worked with the cd command! Now I have a ‘build’ subfolder in my project’s folder, but the generated ‘.exe’ file does nothing Am I supposed to open it with a particular program (like panda3d.exe with the .p3d files)?

rdb · May 9, 2018, 9:49am

Try changing gui_apps to console_apps in setup.py and see if, when running, you get any error message printed to the console.

Alternatively, you can set log_filename to something like $USER_APPDATA/MyGame/output.log and get the command-line output from there.

D_LaMa · May 9, 2018, 10:23am

I made the changes you proposed, but the .exe file still does nothing (this time a quick cmd opening and closing).
The log file results are (sorry for copying it all, I don’t know what it’s not important):

Traceback (most recent call last):
  File "main.py", line 9, in <module>
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 971, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 941, in _find_and_load_unlocked
    _call_with_frames_removed(import_, parent)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 219, in _call_with_frames_removed
    return f(*args, **kwds)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 971, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 955, in _find_and_load_unlocked
    module = _load_unlocked(spec)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 665, in _load_unlocked
    spec.loader.exec_module(module)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 807, in exec_module
    exec(code, module.__dict__)
  File "C:\Users\Dani\Desktop\GIT_Masterarbeit\build\__whl_cache__\matplotlib-2.2.2-cp36-cp36m-win32.whl\matplotlib\__init__.py", line 127, in <module>
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 1023, in _handle_fromlist
    _call_with_frames_removed(import_, from_name)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 219, in _call_with_frames_removed
    return f(*args, **kwds)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 971, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 955, in _find_and_load_unlocked
    module = _load_unlocked(spec)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 665, in _load_unlocked
    spec.loader.exec_module(module)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 807, in exec_module
    exec(code, module.__dict__)
  File "C:\Users\Dani\Desktop\GIT_Masterarbeit\build\__whl_cache__\matplotlib-2.2.2-cp36-cp36m-win32.whl\matplotlib\cbook\__init__.py", line 35, in <module>
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 971, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 955, in _find_and_load_unlocked
    module = _load_unlocked(spec)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 665, in _load_unlocked
    spec.loader.exec_module(module)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 807, in exec_module
    exec(code, module.__dict__)
  File "C:\Users\Dani\Desktop\GIT_Masterarbeit\build\__whl_cache__\numpy-1.14.3-cp36-none-win32.whl\numpy\__init__.py", line 126, in <module>
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 971, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 955, in _find_and_load_unlocked
    module = _load_unlocked(spec)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 665, in _load_unlocked
    spec.loader.exec_module(module)
  File "C:\Users\Dani\Anaconda3\envs\Test_py36\lib\importlib\_bootstrap.py", line 807, in exec_module
    exec(code, module.__dict__)
  File "C:\Users\Dani\Desktop\GIT_Masterarbeit\build\__whl_cache__\numpy-1.14.3-cp36-none-win32.whl\numpy\__config__.py", line 9, in <module>
NameError: name '__file__' is not defined

It seems it has to do with the numpy module, right?
Thanks for your help,

Dani

rdb · May 9, 2018, 11:13am

Yes, it looks like numpy expects __file__ to be set but deploy-ng does not always provide it. I’ll look into it.

D_LaMa · May 9, 2018, 2:20pm

Thanks a lot! I’ll wait for your news!

rdb · May 14, 2018, 3:39pm

Hi, sorry for the delay in getting back to you, it took me a while to find the time to fix these issues. Please update to the latest version (the version tag should be 1.10.0.dev1511+deployng.214) and try again. You will also need to delete the “build” directory before re-running deploy-ng.

You will need to mention the dependency modules such as matplotlib and numpy and xlrd in your requirements.txt file, on separate lines.

You will need to explicitly include the back-end module for matplotlib that you’re using in your code, for example, an easy way to do that is just by adding this import statement in your code:

import matplotlib.backends.backend_tkagg

Let me know if you have any trouble.

Moguri · May 14, 2018, 5:46pm

For the matplotlib backend, you can also use the include_modules option for build_apps. Something like

'include_modules': {'*': ['matplotlib.backends.backend_tkagg']}

should do the trick instead of importing.

D_LaMa · May 16, 2018, 12:25pm

Thanks a lot, you two, it worked!!
I tried what @rdb suggested, but with the modifications in setup.py that @Moguri indicated, instead of importing. Now I have a fully functional .exe file with all my project’s features.

D_LaMa · August 19, 2018, 7:59pm

Hello again @rdb,@Moguri, or whoever that could help !
I have just tried to generate an .exe file from my updated project and it seems that it doesn’t work anymore…I get the following error when I replicate the procces to the .exe file generation which worked for me 3 months ago:

THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    panda3d from http://buildbot.panda3d.org/downloads/495ce50d84b281eb01dba5d31f163dd21f4352fe/panda3d-1.10.0.dev1739+deployng.229-cp36-cp36m-win32.whl#md5=bd5ec734d902610f421e82873d56788a (from -r ./requirements.txt (line 2)):
        Expected md5 bd5ec734d902610f421e82873d56788a
             Got        bd00a3639c01674c2fbe401cbcafb2db

It looks that the new Panda3D version in the dev branch has changed and causes some conflict with pip. Any idea of what could be the reason or how could I fix it?

Thanks in advance!

PD: if it helps, the content of my requirements file is:

--pre --extra-index-url https://archive.panda3d.org/branches/deploy-ng
panda3d
numpy
matplotlib
xlrd

Moguri · August 19, 2018, 10:53pm

Not sure what’s going on here, but I’ve started a new build on the buildbot.

rdb · August 21, 2018, 8:02pm

I think that type of error occurs when the same version is built twice on the buildbot, ie. replacing another .whl with the same name. It happens because the index script caches the MD5 checksums.

I have cleared the checksum cache, although there are new deploy-ng builds anyway.

D_LaMa · August 25, 2018, 12:31pm

Thanks a lot, @rdb and @Moguri!
Now the builder works perfectly I have a functioning executable file with my project again.