ModuleNotFoundError: No module named 'zipimport'

Scripting Issues
1

I’m trying to include google.cloud.storage in an application. The application runs fine during development, and successfully builds, but fails with:

Traceback (most recent call last):
  File "__main__", line 14, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage", line 35, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage.batch", line 31, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage._http", line 19, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "pkg_resources", line 24, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'zipimport'

Poking around here, I attempted to add it to the include_modules in setup.py, but this doesn’t solve the issue:

import setuptools
setuptools.setup(
    name="pandaapp",
    version="0.0.1",
    packages=setuptools.find_packages(),
    options={
        'build_apps': {
            'platforms': ["manylinux2014_x86_64"],
            'include_patterns': [
                "assets/shaders/*",
            ],
            'include_modules': {
                '*': [
                    "zipimport",
                ]
            },
            'plugins': [
                "p3headlessgl",
            ],
            'console_apps': {
                'run-app': "src/something.py",
                'convert-assets': "src/convert_assets.py"
            },
            'package_data_dirs': {
                'simplepbr': [
                    ("simplepbr/shaders*", "", {}),
                ],
            }
        }
    }
)

build log:

python setup.py build_apps
running build_apps
Building platforms: manylinux2014_x86_64
Gathering wheels for platform: manylinux2014_x86_64
Looking in indexes: https://pypi.org/simple, https://archive.panda3d.org/simple/opt, https://archive.panda3d.org/thirdparty, https://__token__:****@gitlab.com/api/v4/projects/${PROJECT_ID}/packages/pypi/simple
Collecting setuptools
  Using cached setuptools-58.2.0-py3-none-any.whl (946 kB)
Collecting wheel
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting numpy==1.21.2
  Using cached numpy-1.21.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.8 MB)
Collecting opencv-python-headless==4.5.3.56
  Using cached opencv_python_headless-4.5.3.56-cp38-cp38-manylinux2014_x86_64.whl (37.1 MB)
Collecting google-cloud-storage==1.42.3
  Using cached google_cloud_storage-1.42.3-py2.py3-none-any.whl (105 kB)
Collecting panda3d==1.10.10.dev1628149046
  Using cached https://gitlab.com/api/v4/projects/${PROJECT_ID}/packages/pypi/files/cf326d988c6856b66ce20211f88e7a96a54bae7d7a88bf9a39c895f6e8a8e34b/panda3d-1.10.10.dev1628149046-cp38-cp38-manylinux2014_x86_64.whl (54.5 MB)
Collecting panda3d-gltf==0.13
  Using cached panda3d_gltf-0.13-py3-none-any.whl (25 kB)
Collecting panda3d-simplepbr==0.9
  Using cached panda3d_simplepbr-0.9-py3-none-any.whl (12 kB)
Collecting gltflib==1.0.12
  Using cached gltflib-1.0.12-py3-none-any.whl (61 kB)
Collecting google-cloud-core<3.0dev,>=1.6.0
  Using cached google_cloud_core-2.1.0-py2.py3-none-any.whl (27 kB)
Collecting requests<3.0.0dev,>=2.18.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting google-auth<3.0dev,>=1.25.0
  Using cached google_auth-2.3.0-py2.py3-none-any.whl (154 kB)
Collecting protobuf
  Using cached protobuf-3.18.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting google-api-core<3.0dev,>=1.29.0
  Using cached google_api_core-2.1.0-py2.py3-none-any.whl (94 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting google-resumable-media<3.0dev,>=1.3.0
  Using cached google_resumable_media-2.0.3-py2.py3-none-any.whl (75 kB)
Collecting dataclasses-json>=0.4.5
  Using cached dataclasses_json-0.5.6-py3-none-any.whl (25 kB)
Collecting typing-inspect>=0.4.0
  Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting marshmallow<4.0.0,>=3.3.0
  Using cached marshmallow-3.13.0-py2.py3-none-any.whl (47 kB)
Collecting marshmallow-enum<2.0.0,>=1.5.1
  Using cached marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting google-crc32c<2.0dev,>=1.0
  Using cached google_crc32c-1.3.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (37 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting mypy-extensions>=0.3.0
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/numpy-1.21.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/opencv_python_headless-4.5.3.56-cp38-cp38-manylinux2014_x86_64.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_cloud_storage-1.42.3-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/panda3d-1.10.10.dev1628149046-cp38-cp38-manylinux2014_x86_64.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/panda3d_gltf-0.13-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/panda3d_simplepbr-0.9-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/gltflib-1.0.12-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/wheel-0.37.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/dataclasses_json-0.5.6-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_api_core-2.1.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_auth-2.3.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_cloud_core-2.1.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_resumable_media-2.0.3-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/protobuf-3.18.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/requests-2.26.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/six-1.16.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/cachetools-4.2.4-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/certifi-2021.10.8-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/charset_normalizer-2.0.7-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/google_crc32c-1.3.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/googleapis_common_protos-1.53.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/idna-3.3-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/marshmallow-3.13.0-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/marshmallow_enum-1.5.1-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/pyasn1_modules-0.2.8-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/rsa-4.7.2-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/typing_inspect-0.7.1-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/urllib3-1.26.7-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/mypy_extensions-0.4.3-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/pyasn1-0.4.8-py2.py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/typing_extensions-3.10.0.2-py3-none-any.whl
Saved ./build/__whl_cache__/manylinux2014_x86_64_cp38/setuptools-58.2.0-py3-none-any.whl
Successfully downloaded numpy opencv-python-headless google-cloud-storage panda3d panda3d-gltf panda3d-simplepbr gltflib wheel dataclasses-json google-api-core google-auth google-cloud-core google-resumable-media protobuf requests six cachetools certifi charset-normalizer google-crc32c googleapis-common-protos idna marshmallow marshmallow-enum pyasn1-modules rsa typing-inspect urllib3 mypy-extensions pyasn1 typing-extensions setuptools
Could not find an optimized wheel (using index https://archive.panda3d.org/simple/opt) for platform: manylinux2014_x86_64
Building runtime for platform: manylinux2014_x86_64
Unknown module: zipimport
There are some missing modules: ['Numeric', 'StringIO', '__builtin__', '_overlapped', '_scproxy', '_winapi', 'apport_python_hook', 'com.sun', 'com.sun.jna', 'com.sun.jna.platform', 'core.abs', 'core.max', 'core.min', 'core.round', 'ctypes.macholib.dyld', 'dummy.Process', 'multiprocessing.AuthenticationError', 'multiprocessing.BufferTooShort', 'multiprocessing.TimeoutError', 'multiprocessing.get_context', 'multiprocessing.get_start_method', 'multiprocessing.set_start_method', 'netbios', 'nose', 'nose.plugins', 'nose.plugins.base', 'nose.plugins.builtin', 'nose.plugins.errorclass', 'nose.util', 'numarray', 'numpy_distutils.command.build_flib', 'numpy_distutils.command.cpuinfo', 'numpy_distutils.cpuinfo', 'numpy_distutils.fcompiler', 'ordereddict', 'org.python.modules.posix.PosixModule', 'pickle5', 'pkg_resources.extern.appdirs', 'pkg_resources.extern.packaging', 'psutil', 'pytest', 'scipy', 'setuptools.extern.more_itertools', 'setuptools.extern.ordered_set', 'setuptools.extern.packaging', 'setuptools.extern.packaging.specifiers', 'setuptools.extern.packaging.tags', 'setuptools.extern.packaging.utils', 'setuptools.extern.packaging.version', 'testing.Tester', 'tkinter', 'win32com.shell', 'win32con', 'win32evtlog', 'win32evtlogutil', 'win32pdh', 'win32wnet']
Unknown module: zipimport
There are some missing modules: ['Cookie', 'OpenSSL', 'OpenSSL.SSL', 'OpenSSL.crypto', 'Queue', 'assets.FREQUENCIES', 'backports.ssl_match_hostname', 'brotli', 'chardet', 'collections.Callable', 'cookielib', 'crcmod', 'cryptography', 'cryptography.exceptions', 'cryptography.hazmat', 'cryptography.hazmat.backends.openssl', 'cryptography.hazmat.backends.openssl.x509', 'cryptography.hazmat.primitives', 'cryptography.hazmat.primitives.asymmetric', 'cryptography.hazmat.primitives.asymmetric.utils', 'cryptography.x509', 'cryptography.x509.extensions', 'google.appengine.api', 'google.protobuf._use_fast_cpp_protos', 'google.protobuf.enable_deterministic_proto_serialization', 'google.protobuf.use_pure_python', 'grpc', 'grpc._channel', 'models.Buffer', 'models.BufferView', 'models.GLTFModel', 'models.Image', 'packages.six.moves', 'packages.six.moves.http_client', 'packages.six.moves.urllib.parse', 'packages.ssl_match_hostname.CertificateError', 'packages.ssl_match_hostname.match_hostname', 'pyu2f.convenience.authenticator', 'pyu2f.errors', 'pyu2f.model', 'requests.packages.urllib3.util.ssl_', 'simplejson', 'six.moves.urllib.parse', 'socks', 'unicodedata2', 'urllib.getproxies', 'urllib.getproxies_environment', 'urllib.proxy_bypass', 'urllib.proxy_bypass_environment', 'urllib.quote', 'urllib.quote_plus', 'urllib.unquote', 'urllib.unquote_plus', 'urllib.urlencode', 'urllib2', 'urlparse', 'util.SKIPPABLE_HEADERS', 'util.SKIP_HEADER', 'utils.create_parent_dirs', 'utils.del_none', 'utils.padbytes']
warning: build_apps: could not find dependency libresolv.so.2 (referenced by libkrb5support.so.0)

Copying data files for module: simplepbr
Copying game files for platform: manylinux2014_x86_64

This is a custom build of Panda - I need features not enabled in any of the published releases, so changing this is not an option for me.
I’ve attempted to change the version of Python (3.7 complains about a missing certificate file, 3.9 fails with the same error).
I’ve attempted to change the manylinux platform (can’t the manylinux1 builder image to build, but manylinux2010 still fails - although the application needs to be built as 2014 because I’m using opencv).

Any insights would be appreciated.

2

@rdb @Moguri

I’m stymied. What should I be looking for here?

3

Unfortunately, I’m not noticing anything. The include_modules looks right and I’m not seeing anything particular for zipimport and PyInstaller. I look at PyInstaller for any module-specific hooks/logic since our deployment tools usually need similar logic. I don’t know if there are any special considerations for this module in a frozen environment since it does module imports. Sorry I cannot be more help.

4

I’m not even sure if I should need the include_modules, because zipimport is a module provided by python itself - it’s not a PyPi package.

zipimport is called out in a comment here, but I’m not sure why it’s the inclusion of google.cloud.storage that would trigger this if that’s what’s causing the issue.

5

It looks like zipimport is frozen as of Python 3.8:

>>> import zipimport
>>> zipimport.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'zipimport' has no attribute '__file__'

Maybe we need to add zipimport to startupModules in FreezeTool.py?

6

Please pick up the latest version of FreezeTool.py and drop it into your Panda installation, I think it should fix the problem:
https://raw.githubusercontent.com/panda3d/panda3d/release/1.10.x/direct/src/dist/FreezeTool.py

7

Thanks, that does fix the zipimport issue.

Doing that, the next thing that comes up is:

Traceback (most recent call last):
  File "importlib.resources", line 97, in open_binary
FileNotFoundError: [Errno 2] No such file or directory: '/workspaces/panda_synthdatagenerator/build/manylinux2014_x86_64/cacert.pem'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "__main__", line 14, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage", line 35, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage.batch", line 26, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "requests", line 133, in <module>
  File "importlib._bootstrap", line 1042, in _handle_fromlist
  File "importlib._bootstrap", line 219, in _call_with_frames_removed
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "requests.utils", line 41, in <module>
  File "certifi.core", line 37, in where
  File "contextlib", line 113, in __enter__
  File "importlib.resources", line 203, in path
  File "importlib.resources", line 111, in open_binary
FileNotFoundError: 'cacert.pem' resource not found in 'certifi'

… adding the listed file to setup.py:

import setuptools
setuptools.setup(
    name="someapp",
    version="0.0.1",
    packages=setuptools.find_packages(),
    options={
        'build_apps': {
            'platforms': ["manylinux2014_x86_64"],
            'include_patterns': [
                "assets/shaders/*",
            ],
            'plugins': [
                "p3headlessgl",
            ],
            'console_apps': {
                'someapp': "src/something.py",
                'convert-assets': "src/convert_assets.py"
            },
            'package_data_dirs': {
                'simplepbr': [
                    ("simplepbr/shaders*", "", {}),
                ],
                'certifi': [
                    ("certifi/cacert.pem", "", {})
                ]
            }
        }
    }
)

… then yields another error:

Traceback (most recent call last):
  File "__main__", line 14, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage", line 35, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage.batch", line 31, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "google.cloud.storage._http", line 19, in <module>
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 671, in _load_unlocked
  File "importlib._bootstrap", line 827, in exec_module
  File "pkg_resources", line 75, in <module>
  File "importlib._bootstrap", line 1042, in _handle_fromlist
  File "importlib._bootstrap", line 219, in _call_with_frames_removed
  File "importlib._bootstrap", line 991, in _find_and_load
  File "importlib._bootstrap", line 975, in _find_and_load_unlocked
  File "importlib._bootstrap", line 657, in _load_unlocked
  File "importlib._bootstrap", line 556, in module_from_spec
  File "pkg_resources.extern", line 52, in create_module
  File "pkg_resources.extern", line 44, in load_module
ImportError: The 'packaging' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.

Fixing this particular error requires adding packaging to the requirements file (and then adding a number of submodules to include_modules). After that, everything works.

8

Thank you, I will make these fixes in the Panda source.