External Relative Importation in Distributable

Scripting Issues
1

For a project that I’m working on, I want in my distributable version to have certain Python scripts be bundled not as part of the executable, but rather as external resources. (Such that one can potentially drop new scripts into position after installation on a user’s machine–e.g. for DLC.)

Now, if I simply have a flat file-hierarchy, with all scripts in the same directory, this seems to work as expected.

However, things get complicated when I want to keep my “external scripts” in a sub-directory: the executable seems to no longer find them, even when they’re present.

By way of demonstration I’ve attached below a zip-file containing two versions of the same mini-program–one in a directory named “Raw” and one in a directory named “Frozen”. The former has the program simply as a collection of Python files, while the latter has the same, but with a “setup.py” and “requirements.txt” with which to build a distributable.
ImportTester.zip (4.0 KB)

(I tried just providing pre-built distributables of the “Frozen” version, but hit the maximum file-size for uploads to the forum.)

To test:

  • Open the “Raw” folder and run the following (or equivalent for your system):
    python3 core.py
    • Note the output produced
  • Open the “Frozen” folder
  • Within that, open the “setup.py” file and modify the “platforms” section to suit your OS
  • Build a distributable by running the following (or, again, the equivalent for your system):
    python3 setup.py bdist_apps
  • Open the new “dist” folder and extract the zip file within to a convenient location
  • Go to that location and open the extracted folder
  • Navigate to the executable (which should be named “Import Tester”) and run it in a console/terminal
    • Note the output produced

The program is simple: it imports two sub-scripts, one via a standard “import”-statement, that one being bundled with the executable, and another dynamically via “importlib”, that other being kept external to the executable.

It should then print out the following, one line coming from each script:

I am a Kitten
I am a Moggy

On my machine, at least, the “Raw” version works as expected, producing the above output.

However, the “Frozen” version doesn’t: it produces the following output:

I am a Kitten
Traceback (most recent call last):
  File "__main__", line 9, in <module>
  File "importlib", line 126, in import_module
  File "importlib._bootstrap", line 994, in _gcd_import
  File "importlib._bootstrap", line 971, in _find_and_load
  File "importlib._bootstrap", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'SubClasses.secondary_dynamic'

Note, however, that the relevant script-file is included with the “Frozen” version, along with an “__init__.py” file.

So, does anyone know where I’m going wrong, and/or how to fix this…?

2

Isn’t this problem solved by the sys.path module?

import sys
sys.path.append("Frozen")
sys.path.append("Raw")
3

Hmm… Perhaps I’m not using it correctly, but it doesn’t seem to help.

Specifically, I’ve tried both this:
sys.path.append("SubClasses")
and this:
sys.path.append("~/Desktop/mew/Import Tester-1.0.0_manylinux1_x86_64/Import Tester/SubClasses/")

In both cases I seem to still get the same error.

4

It would be nice to find out what the current working directory is and try to set a library search relative to it.

5

According to both os.getcwd() and pathlib.Path.cwd() it’s as follows:

/home/thaumaturge/Desktop/mew/Import Tester-1.0.0_manylinux1_x86_64/Import Tester

And even putting both that (plus a “/” at the end) and that appended with “/SubClasses/” in two consecutive calls to sys.path.append produced no significant effect on the error. :/

6

It’s because SubClasses itself is frozen. A frozen module cannot have child modules that aren’t frozen, because when importing a child module, Python looks at the __path__ attribute of the parent module, which is empty for frozen modules (since a frozen module doesn’t have a directory).

It works if you add something like this to SubClasses/__init__.py to tell Python where to find the child modules:

import os
__path__ = [os.path.join(os.path.dirname(__file__), 'SubClasses')]

Alternatively, keep the entire SubClasses folder unfrozen.

7

Aha! That did indeed do it! :smiley:

Specifically, I did two things:

  • Removed the “secondary_static.py”, as well as its reference in code
    But more importantly:
  • Added the following to the options specified in my “setup.py”:
"exclude_modules" : {
                TITLE : "SubClasses"
            },

I’ll have to see at some point whether this fixes the issue in my main–and rather more complex–project, but it looks good!

Thank you very much! :slight_smile:

1 Like