Importing from the directory above

Scripting Issues
1

I’m attempting to import a module that resides in my main program directory from a python file located two subdirectories below that. To illustrate, his is (more or less) my situation:

<Main dir>
  |
  |-- File to be imported
  |
  |-- <Sub-directory 1>
                |
                |-- <Sub-directory 2>
                                 |
                                 |-- File attempting to import the above

From what I’ve gathered via some searching, I should be able to do this as follows:
(In the file attempting the importation)

from ...<File to be imported> import *

However, I’m getting the error “Attempted relative import beyond toplevel package”.

My research has turned up two options:

  1. I could also move the file to be imported, but I suspect that I’ll end up running into the problem again at a later stage, should I want to import something else.
  2. I could move everything but my “main” Python file into a sub-directory of my main directory–but that’s a fairly big change to make.

So, my question: is there another way that I can do this? If not, I suppose that I’ll fall back on the second option, but I want to check for a better idea first.

2

Do you have init.py files in those directories? They are needed for absolute and relative imports.

3

I do, I believe; indeed, I have relative importations working well in the other direction–that is, importing from a sub-directory. In my current case, from what I’ve read, I should be able to import from the directory directly above the directory in question (“Sub-directory 1” and “Sub-directory 2” in my diagram above, respectively), just not from one directory higher, because that directory is the main directory of my program. As I understand it, moving everything but my “main” Python file should fix the problem–but again, I’m hesitant to do so.

4

Here’s a hack I use for the same problem, put it before the imports in question:

if __name__ == "__main__" and __package__ is None:
    from os import sys, path
    ACTIVE_DIR = path.dirname(path.abspath(__file__))
    sys.path.append(path.dirname(ACTIVE_DIR))  # Add parent directory to import path

This gives you access to the parent directory, if you need to go up another directory just change the last line:

sys.path.append(path.dirname(path.dirname(ACTIVE_DIR)))
5

Hmm… Will that work in compiled (via packp3d, etc.) programs, and on computers other than the development machine? Additionally, since you say that it’s a hack, how reliable is it? Is it something that could be relied upon in commercial code?

6

No, that code won’t work in any other environment. Relative imports are the way to go. Make sure you have an init.py in the parent directory, and that no module name shadows a package name.

Maybe it’s even better to always import relative to the root of your source tree.

7

Ah, I’m glad that I asked. :confused:

Fair enough, and thank you.

That seems promising… I’m not managing to find a reference indicating how to go about this–would you mind elaborating, please?

8

Try this minimal sample:
rdb.name/relative_imports.zip

Check out the file engine/renderers/planet_renderer.py in particular. It demonstrates both an import from parent directory, and an import relative to the root of the package tree.
I believe relative imports are preferred from Python 3 onward; the other approach requires the module to be in sys.path, which it will be if you are running in the .p3d environment as well.

9

Ah, thank you!

Hmm… Looking at the sample, I notice that both types of importation don’t import from the directory in which the “main” file is located, but rather from a folder below that (which is what I’d been hoping to avoid, since that means rearranging my folder structure). Am I correct in believing then that there’s no reliable way of importing from the “main” file’s directory?

10

You cannot cross the package boundary with a relative import: relative imports happen within the same package (sub-packages allowed, of course). It also doesn’t make sense in most configurations to import from the directory where your entry scripts (ie. main.py) are located.

The parlance you use combined with the behaviour you seem to expect makes me suspect there might be a more fundamental misunderstanding here. In import statements, Python expects packages and modules, not directories and files. As such, “import …mymod” does not import the file mymod.py from the parent directory but instead imports module “mymod” from the parent package.

This is an important abstraction to understand, and it also explains why what you’re trying doesn’t work: in my example, there’s a package called “world” and a package called “engine”. These are two separate packages, both of which are used by the main.py script (scripts are never part of a package, by the way). The fact that they happen to both be in the same directory really doesn’t matter at this point; as long as both packages are available in sys.path. (Which they are, in this case, since sys.path conveniently includes the current directory.)

As such, when you do “import …renderer_base” inside the “engine.renderers.planet_renderer” module, Python just resolves that to the canonical module name “engine.renderer_base”. If you try to go back up one more level, Python errors because there is no higher level in the “engine” package.

Just putting init.py in the directory of main.py doesn’t work because Python simply never encounters it; it would only work if the whole thing was a giant package that you imported as a whole, in which case the package root would be one level higher. (Of course, then the placement of the main.py script inside the package would seem rather odd, since one would usually only find modules inside of packages, rather than scripts.)

If there are two independent package trees and you want to import one from the other, you should use absolute imports (ie. “import a.b” without dot prefix) and put the other package on sys.path (not usually necessary if it happens to be in the current directory). This is why I used “import world.planet” inside your engine package: I’m really telling Python to search along sys.path for the separate “world” package and import from that.

This isn’t different when we’re talking about stand-alone modules. If you defined a module “utils” by placing a file utils.py alongside main.py, this is really a separate module and is searched along the path like any other module or package. So, it should be imported using “import utils” and not “import …utils” if it isn’t part of the package you’re trying to import it into.

Of course, an easy solution is to just put everything inside a giant package and place main.py outside of that, as you indicated in your original post. This seems a common approach, be it less modular.

I guess understanding more about the nature of the module you’re trying to import is necessary for me to judge the best course of action here. Is it a completely stand-alone module you’re trying to import, that you don’t think has place in a particular package? Then you’re absolutely fine placing it inside main.py, but you should use an absolute import like “import utils”.

11

Ahh, I believe that I do understand a little better now–although I’ll want to re-read your post tomorrow (since it’s somewhat late here, and I’m rather tired). Thank you. :slight_smile:

It’s a fairly simple situation (conceptually, at least): I have my game, composed of various classes and methods spread over various files, all in the same folder, and a few separate modules in sub-folders. All of this works well.

However, I also intend to allow my levels to have an associated “script file” that provides custom behaviour–loading a level-specific shader, or handling puzzle logic, for example.

I really don’t want to mix these level-script files with the less-specific game files, instead wanting to separate them into their own sub-folder. This is fine–until I want to instantiate or inherit from a class defined in one of the main game files (if, for example, I want to spawn an object of a given class). Doing so means importing that class from the directory in which my main file is located, and hence my problem.

I realise that I’m still talking about directories; I suppose that this is at least in part because directories are what’s important to me here: I want to somewhat organise the files that go into my game, keeping the level-scripts separate from the core game-code.

12

I’d just like to report that I ended up moving my game-code (barring the “main” files for the main game and various prototypes, tests and editors, etc. that I’ve made in this project) into a module of their own, and that it seems to have solve the problem.

It was a bit of a pain: not only did it incur changes to a fair few “import” statements and file references, but I have this project under source control with Subversion, and I wasn’t familiar with the process of moving files and folders under SVN. On the other hand, it’s knowledge that will likely come in handy when I come to clean up my directory structure (such as finding better locations for my various game assets).

I am rather glad to have working level-scripts, at the least!

13

Glad you got it working.

Yes, it’s sad that neither previous-generation version control systems like SVN nor current-generation systems like Git and Hg know how to handle move operations very well.