Packaged installer size

I’m a member of another small game development community, where it’s not uncommon - and in fact, I believe, encouraged - for us to upload prototypes and pre-alpha builds for comment, a practice that, I feel, can provide valuable feedback, suggestions and encouragement.

In the hopes of showing my progress on my current Panda project there, I tonight referred to the manual’s section on using Packpanda, packed up the game, and tested it on another machine. All seemed reasonably well, until I noticed a slight delay in a transfer of the file, and subsequently discovered that this simple game’s installer weighed in at a little over 36MB - and this for a game whose own files come to a little over 1.5MB (with eggs converted to bams)!

Unfortunately, this is a little problematic, for three reasons:

1. I’m not sure of how many people in the community that I mentioned have an ADSL connection; while 36MB transferring over a dial-up connection isn’t the end of the world, it does seem likely to me to discourage people.
2. Of those of us who do have ADSL connections, most of us are probably under caps (I am, for example). While 30-odd MB might not be terribly much, these things can add up, I feel.
3. The community has file hosting available, but with a general limit of 10MB per upload, as I recall. I’ve gotten around this once by splitting an upload into two zips, but this doesn’t seem viable here… I could perhaps try the Panda hosting, but it doesn’t seem worth it for this.

Before now (developing in C++, and using OpenGL) it wasn’t generally at all difficult to keep my uploads below 10MB, and I don’t want to go above that for small games, such as my current one.

Bear in mind that such uploads are something that I’d want to be able to do a couple of times a month, perhaps more. I don’t particularly want to do that with a 36MB upload several times a month.

[edit] Similarly, 35-odd MB seems like a bit much for those occasions on which I’m only aiming to put together a quick prototype. [/edit]

Some searching of this forum showed me that such sizes are probably normal. This one game the promising figure of 7.79MB (albeit in a post from more than a year ago; the size would probably be a little bit greater now, I imagine)… but it looks a little “fiddly” for something that I might end up doing with an hour to go before one of the afore-mentioned community’s one-month competitions. ^^;

In another thread, a poster suggested to the person bringing up this issue that they use C++, but the manual indicates that not all of Panda’s functionality is available in C++.

(And admittedly, Python is proving to be a nice language.)

So… First of all, my apologies for perhaps rambling a little. This, along with a few more minor issues, has led to a little frustration on my end, I think. ^^;

Secondly, what suggestions have you?

I’ve attempted to follow those instructions before for a three day game competition. I managed to mess it up despite having built a package to make building the smaller game easier and writing the instructions down in an easier format. I chalk it down to exhaustion though, the instructions worked well enough when I did it without time pressures.

I suspect it wouldn’t be too hard to write a python script to complete those steps without human intervention, it would at least remove operator error as a factor when you want to package your panda game.

Besides Py2Exe, there is also PyInstaller for packaging your game. It has the advantage of being cross platform but I’ve never been able to make it work. I believe pro-rsoft has experience working with it though.

Thank you for that…

Do you (or does anyone else) know what sort of size I might get from using PyInstaller?

The Python script might also be an idea.

Thank you again.

Any further suggestions or comments? I’d really like to find a good way of producing nice, small packages, for small and simple projects, at least…

There are three options, from my point of view:

• Use PyInstaller, it indeed makes much smaller binaries.
• Strip down the Panda3D build to what is essential for your game.
• You also might want to try running your packpanda’ed game through UPX. Dunno if packpanda already does this itself, though.

My game was 82 MB using packpanda, while it was only 32 MB using pyinstaller. Running through UPX and stripping the Panda3D build made it even smaller, though.

You make good points, pro-rsoft - thank you for them. I think that I’ll give the first two a shot to start with, and perhaps see what results UPX gives if I’m still unhappy with the results.

That’s not bad! I wonder how much of that Panda itself accounted for.

Hmm… do you by any chance have any advice for getting it to pick up the Panda modules? I keep getting errors that read “no module named <insert Panda module here, such as direct>” in my warning text file, and of course the resultant exe doesn’t run.

I’ve tried a variety of things, including your advice in this thread and adding various directory paths pointing to the direct or direct/src subfolders to the spec file, to little avail thus far.

I’m using the 1.3 version of PyInstaller (I tried to get the latest, but encountered errors in attempting to run the config program; it’s possible that I didn’t download it correctly, or mistakenly downloaded a highly unstable version, since I don’t think that I’ve used SVN before).

I’ve managed to get it to produce a working executable of a simple, one-file non-Panda program, at least.

Any suggestions?

Are you talking about Panda3D’s c++ classes or python classes?

Well, to be honest I’m not sure.

This is a sample from the beginning of one set of warnings:

W: no module named direct (top-level import by __main__)
W: no module named libpandaskel (top-level import by pandac.libpandaskelModules)
W: no module named direct (delayed import by pandac.libp3directModules)
W: no module named libp3direct (top-level import by pandac.libp3directModules)
W: no module named libdirect (top-level import by pandac.libp3directModules)
W: no module named optik.__version__ (top-level import by optparse)
W: no module named direct (delayed import by pandac.libpandaModules)
W: no module named libpandaexpress (top-level import by pandac.libpandaexpressModules)

Hmm… Looking at that, it looks a little like it’s finding the Python files (since it’s in a position to note anything about pandac.libp3directModules, for example) - perhaps it is the C++ classes? Again, I’m afraid that I’m not sure - sorry. ^^;

(I can copy to a text host a full output of such a file, if it will help.)

Another method would be to just distribute your source files, and have them download the panda engine to run it in. Provided you include instructions on where to put the files and all that I think this is a better option.

I know packpanda makes it easy to install and run, but since this is for only testing, why not just get them to download panda from here?

Robert

Well, it’s an idea, but I’m not sure that many people would go for downloading a nearly 60-MB file just to be able to try out my games. ^^;

[edit] Not to mention the pain of getting everything to connect - I don’t know how many of them might have used Python before, let alone set up the appropriate paths elements. [/edit]

Thank you, however - I do appreciate the suggestion.

Based off of the py2exe thread and some questions asked of pro-rsoft, I came up with this .spec file that seems to work with everything I throw at it…
this is for a single-file exe, but it shouldn’t be too hard to adapt it for a directory build. This means it may take a little for the program to unpack itself and then run…really not ideal. I think that a directory build would work better…but I was using this for a specific purpose that I needed one file.

You will have to change the directory to your files/panda’s files/pyInstaller’s files, but it should just work…

#****************************************PANDA3D******************************************************
PANDA_DIR = 'C:\\Panda3d'
import sys, os, glob
# some C/C++ runtime libraries, machine without these .dlls will crash Panda3D
runtimeLibraries=glob.glob(os.path.join(PANDA_DIR, 'bin/*.manifest'))+\
                 glob.glob(os.path.join(PANDA_DIR, 'bin/vcredist*.exe'))
P3DruntimeLibraries=[]
for r in runtimeLibraries:
	P3DruntimeLibraries.append( (r[r.rfind('\\')+1:],r,'DATA') )

P3DmissingDLLs = []
for d in os.listdir(os.path.join(PANDA_DIR, 'bin/')):
	if d[-4:] != '.dll': continue
	P3DmissingDLLs.append( (d,os.path.join(PANDA_DIR,'bin/%s' %d),'BINARY') )

P3Detc = Tree(os.path.join(PANDA_DIR, 'etc/'),prefix='etc/')
#*******************************************************************************************************

myModels = Tree('C:\\games\\testPyInstaller\\models',prefix='models/')
myLibs = Tree('C:\\games\\tesPyInstaller\\lib',prefix='lib/')

a = Analysis([os.path.join(HOMEPATH,'support\\_mountzlib.py'),
				os.path.join(HOMEPATH,'support\\useUnicode.py'), 
				'C:\\games\\testPyInstaller\\test.py'],
             pathex=['C:\\pandaSupport\\pyinstaller_1.3'])
pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          a.binaries+P3DmissingDLLs,
		  PKG(P3DruntimeLibraries+P3Detc+myModels+myLibs,name='test.pkg'),
          name='testPyInstaller.exe',
          debug=False,
          strip=False,
          upx=True,
          console=False )

Note that myModels and myLibs simply contained egg files and python files respectively, you could add more/take them away if not needed.

the code at the top of test.py:

import sys, carchive, os
this = carchive.CArchive(sys.executable)
all = this.openEmbedded('tunnel.pkg')
targetdir = os.environ['_MEIPASS2']
for fnm in all.contents():
    stuff = all.extract(fnm)[1]
    outnm = os.path.join(targetdir, fnm)
    dirnm = os.path.dirname(outnm)
    if not os.path.exists(dirnm):
        os.makedirs(dirnm)
    open(outnm, 'wb').write(stuff)
all = None

hope this helps!

(It includes ALL of panda’s runtime files (dlls,direct…), some of them you probably won’t need, but that is easily fixed too)

Hope this helped!

Ooh, that looks interesting indeed! Thank you very much - I intend to try it out and report back!

Alas, no luck.

I seem to still be getting the “no module named direct.interval” message and its kin in my warning file.

Additionally (and these I think that I should probably have mentioned earlier, it now seems to me - I’m sorry that I didn’t ^^; ), I have two more symptoms to report:

1. When I attempt to run the resulting executable from cmd (having enabled console output with “console=True”), I get the error “name ‘exit’ is not defined”. This could, of course, simply be due to the process not being successful, in particular in finding Panda.

2. When I run Build.py, I get a warning in the console output, which reads:
   “W: Cannot determine your Windows or System directories
   W: Please add them to your PATH if .dlls are not found
   W: or install starship.python.net/skippy/win32/Downloads.html”

My PATH environment variable includes (amongst other things) %SystemRoot%\system32, %SystemRoot% and %SystemRoot%\System32\Wbem

Note that I’ve gone back to leaving my Panda direct directory with the src sub-folder as installed, for now at least.

Sorry, I forgot to mention that the script I posted requires the direct\src folder to be copied to direct and all liines in the direct_init_.py file to be commented out…Again, to fix this you might hazard a look into the py2exe thread…

The not finding direct.interval is this because it is really at direct.src.interval.

The exit is inside of direct_init_.py or a related file that is incorrect and reads else: exit() when in fact it should read else: sys.exit()…It is a “bug” that has persisted and is only found in the situation when the direct tree cannot be found. I believe that init.py is dynamically created, so it just hasn’t gotten fixed as it is a very low priority to those in charge. It is harmless and would print the error anyways, the exit just obscures it.

Aah, of course. I now feel silly for not trying that (I even thought to do so, but put it aside for some reason >_<), and for not re-instating the init.py file in direct, as Pro-rsoft suggested later in that thread.

I suspect that lack of sleep had something to do with these slips. ^^;

And at last, I seem to have managed to produce a working executable! ^___^

Thank you very much to all of you who have helped me in this - I am very grateful!

I have some experimentation to do now, as there are still a few issues (such as the resulting executable being 143MB big o_o), so I might well be back with more questions, I’m afraid. Nevertheless, I believe that progress has been made.

All right, I seem to have the basics working. I’ve altered the spec file to better match my preferences, including changing the output over to one-directory rather than one-file.

My task now is to decrease the distributable size (naturally this is likely to at least be helped by archiving the final product, but I want to shrink it before that as well). Looking at the directory created, it appears that most of the size is a result of the various dlls present. It also appears that at least some of them can be excluded (for example, I doubt that my program has much use for Maya loaders).

However, while some appear to be obvious, others may not be, leaving me with my next question: How do I go about determining which dlls are actually used by my program, so that I can have the spec file exclude them?

I could simply delete experimentally, but I’d rather find a system that’s quicker to repeat.

PyInstaller includes bindepend.py, which looks useful. When run on my executable, however, it only reports COMCTL32.dll, KERNEL32.dll and WS2_32.dll. Running it on the copy of libpanda.dll in my distribution folder produces a longer list, but I’m not sure that the list includes everything that I should keep - it may be that I should run the program on each of those in turn, and each of their outputs, and so on, or should run it again for some other file as well. It may also be that there are files in that list that I have little use for in this case.

Searches of the forum have brought up mention of a “dependency viewer” - does anyone know anything about this? If it’s a Panda tool, it doesn’t appear to be mentioned in the manual, and I don’t see it in the bin folder. If it’s an external tool, does anyone know anything about it?

Are there any options that I’ve missed, and if so, what are they, please?

[edit] On taking another look at the section on mf.py in the PyInstaller manual, I think that I should perhaps take another look at that. I am still very much interested in any other advice that may be given, however!