Trouble making installer

Legalian · November 29, 2014, 4:57pm

Ive been having trouble making an installer with panda3d.
I tried two approaches:

packpanda --name “Sburb” --dir clientside
produces the following error:

PANDA located at /Developer/Panda3D/lib
Dir            : /Users/plawrence/dev/homestuck/clientside
Name           : Sburb
Start Menu     : Sburb
Main           : /Users/plawrence/dev/homestuck/clientside/main.py
License        : /Users/plawrence/dev/homestuck/clientside/license.txt (MISSING)
Output         : clientside

Copying the game to /Users/plawrence/dev/homestuck/packpanda-TMP...
Cannot copy game to /Users/plawrence/dev/homestuck/packpanda-TMP

packp3d -o Sburb.p3d -d clientside
produces the following error:

:downloader: [0x21c0c00] begin GET [ https://runtime.panda3d.org/contents.xml?1417277938 ]
:downloader: [0x21c0c00] Self-signed certificate from runtime.panda3d.org:443
Unable to download https://runtime.panda3d.org/contents.xml?1417277938
Unable to load Panda3D plugin.

I am trying to make a disk installer. How can I get either of these approaches to work?

Legalian · December 1, 2014, 12:19am

I found an earlier topic about this, but they didn’t get the same errors I did, and they were on windows, while I am on a mac, so I attempted to fix it myself.
I opened up packpanda.py and got rid of the “try:except:sys.exit” lines to see the exact errors and fixed a couple of them.
I just sort of typed in the exact paths specific for my game because the other ones didn’t work.
I copied the ect file with the config.prc and confauto.prc files into the homestuck directory and substituted:

shutil.copyfile("/etc/Config.prc", os.path.join(TMPETC, "Config.prc"))
shutil.copyfile("/etc/Confauto.prc", os.path.join(TMPETC, "Confauto.prc"))

with

shutil.copyfile("/Users/plawrence/dev/homestuck/clientside/etc/Config.prc", os.path.join(TMPETC, "Config.prc"))
shutil.copyfile("/Users/plawrence/dev/homestuck/clientside/etc/Confauto.prc", os.path.join(TMPETC, "Confauto.prc"))

and swapped out:

os.system("cp --recursive %s/direct          %s/usr/share/games/%s/direct" % (PANDA, TMPDIR, BASENAME))
os.system("cp --recursive %s/pandac          %s/usr/share/games/%s/pandac" % (PANDA, TMPDIR, BASENAME))
os.system("cp --recursive %s/models          %s/usr/share/games/%s/models" % (PANDA, TMPDIR, BASENAME))
os.system("cp --recursive %s/Pmw             %s/usr/share/games/%s/Pmw" % (PANDA, TMPDIR, BASENAME))
os.system("cp %s                             %s/usr/share/games/%s/LICENSE" % (LICENSE, TMPDIR, BASENAME))
os.system("cp --recursive /usr/lib/panda3d/* %s/usr/lib/games/%s/" % (TMPDIR, BASENAME))

with this:

os.system("cp -R %s/direct          %s/usr/share/games/%s/direct" % (PANDA, TMPDIR, BASENAME))
os.system("cp -R %s/pandac          %s/usr/share/games/%s/pandac" % (PANDA, TMPDIR, BASENAME))
os.system("cp -R /Developer/Panda3D/models          %s/usr/share/games/%s/models" % (TMPDIR, BASENAME))
os.system("cp -R %s/Pmw             %s/usr/share/games/%s/Pmw" % (PANDA, TMPDIR, BASENAME))
os.system("cp /Developer/Panda3D/LICENSE         %s/usr/share/games/%s/LICENSE" % (TMPDIR, BASENAME))
os.system("cp -R /usr/lib/panda3d/* %s/usr/lib/games/%s/" % (TMPDIR, BASENAME))

But I don’t know what the last line in that six calls to os.system is referring to. Which “panda3d” folder is the path trying to copy?

I am on OSX 10.8.5 and panda3d 1.8.1, and my directories is currently like this:

unimportantfolders
    homestuck
        clientside
            main.py
            otherpictures.png
        ect
            Config.prc
            Confauto.prc

When I run the command, I am in the homestuck folder and I run:
packpanda --name “Sburb” --dir clientside

rdb · December 1, 2014, 2:03am

packpanda.exe is outdated. Please do not use it.

As for the “self-signed certificate” issue with packp3d, this is a known issue with some systems not recognising the Panda3D SSL cert’s certificate authority. There is a workaround here:
viewtopic.php?f=1&t=15637&start=60

Legalian · December 1, 2014, 3:16am

Okay, I am confused, I tried the original command (packp3d -o Sburb.p3d -d clientside) and It worked this time. I used the exact same command and everything. I do however get the message,
There are some missing modules: [‘shutil’, ‘submain’, ‘urllib2’]
submain is a pyc file located in a directory called “game” that was added to the system path so that it could be imported. the “game” directory is inside the clientside directory
Would this p3d file work with the libraries, or would I have to do something else to get it to include them?

rdb · December 1, 2014, 1:23pm

To get shutil and urllib2, you should add the “-r morepy” argument to the packp3d command-line.

As for submain – I believe packp3d makes some effort to scan your import statements to pick up modules that you are using, but it cannot detect if you dynamically add things to sys.path. One thing you can do is use the “-p game” flag to packp3d to tell it that you mean to add “game” to sys.path.

Legalian · December 1, 2014, 10:26pm

Thanks, the morepy tag got the other files included into the file. The submain still isn’t recognized, though.

I am making a launcher for a game I am making. The launcher downloads new files in the form pf pyc,bam,and png files. Now I am wondering if it will work once it is compiled.

Before the program is executed, there is only one file in the ‘game’ directory, which is submain.pyc, because python makes sure that it won’t import any file that wasn’t there when the program started. submain.pyc is meant to be altered and ‘game’ is meant to be populated with other files, namely png and bam files. At no time is there a submain.py, which is why I think panda doesn’t recognize it as a module.

How do I get panda3d to recognize pyc files as a module, and furthermore, would this approach still work once the game is compressed?

rdb · December 1, 2014, 11:38pm

When you use packp3d, Panda3D runs Python in “optimized” mode (ie. the -OO flag), which means that it will be using .pyo files rather than .pyc files.

I’d suggest that you look into the Panda3D package system, though, since this does exactly what you have been describing (and more). pdeploy already automatically creates a small launcher for your game, and if you host some of your modules in downloadable packages, it will be able to download these automatically and put them inside sys.path upon launch.

Legalian · December 4, 2014, 4:11am

It looks like my old approach wasn’t going to work, as I could not edit pyo files within the multifile.
So I looked into ppackage, and made this pdef file:

import sys
packager.setHost('https://raw.githubusercontent.com/legalian/HS/master/')
sys.path.insert(0,'') 


class MyPackage(package):
	require('morepy','panda3d','numpy')
	config(version="0.0",display_name="Sburb")
	# module('core.*')
	dir('models',newDir='models')
	dir('textures',newDir='textures')
	dir('sounds',newDir='sounds')
	dir('data',newDir='data')
	mainModule('main')

Then I ran the command “ppackage -i temp dat.pdef” and got a directory called temp. I pushed that directory to github.
But then how to I get a disk installer, or reference that file in a p3d file, on a computer that doesn’t necessarily have python or panda3d installed?

rdb · December 4, 2014, 4:02pm

You can specify the source URL of a package on the packp3d command-line:

packp3d -r MyPackage,0.0,https://raw.githubusercontent.com/legalian/HS/master/ -o something.p3d

This will cause the Panda3D runtime to automatically download the latest version of MyPackage 0.0 when running your .p3d file. (You use the version to indicate the major version number, ie. incompatible releases, not to indicate minor version changes.)

If you don’t like Panda3D built-in updater (eg. if you think it provides insufficient customizability) you can instead create your own launcher in the main.py that uses PackageInstaller to download the required content:
panda3d.org/manual/index.ph … eInstaller

Legalian · December 4, 2014, 8:51pm

Okay, I did that, but now I get a new error:

:AppRunner: Total Panda3D disk space used: 146 MB
:AppRunner: Configured max usage is: 2048 MB
:HostInfo: Downloading contents file [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]

:downloader: [0x49caa00] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x49caa00] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo: Retrying (1)...
:downloader: [0x49e4400] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x49e4400] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo: Retrying (2)...
:downloader: [0x399a000] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x399a000] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo(warning): Unable to download https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo: Downloading contents file [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]

:downloader: [0x399a000] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x399a000] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo: Retrying (1)...
:downloader: [0x39cca00] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x39cca00] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo: Retrying (2)...
:downloader: [0x718ca00] begin GET [ https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265 ]
:downloader: [0x718ca00] Unable to verify identity of raw.githubusercontent.com:443, verify error code 20
:HostInfo(warning): Could not contact download server at https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
:HostInfo(warning): Status code = 14 SSL invalid server certificate
:HostInfo(warning): Unable to download https://raw.githubusercontent.com/legalian/HS/master/contents.xml?1417725265
Unknown package MyPackage, version "0.0"

rdb · December 4, 2014, 10:54pm

Invalid certificate? That’s certainly not supposed to happen - it is possible that we ship an outdated ca-cert bundle with the runtime.

Not sure what to suggest. You could try using a non-https host. I think GitHub may be HTTPS-only.

Legalian · December 5, 2014, 2:29am

Okay, I switched to my amazon file sharing service, which supports http.
So, if I have a package called MyPackage whose pdef file is:

import sys
packager.setHost('http://s3.amazonaws.com/hs-files/')
sys.path.insert(0,'') 

class MyPackage(package):
   require('morepy','panda3d','numpy')
   config(version="0.0",display_name="Sburb")
   # module('core.*')
   dir('models',newDir='models')
   dir('textures',newDir='textures')
   dir('sounds',newDir='sounds')
   mainModule('submain')

and submain.py has the entire program that I want to redistribute,
except without the app = MyApp() and app.run() lines.
Then I have a separate p3d file that I compress and make a disk installer out of that has just a main.py and that is as follows:

import submain
app = submain.MyApp()
app.run()

But when I run it, it reports:

There are some missing modules: ['submain']

What is the correct way to make an auto updator?

rdb · December 5, 2014, 7:02pm

Can you show the command you specified to packp3d?

Legalian · December 5, 2014, 8:53pm

ppackage -i tempfour dat.pdef
to build the package and
packp3d -o Sburb.p3d -r MyPackage,0.0,http://s3.amazonaws.com/hs-files/ -r morepy -r numpy
to build the p3d file.

Legalian · December 7, 2014, 1:08am

Okay, I figured that out. I needed to set the files to ‘public’ and put /contents/ at then end of the path.
I can now make a .p3d file without complaints, but when I run it I get:


:AppRunner: Total Panda3D disk space used: 156 MB
:AppRunner: Configured max usage is: 2048 MB
Traceback (most recent call last):
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 424, in __taskChainDispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 482, in __dispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 748, in __startIfReady
  File "VFSImporter", line 153, in load_module
  File "main.py", line 5, in <module>
  File "VFSImporter", line 446, in load_module
ImportError
:task(error): Exception occurred in PythonTask Messenger-default
Traceback (most recent call last):
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 638, in run
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/task/Task.py", line 502, in run
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/task/Task.py", line 460, in step
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 424, in __taskChainDispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 482, in __dispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 748, in __startIfReady
  File "VFSImporter", line 153, in load_module
  File "main.py", line 5, in <module>
  File "VFSImporter", line 446, in load_module
ImportError
Successfully joined thread: 0
Failure on startup.

Which is especially weird, because main.py is only 3 lines.
Mabye it is referring to submain, whose fifth line is ‘import numpy’.
I determined that main.py, the file that is not in the package, is not looked at before the error, because I put
[][0]
at the beginning of main.py before all of the import statements, and I get the same error, where I should get an index error.

Legalian · December 7, 2014, 2:39am

Okay, I think I almost have it figured out.
I switched to hosting on localhost.
I wrapped the ‘app = submain.MyApp()’ and ‘app.run()’ lines in a function called main.
This changed the error message to failing on submain, line 5 instead of main, line 5.
I was right, importing numpy kills the program, and I did multiple tests to confirm this.
I don’t know why; my pdef file specifies that numpy is required, as well as the command that I used.

Also, It seems that even if I change something in submain and give the server the new information, and even restart the server, the client keeps the latest downloaded copy of MyPackage, instead of re-downloading the latest version. The only way that I have found to force it to re-download the package is to change the urls in my pdef file, the command that I am using to generate the p3d file, and the server that I am hosting it on.
Right now I am just moving up ports on this localhost server that I set up. The version is always ‘0.0’.

How can I get numpy imported, and get ppackage to download the latest version of the packages?

Legalian · December 7, 2014, 10:15pm

I fixed problem with importing numpy!
I found this: bugs.launchpad.net/panda3d/+bug/1148559
I realized that I wasn’t using anything from morepy, but I included it just because I had included it when I tried to make the installer the other way.
I took the morepy lines out, and It works, provided the port changes in the pdef file, the command to make the p3d file, and the server itself. Now I just need to convince panda to download the newest versions without changing the url.
It seems that when I change the url or version panda thinks of it as a new package, and this forces it to download the newest version instead of using the already downloaded version.

rdb · December 8, 2014, 7:16pm

It depends on how you are updating your tree. If you simply copy-paste your new directory on top of your old one, it won’t work. You should either call ppackage with -i pointed at your old directory, or you should do it to a separate directory, and then use pmerge.p3d to merge the two trees together.

You can use ppatcher.p3d after every time you updated a directory to let Panda3D generate patches so that the user won’t have to redownload everything after an update.

Also, if you are using pdeploy, you can control how aggressively it checks for and installs updates by using “-t verify_contents=somevalue”, where somevalue can be one of “never”, “none”, “normal”, or “force”. This explains the difference between the various options:

   If verify_contents is P3D_VC_none, then the server will not be
   contacted unless the current contents.xml cannot be read at all.
   If it is P3D_VC_normal, the server will be contacted whenever the
   contents.xml has expired.  If it is P3D_VC_force, each server will
   be contacted initially in all cases, and subseqeuntly only whenever
   contents.xml has expired for that server.  The opposite of
   P3D_VC_force is P3D_VC_never, which forces the plugin never to
   contact the server and not to verify the contents at all.  This
   option should only be used if the host directory is prepopulated.
   Normally, a web plugin should set this to P3D_VC_normal.

Legalian · December 10, 2014, 2:46am

huh, now whenever I try to use my online file hosting services I get new errors.
The same command with amazon that worked before now raises:

:AppRunner: Total Panda3D disk space used: 156 MB
:AppRunner: Configured max usage is: 2048 MB
:AppRunner: Freshening http://s3.amazonaws.com/hs-files/contents/MyPackage/0.0/MyPackage.0.0.import.xml
:downloader: [0x70b1e00] begin GET [ http://s3.amazonaws.com/hs-files/contents/MyPackage/0.0/MyPackage.0.0.import.xml ]
:AppRunner: http://s3.amazonaws.com/hs-files/contents/MyPackage/0.0/MyPackage.0.0.import.xml is still no good after downloading.
Traceback (most recent call last):
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 424, in __taskChainDispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 482, in __dispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 748, in __startIfReady
  File "VFSImporter", line 153, in load_module
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/packp3d.py", line 225, in <module>
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/packp3d.py", line 208, in makePackedApp
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 3045, in do_require
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 3071, in requirePackagesNamed
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 2789, in findPackage
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 2894, in __findPackageOnHost
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/directnotify/Notifier.py", line 132, in error
StandardError: Couldn't download import file.
:task(error): Exception occurred in PythonTask Messenger-default
Traceback (most recent call last):
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 638, in run
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/task/Task.py", line 502, in run
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/task/Task.py", line 460, in step
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 424, in __taskChainDispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/showbase/Messenger.py", line 482, in __dispatch
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/AppRunner.py", line 748, in __startIfReady
  File "VFSImporter", line 153, in load_module
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/packp3d.py", line 225, in <module>
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/packp3d.py", line 208, in makePackedApp
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 3045, in do_require
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 3071, in requirePackagesNamed
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 2789, in findPackage
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/p3d/Packager.py", line 2894, in __findPackageOnHost
  File "/Users/buildbot/slave/release_rtdist_osx/build/panda3d/built_cmu/direct/directnotify/Notifier.py", line 132, in error
StandardError: Couldn't download import file.
Successfully joined thread: 0
Failure on startup.

And I tried to use divshot, in case it was just amazon being picky, but I get:

:AppRunner: Total Panda3D disk space used: 156 MB
:AppRunner: Configured max usage is: 2048 MB
:HostInfo: Downloading contents file [ http://production.homestuck-files.divshot.io/contents.xml?1418090442 ]

:downloader: [0x4c4c000] begin GET [ http://production.homestuck-files.divshot.io/contents.xml?1418090442 ]
:HostInfo(warning): Successfully downloaded http://production.homestuck-files.divshot.io/contents.xml?1418090442
:HostInfo(warning): Failure reading http://production.homestuck-files.divshot.io/contents.xml?1418090442
:HostInfo: Downloading contents file [ http://production.homestuck-files.divshot.io/contents.xml?1418090442 ]

:downloader: [0x4c4c000] begin GET [ http://production.homestuck-files.divshot.io/contents.xml?1418090442 ]
:HostInfo(warning): Successfully downloaded http://production.homestuck-files.divshot.io/contents.xml?1418090442
:HostInfo(warning): Failure reading http://production.homestuck-files.divshot.io/contents.xml?1418090442
Unknown package MyPackage, version "0.0"

Also, (using localhost) when I call ppackage with -i at my old directory, I noticed that the “seq=” changes, but it still doesn’t download the latest version when it is run.

rdb · December 10, 2014, 11:46am

Hmm, it doesn’t like your .xml files, presumably because the hash isn’t consistent. I’ve had similar issues before, in which cases it turned out to be a line ending issue (CRLF converted to LF or vice-versa).

Oddly enough, that doesn’t seem to be the issue for your amazon files. I downloaded the .xml files and the md5 checksums seem perfectly consistent with your contents.xml file.

As for your other host - when I download your .xml file with wget on the command-line, I get garbled data. Odd. I would guess that that’s some sort of issue with configuration of your web host.

Could you (in either case) manually check the location that Panda3D downloads the .xml file in and see if it is any good, and take its checksum and checking line endings?

In fact, maybe you could try deleting your host directory entirely in ~/Library/Caches/. I’m wondering if there are some files left over from a previous iteration that are getting in the way.

As for your package not updating: is this just with pdeploy, or also when you run your .p3d directly with the Panda3D plug-in? If the former, did you try with the -t verify_contents=force flag? Does it work when you run your p3d file with “panda3d.exe -f” ?