packpanda.py and packp3d.p3d still missing

General Discussion
41

I don’t know how to express my (our) gratitude… Do you like a statue representing you nearby the Colosseum?

Not exactly… You have a package, which is a multifile… But, at the end, even a p3d file is a multifile, so they’re similar.
Why don’t you want the files inside a multifile? The fastest way to load many files (which is the typical scenario in a game) is from a compressed file, and not from filesystem. And consider that, with Panda, that happens in a transparent way, while with other engines you’ve to specify the stuff to manage the compressed files inside your code. I think that using packages is exactly what you want for an efficient game. :slight_smile:

42

Ive explained that already, you probably missed it.

43

I missed the entire second page of posts. :blush:
Now I see your reasons. In the meantime, I’d insert your initial assets in a package, and (in the code) I’d copy them from the virtual file system to the real file system (you could check if the destination directory already exists and copy them only the first time). It’s a bad solution because you’ve two copies of the initial files, but this could be a temporary workaround.

44

Well, this tool is something which is used when you already want to distribute your application, so i wouldnt call this a ‘temporary workaround’, because its not a warkaround used during the development process. Does this make sense?
Having a copy of the files would contradict the whole point of using pdeploy instead of packpanda. The folder size wouldn’t end up smaller (if not bigger) and thats the whole point of me chhosing pdeploy over packpanda, now that I know I can use packpanda with 1.7.1 by manually downloading it.

45

Ok, now I see the difference. I adopt early testing in my projects, so I’d like to test the mod-features during the development process. Ok, that is a useless workaround for your development methodology, if you plan tests at the end of the project.

46

Guess not, thats not what i meant, but forget it. offtopic.

47

:angry:

Why doesnt this system work for me, like at all?

Okay, the standalone doesnt work quite right yet, fine. Files must be in a multifile, fine.
I can’t even get the installer mode to work.

Here, I made this main.py file:

from pandac.PandaModules import *
import direct.directbase.DirectStart

run()

this is what i get:

panda3d.exe c:\panda3D-1.7.1\bin\pdeploy.p3d -N "test" -v 0.0.1 c:\Users\User\Desktop\test.p3d installer

:Installer(warning): Makensis utility not found, no Windows installer will be bu
ilt!
:Installer: Creating .//linux_i386/test_0.0.1_i386.deb...
:PackageInfo: p3dembed downloading http://runtime.panda3d.org/p3dembed/linux_i38
6/p3dembed.linux_i386.xml
:Standalone: Creating /c/Users/User/AppData/Local/Temp/test_deb_9b3fc2/usr/bin/tes
t...
:Installer: Creating .//osx_i386/test.app...
:PackageInfo: p3dembed downloading http://runtime.panda3d.org/p3dembed/osx_i386/
p3dembed.osx_i386.xml
:Standalone: Creating .//osx_i386/test.app/Contents/MacOS/test...
:Installer: Creating .//osx_i386/test 0.0.1.pkg...
Traceback (most recent call last):
  File "C:\panda3d-1.7.0\built_cmu\direct\showbase\Messenger.py", line 352, in _
_taskChainDispatch
  File "C:\panda3d-1.7.0\built_cmu\direct\showbase\Messenger.py", line 410, in _
_dispatch
  File "C:\panda3d-1.7.0\built_cmu\direct\p3d\AppRunner.py", line 493, in __star
tIfReady
  File "VFSImporter", line 153, in load_module
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/pdeploy.py", line 2
46, in <module>
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 242, in buildAll
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 260, in build
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 528, in buildPKG
  File "C:\Python26\Lib\zipfile.py", line 1022, in write
    zinfo.header_offset = self.fp.tell()    # Start of header bytes
  File "C:\panda3d-1.7.0\built_cmu\direct\stdpy\file.py", line 218, in tell
ValueError
:task(error): Exception occurred in PythonTask Messenger-default
Traceback (most recent call last):
  File "C:\panda3d-1.7.0\built_cmu\direct\p3d\AppRunner.py", line 411, in run
  File "C:\panda3d-1.7.0\built_cmu\direct\task\Task.py", line 496, in run
  File "C:\panda3d-1.7.0\built_cmu\direct\task\Task.py", line 454, in step
  File "C:\panda3d-1.7.0\built_cmu\direct\showbase\Messenger.py", line 352, in _
_taskChainDispatch
  File "C:\panda3d-1.7.0\built_cmu\direct\showbase\Messenger.py", line 410, in _
_dispatch
  File "C:\panda3d-1.7.0\built_cmu\direct\p3d\AppRunner.py", line 493, in __star
tIfReady
  File "VFSImporter", line 153, in load_module
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/pdeploy.py", line 2
46, in <module>
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 242, in buildAll
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 260, in build
  File "/root/pandaworker/panda3d-1.7.0/built_cmu/direct/p3d/DeploymentTools.py"
, line 528, in buildPKG
  File "C:\Python26\Lib\zipfile.py", line 1022, in write
    zinfo.header_offset = self.fp.tell()    # Start of header bytes
  File "C:\panda3d-1.7.0\built_cmu\direct\stdpy\file.py", line 218, in tell
ValueError
Failure on startup.

I know you guys worked hard on making this, but its so hard to get running.

I still dont understand the point in making a p3d file first if you never intend to make a web application.

48

I notice you’re running Windows. I’ve always used it from other platforms. I guess I should set up a Windows machine somewhere and do some testing before 1.7.1 comes around.

A .p3d package has in concept nothing to do with web applications. A .p3d package simply an archive that holds your game and its assets, to distribute to others so that they can run it without installing the heavy Panda3D SDK, and so that it can benefit from highly optimised and fast Panda3D builds.

In fact, a .p3d package is nothing more than a multifile with a special p3d_info.xml file in it that gives some special information about how the game in it should be run.

The advantage of this system is that you can use packp3d (or ppackage) to gather and compile the assets and scripts of your game, and then distribute that in any way you want.

Normally, you’d just distribute the .p3d file and people who want to play it can just double-click it in their file manager and they can play the game.

The problem of .p3d packages is that people still need to install the Panda3D runtime. A few developers prefer to ship a single installer that doesn’t require people to install the Panda3D runtime. So pdeploy packs the .p3d file and the runtime into a single executable, and optionally can generate an installer and make the game able to run without requiring an internet connection.

Of course, you can still use packpanda if you want, but the tool will always generate suboptimal results compared to packp3d/pdeploy (for instance, games produced with packpanda are significantly slower and take up significantly more disk space than with packp3d/pdeploy).

tl;dr: packp3d simply but intelligently packs the game and the assets that it uses into a single archive (with the .p3d extension), which you can distribute to other Panda3D users to run. PDeploy can pack these assets with the Panda3D runtime, and can generate a graphical installer to install those into the system.

49

Okay. I understand the purpose of p3d files.
A p3d requires internet connection the first time its run, so this gives a problem if you dont have (permanent) access to the net. Also, people might simply not want to make their users go through extra steps before they can play the game and get them annoyed. Im in this group, so I decided to use the installer mode. It seems like the installer mode can make an installer which will still require an internet connection the first time. Personally, ive never encountered something like this before.
Anyway, the installer mode can also generate a standalone installer, which seems like the one im after. But im just wondering, if i was going to use the -standalone -installer mode, why would I need to pack the files into an .p3d (use another tool) in the first place? Sounds like an additional unnecessary step. But im okay with that.
And I dont understand why Im forced to have my assets in a multifile currently and have pyc and bam files. Packpanda had that simply as optional.
But like we know Packpanda includes the complete panda runtime with all the third party tools even if your game never uses them, and the installation folder ended up around 300 mb, so if the new tools provided all its functionality and allowed you to choose what libraries to include or not, then it would work for me…

Also, ive heard you can also generate a standalone exe (not an istaller), but it if I understand correctly, everything ends up in one huge exe. It would be nice to be allowed to choose wether you want to merge to files in the exe or have them in folders. I encounter the latter more often in practice than the 1st one, and those are usually very small games.

50

I have to explain the three modes of pdeploy first.

  • There’s the standalone mode. It generates an executable that embeds the .p3d file, which will connect to the internet to download the panda3d libraries (just like the web plugin).
    Try to think of those executables as a combination between panda3d.exe and the .p3d file - it contains basically the .p3d file and what’s needed to run it.

  • Then there’s the installer mode. This invokes the standalone mode, and then builds a graphical installer around it.

  • Thirdly, there’s the installer mode but with the ‘-s’ flag added, which stands for self-contained (not for standalone). Basically, what it does (on top of the other things) is it contacts the hosts that the .p3d package references (like runtime.panda3d.org), downloads the packages it depends on (like ‘panda3d’ and ‘fmod’ and ‘wx’) and also packs these into the installer, and configures the standalone binary to read the packages from a local directory instead of downloading them from the web.
    This it the only mode that will produce binaries that will not contact the internet.
    (That said, at the moment, these binaries will still contact the internet because of bugs in SDK 1.7.0, runtime 1.0.0 and rtdist rev1. The system has been overhauled and polished after that point, so the next revision of these three release cycles should bring a much better version of pdeploy soon.)

It may seem so, but it does significantly reduce the complexity in how the systems work.

That’s because the .p3d is supposed to be a compiled version of your game, similar to what .jar is to Java or .swf to Flash. Having them as compiled hides the source code, and significantly speeds up load time.
If people don’t want this, they want to be distributing their source code instead.

Because the runtime packages of Panda3D are so optimised, they are not meant for development. For instance, because a lot of error checking is compiled out, it will simply crash if you pass a wrong type of parameter to a function. That’s why you’re supposed to compile your game to .p3d once, and not modify the source code when running it in the runtime environment.

51

I think the problem is we (the users) are now dependant on the p3d system too much. Actually it seems the only way of distributing your game for 1.7.1.
And as practice showed (my case :laughing: ), it just doesnt work in some cases.

We have probably talked about the advantages/disadvantages already. Just in summary, i can think of 3 ways i would want to distribute my game:

  1. As an archive
  2. As an installer
  3. As a single exe

The new system allows the creation of the last 2, BUT it has these ‘disadvantages’:

  1. Requires access to the internet and you can avoid that only for the 2nd case.

  2. Requires the files to be in a multifile (.p3d) and the sourcecode in pyc and models in bam. Your last post explained why modifying the py source wont work, but I still dont understand why the rest of the files must be in a multifile. I understand a bam is faster, and packpanda allowed to do that too, but simply as an option, not the only way around.
    And I gave my reasons why I wouldnt want my files in a multifile. You said that you might be nice enough to add a feature to extract them, but not all of them, which wont really work in my case (and i didnt quite understand which), and again why would we, wanting to distribute our game like this (option 2, files extracted) needed to put them in a multifile in the first place? I understand thats because of how packpanda/pdeploy work, but it doesnt seem more convenient than Packpanda for this case. The only argument I see is the reduced size of the panda3d library, but (again for only this case) the cons outweight the pros.

So it seems like the p3d ways of distributing a game are not a replacement to packpanda and it should be included as an alternative way of distributing your game.
I know I can just download the script separately now, but what about the other users? Packpanda is a small script anyway, right? I dont see whats the problem in keeping it.

I hope you wont take this post as a rant or something, I understand that you guys worked hard on this and probably didnt even get paid for it, I just want to let you guys know that this system just doesnt work in some cases. I think showing a real case of using the tools will help you in understanding what is needed to change/improve.
I did spend hours trying to get them to work for me with no luck, so yeah, Im a bit angry, but I hope i didnt express it here.
And Im sure Ill be using the p3d way of distributing a game for other types of projects in the future.

52

I don’t want to maintain packpanda any longer. It’d become too much of a burden having to maintain it for a quite small group of people.

I just don’t understand; if you want people to be able to modify the source code, why not just ship a source archive for those people?

It’s a pretty common thing for projects to ship two things separately; binary installers and source code archives. I’ve never seen projects that have installers for something that’s supposed to be modifiable; that doesn’t seem to make sense for me anyway.

53

What do you mean ‘maintain’? Just keep it as is. Works for me.

And like i said its not just the sourcecode.

54

Packpanda is already buggy; with large rapidly-changing software like Panda3D, unmaintained code easily rots. That’s especially the case for packpanda, which constantly needs to be tested and constantly bugs appear that require to be fixed. It’s a significant extra burden.

What do you mean, “it’s not just the source code” ? I don’t understand exactly why people who want to modify the assets or source, wouldn’t be prepared to download the source archive of your game.

The system is just not made to allow people to modify the assets or code after they are packaged. The whole .p3d architecture was designed while keeping in mind that the game is binary and immutable.

The moral of the story: the runtime system is designed for packaged games that are immutable and work as efficient as possible, while people that want to modify anything will want to use the SDK (because essentially, by modifying assets or code, they are ‘developing’ the ‘software’, hence they need the ‘Software Development Kit’).

Because packpanda was developed long before work was started on the runtime system, essentially it packs the game together with the SDK. This generates inefficient and bloated binaries, and we no longer want to encourage this workflow.

55

One word: “modding”

56

I don’t understand why that is a problem. You can still load external .py files or .egg files if you package your game using this system.

You could even use packages for mod packs. Modders who want to release their mods as compiled packages could use ppackage to make them into packages.

57

How is this not a problem?
Modders will have to unpack tons of files to be able to edit a single one and then pack all of them into one again.

And I dont see whats the point with forcing a single way of doing things.

58

Design your system in a way that modders won’t need to modify any files, but use a plugin-like interface so that modders can create their own modules that add or change functionality of the game.

Isn’t it reasonable, though, that someone who wants to modify the game’s source files will have to download the source of the game? :confused:

59

Im talking about models files, image files and data files (binary/text).

You explained that modifying the code might crash the game because of the ‘optimized’ panda runtime, Im not talking about code. Though if it was possible, i dont see why not?

60

Why not do something like this: if “.egg” file exists in the program directory, load it, otherwise just load “” from the virtual file system?