using BAM format as mod sharing format?

General Discussion
1

Here’s my situation.
We’ve made a game where people can create and share their own model files. Till now we have been using a (limited) binary model format.

I’d like to change or add support for EGG models for features like normal mapping, but that format wouldn’t be too great for modders. It’s text-based, so loads slowly and has larger size. Sure, Panda generates BAMs from them and reuses that, but I’ve already gotten complaints from testers about the very slow loading times of the models when they use it for the first time and the quickly growing cache folder from the large amount of model mods already done.

BAM format as a mod sharing format would be perfect if it wasn’t for the format becoming incompatible quickly between different Panda versions.

I’m thinking maybe we sould write a tool that uses different versions of the bam2egg and egg2bam command line tools to convert BAMs between different Panda versions (read: “different versions of our game”).
Does this sound like a good idea?

2

For file size there’s .pz (as in pzip and punzip) that makes egg files nice and small(-er), but that’s no help with first slow load or flooded cache folder.

3

Pzip might solve the issue with large file sizes, but it actually worsens loading times compared to ordinary EGG as the file needs to be decompressed before being parsed.

4

bam2egg -> egg2bam isn’t a great way to convert between bam versions, because bam2egg isn’t a complete converter and could lose some properties that it doesn’t support. In any case, bam2egg -> egg2bam would be just as slow as loading up the original egg file, so I’m not sure what it really buys you over just distributing the egg file instead.

If you really need a fast-loading model format that’s version safe, you’ll have to design your own format and write your own model loader.

David

5

Is there at least an incomplete list of such properties?

Making a more powerful version of our current custom format is though choice as we’ll need to write all the importers/exporters for the 3d modellers and doubt we can support all the 3d packages which EGG/BAM does currently.
So I’ll want to be pretty darn sure the drawbacks of BAM really make it not worth it before going this road.

I was talking of an offline converter users could use to make their BAM mods work with different versions of the game.

6

bam2egg is really, really incomplete. It supports only the fundamental things: polygons, normals, simple textures. Anything else is omitted, including animation, shaders, multitextures, and so on. bam2egg was intended to be a convenient way to export basic models to external packages; it was never meant to fully support everything that Panda supports.

I’m sure you could find some compromise solution, though. For instance, you could store the egg file as the de facto format, and have your offline converter generate an appropriate bam file for each Panda version supported, or something like that. As long as you always keep the source egg file, you’ll be fine.

As a reminder, Panda’s two native formats are designed to have the following properties:

(1) egg - human readable, very generic format, can be easily converted to or from any number of other third-party formats.

(2) bam - closely matches Panda’s internal memory layout, can be loaded quickly and easily into one particular version of Panda (and maybe into future versions, but no guarantees).

Two side effects of (2)'s design is that bam files can be loaded quickly, and also they are tied to a particular version of Panda. But this also means that bam files can’t easily be generated from third-party formats directly; and they can’t be converted into third-party formats.

You’re seizing on the property of bam files that they can be loaded quickly; but this is exactly the property that also makes them inappropriate for the purpose you have in mind.

The bam-cache mechanism is designed to mitigate the longer load time of egg files, but of course you still have to load them once. Is that really so bad that you need to make all of this engineering effort to prevent that?

David