egg-palettize and image filenames

Scripting Issues
1

Hi

I’m using egg-palettize to group my textures and hit an interesting problem.
To make parameters for an image file to work I need to specify the filename without any directory part in it.
Let me explain. Here’s a txa file that doesn’t do what I want:

:group group1 dir group1
:imagetype .png

canapee.egg : group1
shelf.egg : group1

# textures for static geometry
data/textures/world/indoor/furniture/canapee01.png : force-rgba
data/textures/world/indoor/furniture/shelf_stock_1.png : force-rgba

in principle, it works fine: I get a palettized image, in the group1 directory and all, but the textures are not forced to RGBA.

To get it to work, I need to change the txa as follows:

:group group1 dir group1
:imagetype .png

canapee.egg : group1
shelf.egg : group1

# textures for static geometry
# I need to strip data/textures/world/indoor/furniture/ for force-rgba
canapee01.png : force-rgba
shelf_stock_1.png : force-rgba

So as you see, I need to strip the leading relative path of the textures to get it to work.
This is fine for this example, but for our production code that’s like a ticking time-bomb because some textures might have the same basename, although they are in different directories.

Is there a way to make egg-palettize to accept the full path for the image options?

Again, in my simplified example one could also fix it by a *.png: force-rgba, but for production code with loads and loads of files that wouldn’t work either because not all pngs referenced in the txa should be forced to rgba

Any comments?

Cheers,

Erik

2

No, sorry, this is an intentional design decision. egg-palettize assumes that all textures that have the same basename, regardless of their directory name, are actually intended to be the same image.

We made this decision because, frequently, artists will copy textures from one model to use in another model, and then you will have the same texture file appearing in two different directories. Allowing egg-palettize to group them together by filename allows us to avoid the wasting of texture memory that this would otherwise cause.

The flip side, of course, is that sometimes artists will give two different textures the same name, even if they are actually intended to be different textures. This is especially common with generic names like “floor.tiff” or “walls.tiff”.

When this happens, then egg-palettize will collapse the textures together, and apply the same texture in two different places. In one place, it will be the correct texture; in the other place, it will be the wrong texture.

Some time ago we analyzed it this way. There are two possibilities: either (a) egg-palettize should consider all different texture files to be globally unique, or (b) it should consider textures with the same basename to be the same texture.

In case (a), you will occasionally get textures needlessly duplicated in your palettes, wasting texture memory but causing no visible problems.

In case (b), you will occasionally get textures incorrectly applied to the wrong surface, causing visible problems.

We concluded that case (b) was preferable, because we would presumably be able to see when there was a problem, which would make it much easier to know when to fix it. In case (a), it might be very hard to discover the problems.

At the time we made this decision, Panda3D was not yet an open-source project. If it had been, we probably would have made this behavior selectable, so that different users could choose different behavior. Since egg-palettize was (at the time) only intended as an in-house tool, though, we hard-coded the behavior we selected into egg-palettize, and that’s the way it remains today.

So, to use egg-palettize safely you have to instruct your artists to follow this rule: assign each unique texture image a unique filename. We suggest that when textures are custom to one particular model, you name them with a prefix that suggests that model, e.g. building_a2_wall.png, and when they are intended to be shared by multiple models, you name them with a generic prefix, e.g. generic_brick_floor.png.

Incidentally, using a rule like *.png: force-rgba wouldn’t work anyway, because egg-palettize also strips off the filename extension (artists also will frequently convert image file types, e.g. walls.png to walls.tiff, without changing the contents of the image).

David

3

how about use an md5 to see if textures are the same?

4

We’ve thought of that, and considered implementing something like that. But we never got around to it.

One problem with the idea is that it would only work if the files are exactly the same, but we’ve had artists convert them to another format (as mentioned above), or even rescale them, without otherwise meaningfully changing the image.

David

5

i once wrote a small python script which was able to detect and delete duplicated image by their visual content.
its actually pretty easy. works like this
create small thumbnails of all your images. needs to be a fixed size. i found 8x8 pixels thumbnails to work fine for finding duplicates amongst 10thousands of images.
well all thats left to do is to compare the thumbnails pixel by pixel and sum up the differences between the color values.

the really slow part is the rescaleing since i did that using PIL.
if you just compare a few images (when they have the same name) it should be working just fine.

just as idea in case anyone wants to add something like that.
just ask for it and i’ll upload my small python command-line-tool

6

Not that I’m the lord of all things Panda, but would that not be a simple enough thing to make be an option for times like this?

By default, it can act as you decided was preferable, or else it can act as some people may want it to, with some override arguments… no?

7

It sounds simple, doesn’t it? But egg-palettize is a very big and complex program.

I’m sure it wouldn’t be very hard to make this change, but it’s a bit more than simple. And since it doesn’t benefit Disney any, it’s not high on my own priority list (I can only work on things like this on my own time).

If some enterprising community member wanted to investigate making this work, I’d be happy to accept patches, though. :slight_smile:

David