Geom filtering and Culling.

Scripting Issues
1

Hi everyone,

I realized today that the geom were the cullling primitive of panda. As we have culling problem (more than 70% of our time according to pstats), I wonder if there is a preexisting way to filter the Geoms of the scene-graphs.

More precisely, I want to group identic geoms by geographical zone, so to call flattenStrong by zone and by geoms type.

For example, flattenStrong on the tree geom type outside the game area.

My idea is to use a quattree/octree to split the world and then navigate recursively the zones to group the geoms by zone but maybe there is another simpler way ?

2

The phase known as “cull” is not strictly performing view-frustum culling, despite its name. Most of the time spent in cull is time spent traversing the scene graph, accumulating transforms and state along the way.

If your scene is dominated by time spent in cull, you will best optimize it by reducing the number of nodes. To a lesser extent, you can also improve frame rate by reducing the number of transforms and state changes on those nodes, as well as by reducing the number of individual Geoms.

flattenStrong() achieves all of these. Naively, you could just assemble your scene and then call root.flattenStrong(). However, there are limits to the effectiveness of flattenStrong(); in particular, it can’t combine Geoms with different states (e.g. different textures), which is the most common factor limiting scene graph optimization. So you might need to put some effort into ensuring that your models–especially pieces of a single model–share one common texture instead of lots of little textures.

One tool that does this is egg-palettize. By running your models through egg-palettize, with a carefully grouped textures.txa files, you can reduce lots of little textures to a few large textures, which not only directly improves rendering performance, but it also improves the effectiveness of flattenStrong(), leading to further performance improvements.

The art of optimizing your scene for rendering performance is a complex art, and I can’t possibly describe all the techniques in one little post. But I hope I have given you an idea of where to begin.

David

3

Thank you for your answer, I assumed as much from other threads.

Another question, is FlattenStrong compatible with the instanceTo mecanism?

They seems incompatible : multi-parenting vs pushing the transform into the vertices.

4

Actually, if I understand it correctly, flattenStrong does more than applying transforms. I think it stores them all into 1 Geom object. If you just need to apply the transform onto the vertices, you can use flattenLight. (or flattenMedium, if you like)

5

We can’t use flattenLight or flattenMedium.
We need to diminish the node count (we are over 100.000). FlattenStrong seems the only easy way to do that .

I was just underlining the possible problem with instanceTo as we use instanciation too.

6

Right, flattenMedium() will reduce the node count somewhat, though not as effectively as flattenStrong(). The difference is that flattenStrong() might break the hierarchical culling groups in your scene graph, so it is more appropriate to call flattenMedium() on the toplevel of your scene, and only call flattenStrong() on the individual cullable units (like a car, or a tree, or whatever). The danger of overflattening is that you will make culling less effective by combining offscreen objects together with onscreen objects, so that your graphics card will have to render more vertices.

All the flatten methods–flattenLight, flattenMedium, and flattenStrong–will duplicate out instanced nodes into duplicate copies. This is generally faster for rendering anyway. The only reason to use instancing is if you really have some extremely tight memory restrictions, or if you need to take advantage of the shared transforms that instancing causes (for instance, you want to animate one node only once, even though it is appears in the scene graph multiple times).

David

7

thank you, david but my pb are not over …yet.

FlattenMedium doesn’t like me.

I do:
self.node.flattenMedium()

and I got :
AssertionError: !(pos.is_nan() || quat.is_nan() || scale.is_nan() || shear.is_nan())

I am a bit puzzled, as it’s normal node, setup as any other…

8

This means that some part of one of the transforms at self.node or below has a NaN in it: this is computer-shorthand for “not a number”, or an invalid numeric value.

Typically you get these invalid numeric values by dividing by zero, or by inverting a singular matrix (e.g. a scale to 0.0 in one or more axes), or by some other invalid numeric operation. Sometimes they come from uninitialized memory or a similar problem, though this is less common with Python code.

If you have a scale to 0.0, try replacing it with a scale to 0.001 instead. If you have an actual NaN in your scene graph, see if you can figure out where it came from and fix it. You can try self.node.ls() to list the node and its descendants, which should show you any NaN’s.

David

9

[UPDATE] I reread david post and I think It gave the explanation :

Thank you, david, I’ve done that.
but I have a last question (for the time being) : when I flattenStrong, if many Geoms are under a geomNode, they are not merged.

for example :

ModelRoot SFR_BatType6b.egg [material_type] T:(scale 0.000735293)
  Character SFR_BatGenType6b
    GeomNode  (27 geoms: TextureAttrib TransparencyAttrib)

turn into :

PandaNode model [material_type]
  GeomNode  (26 geoms: TextureAttrib TransparencyAttrib)
None

As each Geom is a drawPrimitive call, I am a bit worried BatGenType6b is a quite simple building. If it needs 26 or 27 (why the one missing ?) drawPrimitives, I don’t know what more complex building will need…

10

There are several things that can keep Geoms from being combined together.

Most commonly, it’s the render state. Geoms that have a different render state cannot be merged. (This is for the simple reason that each Geom is a single call. Panda has to change the render state, render the Geom, change the render state, render the next Geom, and so on.) So if some of your Geoms have a TransparencyAttrib and some do not, they will be kept separate. Also, if some of them have texture A applied, and some have texture B applied, they will be kept separate.

Also, Geoms with a different vertex format cannot be combined. So if some of your polygons have normals and some don’t, for instance, they will be kept separate. Similarly for named texture coordinates.

Also, Geoms cannot contain more than a certain number of vertices. This limit is determined by your graphics card. There is one limit to the total number of vertices in the GeomVertexData, and a separate limit to the total number of vertices referenced by each Geom. So if you have many thousands of vertices here, Panda may need to separate them into different Geoms just to satisfy the limits of your graphics card.

Finally, a single Geom cannot contain combinations of points, lines, and polygons; each Geom must be all points, all lines, or all polygons.

If you’re not sure which case is causing your Geoms to be kept separate here, you can examine your Geoms one at a time, like this, after flatten:

gnp = root.find('**/model/+GeomNode')
gn = gnp.node()
for i in range(gn.getNumGeoms()):
  print "Geom %s:" % (i)
  print gn.getGeomState(i)
  gn.getGeom(i)
  gn.getGeom(i).getVertexData()

There is also a possibility that the ModelRoot and/or the Character node are interfering with the flattening (that’s what a ModelRoot means–“don’t flatten this node”). I don’t think they’re supposed to prevent Geoms from being combined, though, so that’s probably not the problem, but it might be worth doing a find to the Geom node and trying another flattenStrong() from there just to prove it.

David

11

I think the Character nodes are interfering with the flattening.

I removed all the model nodes and the mesh “ob_lizd_shrd_Barrel_Conus” (see below) are low poly (<100). Hence, The limit of my card is surely not reached (it’s a nvidia 7800).

Conclusion : I need to get rid of the Character nodes and the Model Node in one swift movement.

before flattenStrong :

PandaNode textured_ground113 T:(pos 490.902 39.1353 15.5 hpr 270.116 0 0)
  PandaNode cross T:(pos 0 0 0.2) S:(TransparencyAttrib) (hidden)
    GeomNode Polygon114 (1 geoms)
  PandaNode parcel T:(hpr 90 0 0)
    CollisionNode cnode (1 solids) [default_tag] T:(scale 40 40 1) (hidden)
    PandaNode model [material_type] T:(pos 11.8863 13.2 0 hpr 41.0152 0 0 scale 0.025) S:(ShaderAttrib TransparencyAttrib)
      Character ar_lizd_town_MarketShop
        GeomNode  (1 geoms)
      CollisionNode cnode (1 solids) [default_tag] (hidden)
    PandaNode model [material_type] T:(pos 8.90611 5.40832 0 hpr 293.127 0 0 scale 0.025) S:(ShaderAttrib TransparencyAttrib)
      Character ar_lizd_town_MarketShop
        GeomNode  (1 geoms)
      CollisionNode cnode (1 solids) [default_tag] (hidden)

after flattenStrong :

PandaNode textured_ground113
  PandaNode cross (hidden)
    GeomNode Polygon114 (1 geoms: TransparencyAttrib)
  PandaNode parcel
    CollisionNode cnode (1 solids) [default_tag] (hidden)
    PandaNode model [material_type]
      Character ob_lizd_shrd_Barrel T:q(pos 485.555 49.2174 15.5 hpr 88.2551 0 0 scale 0.025)
        GeomNode  (1 geoms: ShaderAttrib TransparencyAttrib)
      Character ob_lizd_shrd_Barrel_Conus T:q(pos 485.557 48.0717 15.5 hpr -5.83961 0 0 scale 0.025)
        GeomNode  (1 geoms: ShaderAttrib TransparencyAttrib)
      Character ob_lizd_shrd_Barrel_Conus T:q(pos 485.56 46.9261 15.5 hpr -7.59106 0 0 scale 0.025)
        GeomNode  (1 geoms: ShaderAttrib TransparencyAttrib)
12

If your Character nodes represent Actors, you can create your Actors with the flag flattenable = 1. This means not to create a ModelRoot and to allow its nodes to be flattened together with sibling Actors.

A flattened Actor is a little funny; it may reference the same node as another Actor (e.g. if you reparent it somewhere else, you may get both of them at once). This is because the geometry of multiple Actors has been combined together into a single node. But each Actor remembers which subset of geometry within the node it animates, and so animations still work in the normal way.

A different solution is just to flattenStrong below the Character nodes. Something like this:

nodePaths = root.findAllMatches('**/+Character')
for np in nodePaths.asList():
  np.flattenStrong()

David

13

Oh, wait, I see the problem. It’s not the Character node itself, it’s the fact that you have a different transform on each Character node. This prevents Panda from being able to combine the sibling Character nodes together.

The next version of Panda (it will be numbered 1.5 when it is released) will include code that should allow these nodes to be combined anyway. By a coincidence, Josh has just finished adding the daily-build downloadables to this website; you might try downloading the latest daily build to see if this solves your problem.

David

14

thanks david,
I reparented the node under the Character and flattened after that.
It works like a charm : x10 in performance. :slight_smile: