Egg Octree

Well, it would allow you to ditch the collision system for the falling-through-terrain checking and use getElevation.
If you don’t want to use the collision system for height checking you could also make a top orthographic depth render of the scene, then you’ll have a heightmap which you can poll for terrain height (while still using your own mesh).

This came up in another thread, but it really belongs here.

I converted the eggoctree by Treeform/Lethe to using the geomVertexReader/writer interface and collisionpolygons. Simply supply it a node and it will return a hiarchy of nodes with collision polygons. Any geoms underneath the node supplied will be represented, no need for a flatten strong or clever egg manipulation. The result is not exactly render friendly though, you lose everything except for vertex positions so this works pretty much only for collisions.

edit: made it safe to use with invalid triangles


"""
							 _                 
							| |               
  ___  __ _  __ _  ___   ___| |_ _ __ ___  ___
 / _ \/ _` |/ _` |/ _ \ / __| __| '__/ _ \/ _ \
|  __/ (_| | (_| | (_) | (__| |_| | |  __/  __/
 \___|\__, |\__, |\___/ \___|\__|_|  \___|\___|
	   __/ | __/ | by treeform	   
	  |___/ |___/  modified by mindstormss                                                            
			 
This is a replacement of raytaller wonderful
Egg Octree script many people had problem using it
( i always guessed wrong about the size of cells )
and it generated many "empty" branches which this
one does not.  This one also uses geoms/primitives instead of egg data for real time usage
original see : ( https://discourse.panda3d.org/viewtopic.php?t=2502 )
This script like the original also released under the WTFPL license.
Usage: octreefy(node)
node -> node to be turned into an octree. Will create an octree for this node and a seperate octree for each child of this node
returns the octree as a node. only vertex information is transfered to this new node
"""
__all__ = ['octreefy']
from pandac.PandaModules import *
   
def getCenter(vertexList):
	""" get a list of Polywraps and figure out their center """
	# Loop on the vertices determine the bounding box
	center = Point3(0)
	i = 0
	for vtx in vertexList:
		center += vtx.center
		i+=1
	if i:
		center /= i
	return center

def flatten(thing):
	""" get nested tuple structure like quadrents and flatten it """
	if type(thing) == tuple:
		for element in thing:
			for thing in flatten(element):
				yield thing
	else:
		yield thing

def splitIntoQuadrants(vertexList,center):
	"""
		+---+---+    +---+---+
		| 1 | 2 |    | 5 | 6 |
		+---+---+    +---+---+
		| 3 | 4 |    | 7 | 8 |
		+---+---+    +---+---+
		put all poly wraps into quadrents
	"""
	quadrants = ((([],[]),
				 ([],[])),
				 (([],[]),
				  ([],[])))
	for vtx in vertexList:
		vtxPos = vtx.center
		x =  vtxPos[0] > center[0]
		y =  vtxPos[1] > center[1]
		z =  vtxPos[2] > center[2]
		quadrants[x][y][z].append(vtx)
	quadrants = flatten(quadrants)
	return quadrants

class Polywrap:
	"""
		its a class that defines polygons center
		so that it does not have to be recomputed
	"""
	polygon = None
	center = None
   
	def __str__(self):
		""" some visualization to aid debugging """
		return str(self.polygon.getNumVertices())+":"+str(self.center)
   
def genPolyWraps(vdata,prim):
	""" generate a list of polywraps from a group of polygons """
	vertex = GeomVertexReader(vdata, 'vertex')
	for p in range(prim.getNumPrimitives()):
		s = prim.getPrimitiveStart(p)
		e = prim.getPrimitiveEnd(p)
		center = Vec3(0)
		num = 0
		for i in range(s, e):
			vertex.setRow(prim.getVertex(i))
			center+=vertex.getData3f()
			i+=1
		if i: center/=i
		pw = Polywrap()
		pw.polygon = p
		pw.center = center
		yield pw
		 
def buildOctree(vdata,prim,maxNumber,verbose):
	""" build an octree from a primitive and vertex data """
	polywraps = [i for i in genPolyWraps(vdata,prim)]
	if verbose: print len(polywraps),"triangles"
	center = getCenter(polywraps)
	quadrants = splitIntoQuadrants(polywraps,center)
	node = NodePath(PandaNode('octree-root'))
	for n in recr(quadrants,vdata,prim,maxNumber,verbose):
		n.reparentTo(node)
	return node

def recr2(quadrants,vdata,prim,maxNumber,verbose,indent=0):
	"""
		visit each quadrent and create octree there
	"""
	vertex = GeomVertexReader(vdata,'vertex')
	qs = [i for i in quadrants]
	if verbose: print "    "*indent,"8 quadrents have ",[len(i) for i in qs]," triangles"
	for quadrent in qs:
		if len(quadrent) == 0:
			if verbose: print "    "*indent," no triangles at this quadrent"
			continue
		elif len(quadrent) <= maxNumber:
			center = getCenter(quadrent)
			if verbose: print "    "*indent," triangle center", center, len(quadrent)
			p = GeomTriangles(Geom.UHStatic)
			for pw in quadrent:
				s = prim.getPrimitiveStart(pw.polygon)
				e = prim.getPrimitiveEnd(pw.polygon)
				l = []
				for i in range(s,e):
					l.append(prim.getVertex(i))
				p.addVertices(*l)
				p.closePrimitive()
			geom = Geom(vdata)
			geom.clearPrimitives()
			geom.addPrimitive(p)
			geomnode = GeomNode('gnode')
			geomnode.addGeom(geom)
			node = NodePath('leaf-%i'%indent)
			node.attachNewNode(geomnode)
			yield node
		else:
			node = NodePath('branch-%i'%indent)
			center = getCenter(quadrent)
			for n in recr(splitIntoQuadrants(quadrent,center),vdata,prim,maxNumber,verbose,indent+1):
				n.reparentTo(node)
			yield node

def recr(quadrants,vdata,prim,maxNumber,verbose,indent=0):
	"""
		visit each quadrent and create octree there
	"""
	vertex = GeomVertexReader(vdata,'vertex')
	qs = [i for i in quadrants]
	if verbose: print "    "*indent,"8 quadrents have ",[len(i) for i in qs]," triangles"
	for quadrent in qs:
		if len(quadrent) == 0:
			if verbose: print "    "*indent," no triangles at this quadrent"
			continue
		elif len(quadrent) <= maxNumber:
			center = getCenter(quadrent)
			if verbose: print "    "*indent," triangle center", center, len(quadrent)
			p = GeomTriangles(Geom.UHStatic)
			collNode = CollisionNode('leaf-%i'%indent)
			
			for pw in quadrent:
				s = prim.getPrimitiveStart(pw.polygon)
				e = prim.getPrimitiveEnd(pw.polygon)
				l = []
				for i in range(s,e):
					l.append(prim.getVertex(i))
				for i in range(0,len(l),3):
					v = []
					for i2 in range(3):
						vertex.setRow(l[i+i2])
						v.append(Point3(vertex.getData3f()))
					if not CollisionPolygon.verifyPoints(*v): continue	#not a valid triangle
					p = CollisionPolygon(*v)
					collNode.addSolid(p)
			
			node = NodePath('leaf-%i'%indent)
			node.attachNewNode(collNode)
			yield node
		else:
			node = NodePath('branch-%i'%indent)
			center = getCenter(quadrent)
			for n in recr(splitIntoQuadrants(quadrent,center),vdata,prim,maxNumber,verbose,indent+1):
				n.reparentTo(node)
			yield node
	
def combine(node):
	'''combines all of the geoms into one. a preprocessing step'''
	newVdata = GeomVertexData('name', GeomVertexFormat.getV3(), Geom.UHStatic)
	vertexWriter = GeomVertexWriter(newVdata, 'vertex')
	newPrim = GeomTriangles(Geom.UHStatic)
	startingPos = 0
	pos = 0
	
	for node in node.findAllMatches('**/+GeomNode'):
		geomNode = node.node()
		for i in range(geomNode.getNumGeoms()):
			geom = geomNode.getGeom(i).decompose()
			vdata = geom.getVertexData()
			vertexReader = GeomVertexReader(vdata,'vertex')
			while not vertexReader.isAtEnd():
				v = vertexReader.getData3f()
				vertexWriter.addData3f(v[0],v[1],v[2])
				pos+=1
			for i in range(geom.getNumPrimitives()):
				prim = geom.getPrimitive(i)
				for i2 in range(prim.getNumPrimitives()):
					s = prim.getPrimitiveStart(i2)
					e = prim.getPrimitiveEnd(i2)
					for i in range(s, e):
						newPrim.addVertex(prim.getVertex(i)+startingPos)
					newPrim.closePrimitive()
			startingPos=pos
	return [newVdata,newPrim]
			
def octreefy(node,maxNumber=3,verbose=False):
	"""
		octreefy this node and it's children
		using the buildOctree functions
	"""
	vdata,prim = combine(node)	#combine all of the geoms into one vertex/triangle list
	#print vdata
	#print prim
	return buildOctree(vdata,prim,maxNumber,verbose)	#build the octree

Wouldn’t it be better if you took thos and then save it to be loaded off instead so you dont have to keep reloading it?

But, nice work.

in some cases, not in all. You can still save this as a bam file (NodePath.writeBamFile()) which loads faster than the egg, but more importantly this allows for dynamic generation of terrain or whatever–if it is different each time. This also allows you to also create an octree of multiple models from seperate egg files – not quite as easy as with the egg file implementation.

Essentially it still does the same thing and can be used in the same way (precompute and save) but uses a different back end that allows for a little bit more flexibility (and seems to be slightly faster too).

For my purposes the egg file version would not work without some significant work arounds which would take much more time and so this was the result.

First of all, I’d like to give a great big thanks to the hard work of Raytaller for his original EggOctree script, and Treeform for converting it to use Panda3d’s realtime geometry representations. And finally, for Mindstormss for converting it to use CollisionPolygons.

Here’s my contribution to this wonder bit of code: A version which generates octrees or quadtrees, output as either geometry or CollisionPolys. I also cleaned up the script a bit, removing what I considered to be redundant code. (Combining GeomNodes by hand, for example, when all you need to do is just FlattenStrong on the incoming code first. This is a bit more flexible if you want to use this for rendering, as it’s not necessary to deal with normals, texcoords, etc.)

I also found a pretty heinous bug. genPolyWraps() was incorrectly calculating the center. I was wondering why the resulting trees were very unevenly distributed… It was very noticeable when I modified the code to generate quadtrees. This resolves that quite nicely, and now the leaves are correctly split.

An example of quadtreefied terrain:

Anyway, here it is, enjoy! You can also find it on my Panda3d page here: http://www.gapingwolf.com/panda3d

"""
                                       _                            ___
                                      | | _                        / __) _
  ___    ____  ____  _   _   ____   _ | || |_    ____  ____  ____ | |__ | |_  _   _
 / _ \  / ___)/ _  || | | | / _  | / || ||  _)  / ___)/ _  )/ _  )|  __)|  _)| | | |
| |_| |( (___| | | || |_| |( ( | |( (_| || |__ | |   ( (/ /( (/ / | |   | |__| |_| |
 \___/  \____)\_|| | \____| \_||_| \____| \___)|_|    \____)\____)|_|    \___)\__  |
                 |_| original by treeform, modified by mindstormss           (____/
                     bug fixes and quadtree support by fenrirwolf


This is a replacement of raytaller's wonderful Egg Octree script.

Many people had problem using it (I always guessed wrong about the size of
cells) and it generated many "empty" branches which this one does not.  This one
also uses geoms/primitives instead of egg data for real time usage

original see : ( https://discourse.panda3d.org/viewtopic.php?t=2502 )
treeform's rewrite: ( https://discourse.panda3d.org/viewtopic.php?p=23267#23267 )
mindstormss's mod: ( https://discourse.panda3d.org/viewtopic.php?p=41362#41362 )
fenrirwolf's mod: ( https://discourse.panda3d.org/viewtopic.php?p=44007#44007 )

This script like the original also released under the WTFPL license.

Fenrir: I have also modified the script to support quad trees, plus fixed a bug
with calculating polywrap centers.  I removed combine(), because I felt it was
redundant.

Usage:
    newnode = octreefy (node, type='colpoly', maxDensity=64, verbose=0)
    newnode = quadtreefy (...)   [same parameters as above]

The input node is the node to be turned into an octree.  This can either be a
GeomNode or a PandaNode with a GeomNode child.  Will create a quad/octree for
this node.  Note that we do not expect multiple GeomNodes -- Flatten your
heirarchy first before you hand it over.  (If you want the old combiner method
back, just copy the combine function over from Mindstormss's script.)

The quad/octree is returned as a new node.  This node does not contain the
original node's states, so you will need to assign as appropriate.

Set verbose to 1 if you want to see a breakdown of what is returned.  Set it to
2 if you would also like to see tight bounds plus a random color for each leaf.

The type parameter controls what kind of geometry is returned.  If it set to
'geom', then a GeomNode with Primitives will be returned.  If it set to
'colpoly', then CollisionPolygons are returned.  You want to use CollisionPolys
if you intend to use this quad/octree for collisions, as it is much faster than
using GeomNodes.
"""

def getCenter(vertexList):
    """ Get a list of Polywraps and figure out their center """
    # Loop on the vertices determine the bounding box
    center = Point3(0)
    i = 0
    for vtx in vertexList:
        center += vtx.center
        i+=1
    if i:
        center /= i
    return center

def flatten(thing):
    """ Get nested tuple structure like quadrents and flatten it """
    if type(thing) == tuple:
        for element in thing:
            for thing in flatten(element):
                yield thing
    else:
        yield thing

def splitIntoQuadrants(vertexList,center):
    """
      +---+---+    +---+---+
      | 1 | 2 |    | 5 | 6 |
      +---+---+    +---+---+
      | 3 | 4 |    | 7 | 8 |
      +---+---+    +---+---+
      Put all poly wraps into quadrants
    """
    quadrants = ((([],[]),
             ([],[])),
             (([],[]),
              ([],[])))
    for vtx in vertexList:
        vtxPos = vtx.center
        x =  vtxPos[0] > center[0]
        y =  vtxPos[1] > center[1]
        z =  vtxPos[2] > center[2]
        quadrants[x][y][z].append(vtx)
    quadrants = flatten(quadrants)
    return quadrants

def splitInto2DQuads(vertexList, center):
    """
        +---+---+
        | 1 | 2 |
        +---+---+
        | 3 | 4 |
        +---+---+
        Put all polywraps into 2d quads.
        Note we assume Z-up for standard Panda coordinate space
    """
    quadrants = (([],[]),
                 ([],[]))
    for vtx in vertexList:
        vtxPos = vtx.center
        x =  vtxPos[0] > center[0]
        y =  vtxPos[1] > center[1]
        quadrants[x][y].append(vtx)
    quadrants = flatten(quadrants)
    return quadrants

class Polywrap:
    """
        It's a class that defines polygons center, so that it does not have to
        be recomputed.
    """
    polygon = None
    center = None

    def __str__(self):
        """ Some visualization to aid debugging """
        return str(self.polygon.getNumVertices())+":"+str(self.center)

def genPolyWraps(vdata, prim):
    """ Generate a list of polywraps from a group of polygons """
    vertex = GeomVertexReader(vdata, 'vertex')
    for p in range(prim.getNumPrimitives()):
        s = prim.getPrimitiveStart(p)
        e = prim.getPrimitiveEnd(p)
        center = Vec3(0)
        num = 0
        for i in range(s, e):
            vertex.setRow(prim.getVertex(i))
            center+=vertex.getData3f()
        center/=e-s
        pw = Polywrap()
        pw.polygon = p
        pw.center = center
        yield pw

def recr(quadrants, vdata, prim, type, maxDensity, verbose, quadsplitter, \
        indent=0):
    """
    Visit each quadrant and create a tree.

    quadrants = iterator of quadrants that have been generated for this
        branch

    vdata,prim = data carried over from initial combine

    type = 'geom' or 'colpoly', indicates what kind of PandaNodes to
        generate

    maxDensity = How many triangles to allow per leaf

    verbose = Verbosity debug level, 0=lowest, 2=highest

    quadsplitter = The quadrant space splitting function (can be quadtree or
        octree)
    """
    vertex = GeomVertexReader(vdata,'vertex')
    qs = [i for i in quadrants]
    if verbose: print "    "*indent,len(qs),"quadrants have ",[len(i) for i in qs]," triangles"
    for quadrant in qs:
        if len(quadrant) == 0:
            if verbose: print "    "*indent," no triangles at this quadrant"
            continue
        elif len(quadrant) <= maxDensity:
            center = getCenter(quadrant)
            if verbose: print "    "*indent," triangle center", center, len(quadrant)
            p = GeomTriangles(Geom.UHStatic)
            if type is 'colpoly':
                colNode = CollisionNode('leaf-%i'%indent)
            for pw in quadrant:
                s = prim.getPrimitiveStart(pw.polygon)
                e = prim.getPrimitiveEnd(pw.polygon)
                l = []
                for i in range(s,e):
                    l.append(prim.getVertex(i))
                if type is 'geom':
                    p.addVertices(*l)
                    p.closePrimitive()
                elif type is 'colpoly':
                    for i in range(0,len(l),3):
                        v = []
                        for i2 in range(3):
                            vertex.setRow(l[i+i2])
                            v.append(vertex.getData3f())
                        p = CollisionPolygon(*v)
                        colNode.addSolid(p)

            node = NodePath('leaf-%i'%indent)
            if type is 'geom':
                geom = Geom(vdata)
                geom.clearPrimitives()
                geom.addPrimitive(p)
                geomNode = GeomNode('gnode')
                geomNode.addGeom(geom)
                node.attachNewNode(geomNode)
            elif type is 'colpoly':
                node.attachNewNode(colNode)
            if verbose>1:
                if type is 'geom':
                    node.setColor (random.uniform(0,1), random.uniform(0,1), \
                        random.uniform(0,1), 1)
                node.showTightBounds()
            yield node
        else:
            node = NodePath('branch-%i'%indent)
            center = getCenter(quadrant)
            for n in recr(quadsplitter(quadrant,center), vdata, prim, type, \
                                maxDensity, verbose, quadsplitter, indent+1):
                n.reparentTo(node)
            if verbose>1:
                if type is 'geom':
                    node.setColor (random.uniform(0,1), random.uniform(0,1), \
                        random.uniform(0,1), 1)
                node.showTightBounds()
            yield node

def octreefy(node, type='geom', maxDensity=4, verbose=0, \
    normal=False, texcoord=False, binormal=False):
    """
    Octreefy this node and it's children.

    type = 'geom' or 'colpoly'.  Will generate either GeomNodes or
        CollisionPolys.

    maxDensity = How 'deep' to make the tree, will make sure each leaf has
        no more than X triangles in it

    verbose = Enable some debugging info, set to 1 for console output, 2
        for debug info
    """
    # Sanity check
    if type is not 'geom' and type is not 'colpoly':
        print 'Unknown type of',type,',only geom or colpoly allowed!'
        return

    # Let's look for a GeomNode under this nodepath.
    # We don't search too deep, only checking the first child, because we are
    # expecting a flattened structure.
    if not node.node().isGeomNode():
        geomNode = node.getChild(0).node() # Try to get first child
    else:
        geomNode = node.node()
    if not geomNode.isGeomNode():
        print 'We require a single GeomNode.  Flatten first!'
        return
    geom = geomNode.getGeom(0).decompose()
    vdata = geom.getVertexData()
    prim = geom.getPrimitive(0)

    # Generate polywraps for our vertices
    polywraps = [i for i in genPolyWraps(vdata,prim)]
    if verbose: print len(polywraps),"triangles in polywraps"

    # Find the center of the entire mess
    center = getCenter(polywraps)

    # Do first split
    quadrants = splitIntoQuadrants(polywraps, center)

    # Now let's start working our way down the tree
    node = NodePath(PandaNode('octree-root'))
    for n in recr(quadrants, vdata, prim, type, maxDensity, verbose, splitIntoQuadrants):
        n.reparentTo(node)

    return node


def quadtreefy(node, type='geom', maxDensity=4, verbose=0, \
    normal=False, texcoord=False, binormal=False):
    """
    quadtreefy this node and it's children.

    type = 'geom' or 'colpoly'.  Will generate either GeomNodes or
        CollisionPolys.

    maxDensity = How 'deep' to make the tree, will make sure each leaf has
        no more than X triangles in it

    verbose = Enable some debugging info, set to 1 for console output, 2
        for debug info
    """
    # Sanity check
    if type is not 'geom' and type is not 'colpoly':
        print 'Unknown type of',type,',only geom or colpoly allowed!'
        return

    # Let's look for a GeomNode under this nodepath.
    # We don't search too deep, only checking the first child, because we are
    # expecting a flattened structure.
    if not node.node().isGeomNode():
        geomNode = node.getChild(0).node() # Try to get first child
    else:
        geomNode = node.node()
    if not geomNode.isGeomNode():
        print 'We require a single GeomNode.  Flatten first!'
        return
    geom = geomNode.getGeom(0).decompose()
    vdata = geom.getVertexData()
    prim = geom.getPrimitive(0)

    # Generate polywraps for our vertices
    polywraps = [i for i in genPolyWraps(vdata, prim)]
    if verbose: print len(polywraps), 'triangles in polywraps'

    # Find the center of the entire mess
    center = getCenter(polywraps)
    if verbose: print center, 'is center'

    # Do first split
    quadrants = splitInto2DQuads(polywraps, center)

    # Now let's start working our way down the tree
    node = NodePath(PandaNode('quadtree-root'))
    for n in recr(quadrants, vdata, prim, type, maxDensity, verbose, splitInto2DQuads):
        n.reparentTo(node)

    return node

I have created a git hub of the scripts
github.com/treeform/eggOctree

Please if you have any bug fixes feel free to just commit them. Any one can write to the tree. Thanks!

Some reason I keep getting

Traceback (most recent call last):
  File "backup.py", line 627, in LoadingList
    self.loadzone()
  File "backup.py", line 1448, in loadzone
    self.collisionenviron = octreefy(self.TempMap, type='colpoly', maxDensity=64
, verbose=0)
  File "C:\game\eggoctree.py", line 253, in octreefy
    polywraps = [i for i in genPolyWraps(vdata,prim)]
  File "C:\game\eggoctree.py", line 129, in genPolyWraps
    vertex = GeomVertexReader(vdata, 'vertex')
NameError: global name 'GeomVertexReader' is not defined

Note that the script is just a snippet, really.

If you toss in…

from pandac.PandaModules import *

…then it should work.

Awesome!

i think i might have done some thing wrong

from pandac.PandaModules import *

Yea I all ready did that… so werid… maybe it didn’t save or something, i’ll try again in a sec.

i think i might have done some thing wrong

I’m not sure; I didn’t try that hard to see what was up. I was just testing the code. I’ll see if maybe something didn’t save right.

Update**
KK got it to run but its missing alot of the models for collision, i’ll see whats up with that and rework on that.

I guess the file didn’t save the above code the frist time, so thats why it wasn’t working.

could you commit the working version back to git hub?

I am thinking of making a *.p3d tool with octree script could that be included in the SDK?

But if it’s included in the SDK anyway, where a Python interpreter is certainly available, why not just distribute it as a .py file? I’m not sure that shipping it as a p3d file makes it any easier to use; and it certainly makes it harder to understand what it is.

The only reason we ship packp3d et al as a p3d file is because doing so binds in the specific information required (URL and version string) for downloading the matching version of Panda.

David