Panda3d paged geometry?

[Anon]

I wanted to keep this the last thing I’d do. It sounds like a difficult thing to implement. But I can’t save this for the later any longer.
I just have to start building my game world to be able to add other features.

I searched the net for some documentation with not much luck.
I came across this page: ogre3d.org/tikiwiki/PagedGeo … =Libraries
It pretty much sums up what I want to make.

I haven’t used Ogre before, nor I am a C++ user, so though ive read the manual, I can’t read the code to find out how they made it.

I hope you guys can give some tips and techniques used in such systems.

I wish to convert that system to python and Panda3d. The final goal is to post the results in Showcase section. I have few tree models and bunch of textures lying around.

According to the Ogre wiki page,

The PagedGeometry engine is an add-on to Ogre which provides highly optimized methods for rendering massive amounts of small meshes, covering a possibly infinite area. This is especially well suited for dense forests and outdoor scenes, with millions of trees, bushes, grass, rocks, etc., etc.

If you dont want to read their manual, basically you create a PagedGeometry instance and call some of its methods.

If it was in Python, the code could look like these (the quotes are from their manual):

trees = Pagedgeometry()
trees.setCamera(base.camera)

The camera is used to calculate LODs and cache pages of geometry smoothly. Without supplying a
camera to PagedGeometry, nothing can be rendered since most of the internal optimization algorithms are
based on the camera’s position

trees.setPageSize(50)

setPageSize(50) sets the size of a single “page” of geometry to 50x50 units. Internally, everything is
stored in a big grid of these blocks (called “pages”). Larger pages generally give better frame rates than
smaller ones, although they can cause stuttering as they are dynamically loaded if too large. The page
size is something that should be experimented with (later) to find the best performance.

trees.setInfinite()

setInfinite() instructs PagedGeometry not to force any boundaries on what geometry (trees, etc.) that
you are going to add. “Infinite mode” is used by default, so normally you really don’t need to call
setInfinite() at all, but it is used here for demonstration.

trees.setBounds()

As an alternate to setInfinite(), you could call PagedGeometry.setBounds(). This function allows you to
supply PagedGeometry with the boundaries where all your trees (or anything else this PagedGeometry object
will manage) will be contained. The only advantage to setBounds() over setInfinite() is that you may get a small
speed boost, since the cache efficiency will be inreased slightly based on the knowledge of your world size.

If you tried displaying something through PagedGeometry at this point, nothing would appear. Before
anything can actually be displayed on the screen, you need to configure how PagedGeometry is going to
represent your trees (or whatever) on the screen. This is done with the addDetailLevel() function:

trees.addDetaillevel('BatchPage', 150, 30)
trees.addDetaillevel('ImpostorPage', 400, 50)

The first line means all trees up to 150 units from the camera will be displayed with BatchPage (basically
no reduction in visual quality, but much faster than using plain entities). The second parameter (30) means that
this LOD will transition (fade) out 30 units beyond the 150 unit point. In this case it will be “morphing” into the
next LOD. Note that the transition parameter is optional, and should be avoided where possible since transitions
will reduce performance.
The second line adds a second detail level, ImpostorPage, which is displayed up to 400 units from the
camera. ImpostorPage uses flat billboard images in place of your trees, which are automatically textured to look
just like the “real” mesh. Impostors are very fast, but should be used at a distance to avoid the user noticing they
are really flat images. Notice that the transition parameter is used here also (50). Since there is no additional
LODs after this one, the transition means the impostors will smoothly fade out into the distance for 50 units.

Dont know what they mean by billboards. Maybe sprites? From the images the billboards could go over 300, or even 1000.
Also they say BatchPage is much faster than plain entities/ node(paths). Im guessing they use something like flatten() or rigid bodie combiner. As far as I know those can’t be ‘un-done’, so I would like to hear your opinion.

Note that addDetailLevel() is the only function in PagedGeometry that must be called in any specific
order; when adding detail levels, you must add them in order from closest to farthest. For example, you can’t add
a detail level that has a 400 unit range before a detail level that has a 150 unit range.

At this point PagedGeometry is ready to display your entities.

You have to give PagedGeometry a list of what you want displayed, and it will do so.

PagedGeometry class itself contains no functions for the addition / removal of
entities. Instead, you must create an instance of a PageLoader-derived class, and supply that instance to
PagedGeometry. You can think of a PageLoader as a list of entities that you give to PagedGeometry (although
the internal operation is much more complex and optimized).

treeLoader = TreeLoader(trees, TBounds(0, 0, 1500, 1500)
# And add a entity at the desired position/rotation/scale
treeLoader.addTree(myModel, position, hpr, scale)

Unlike the PagedGeometry engine itself,
TreeLoader3D always requires a boundary in which all your entities will be placed, due to the way they are
stored in memory (using a compression technique, so you can store millions of trees in only a few MBs of RAM)

No idea what compression technique it is.

After all your entities have been added to treeLoader, you need to assign it to your PagedGeometry
object using PagedGeometry.setPageLoader(), like this:

trees.setPageLoader(treeLoader)

Now PagedGeometry is fully configured and ready to render your trees, but there’s just one more thing;
each frame, you will need to call PagedGeometry.update() for your PagedGeometry instance. This function is
used to perform all the caching, batching, impostoring, transitioning, etc that you have configured. If you don’t
call it every frame, your trees (or whatever) won’t appear correctly

We could put this in a panda task:

trees.update()

Bushes, rocks, and other small ground details aren’t nearly as important as trees in the distance. If you
reduce the viewing range of this “ground clutter”, you’ll get a considerable performance boost without much
noticeable loss of visual quality. For example, if your trees are visible up to a half of a mile, your bushes / rocks /
etc. may only need to be rendered up to one or two hundred yards.

At this point you may be wondering how you’re going to assign multiple view ranges to different types of
entities, etc. Fortunately, there is a very easy way to achieve this: Create two instances of PagedGeometry - one
is set up to render trees how you want, and the other is set up to render bushes how you want (with the closer
view range)

The system has a separate way of handling grass, it also has a way to add shadowmaps to grass and trees, but that one is probably beyond my knowledge.

You can’t use ImpostorPage to display grass. Since grass is displayed using many small non cameraaligned
billboards,

i think they mean quads/planes when they say billboards here?
Im not really a fan of using billboards for grass. many games use it, but in my opinion its pretty noticeable and looks fake.

You should use either BatchPage, or preferably “GrassPage” to display grass, the latter being
the most optimized.

Fade transitions are always used on grass, and do not cost any performance. When configuring the
PagedGeometry object used to render your grass, do not enable transitions yourself. Doing this will only
reduce performance unnecessarily.

I would like to know about your opinion on what additional/unique optimizations can be used for the grass.

In most cases, you won’t want the grass to grow everywhere. You might only want grass to appear in
fields, or on lawn areas, for example. GrassLoader gives you complete control over the density levels of grass
anywhere on your terrain through density maps.

I don’t really like the idea of having density maps for trees and big vegetations. There are games with huge worlds where say, you have to look for treasure hidden under a specific tree. But I like the idea of using them for grass. I think there is a code snippets somewhere here on the forums which I could use as a reference.
---------------------------------------
Now, for positioning the trees, I could make a Blender script which would export positions, rotations ans scales of objects (empties?) to a simple text file, and shoot a collision ray for each tree in the beginning to get the z position.

Theres nothing mentioned about the terrain loading. Should we only use geomipterrains in this case?

For the BatchPage trees… how would you use RigidBodyCombiner in this case?

I would like to know of any optimization techniques which could be used in implementing such a system.

Im sorry for the huge post and I know this sounds pretty ambitious, but I hope the advice and techniques posted on this page can be helpful to others as well and will share all my results with the community.

[ynjh_jo]

I’ve done basically the same thing here :
[how to make these kind shodow?)

It’s grid based, so what I do is instance/copy the terrain chunks around the avatar from the pool to the scenegraph.

This is the relevant piece in updateDropShadow() :

      if self.CH.getAirborneHeight()<.1:
         minx = max( 0, int((w.terrainMinX+pos[0]-self.radius)/GRIDX) )
         miny = max( 0, int((w.terrainMinY+pos[1]-self.radius)/GRIDY) )
         maxx = min( int((w.terrainMinX+pos[0]+self.radius)/GRIDX), w.XSIZE-1 )
         maxy = min( int((w.terrainMinY+pos[1]+self.radius)/GRIDY), w.YSIZE-1 )
         idx = ( (minx,maxx), (miny,maxy) )
         if self.DSidx!=idx:
            self.DSidx=idx
            self.dropShadowGeom.removeChildren()
            ds = self.dropShadowGeom.attachNewNode('')
            for yi in range(miny,maxy+1):
                for xi in range(minx,maxx+1):
                    gnp = w.glist[xi][yi]
                    if gnp:
                       gnp.instanceTo(ds,0)
            ds.flattenStrong()

Enter debug mode and see how the white terrain chunks around the avatar are attached and removed from scenegraph.

[Anon]

What is that?

[tah]

I have the Ogre3D source, and Ogitor Editor to, they include the PagedGeometry; also have source for an open source game project called DungeonHack
[dungeonhack.sourceforge.net/Main_Page]
That uses the PagedGeometry in its infinite terrain game in development. Took a bit work to download all the parts involved and compile, also had to fix a few things for it to work for me, but it lays out terrain and trees and grass (with movement) very fast as you move yourself (FPS character) across the terrain.
The impostor trees look good but sometimes are shaded lighter than they should given an evening sun position, then when they switch to actual entity they get darker.
But all in all it’s pretty cool.
I do program in C++ and I was thinking of looking at the code and see how it could be used with Panda…

tim

[Anon]

Okay…

I didnt think of an exact copy. So you can share any of your optimization tricks, basically.

[Craig]

This kind of thing is planned to be supported in my terrain system, but is not done yet (its hardly started): https://github.com/Craig-Macomber/Panda3D-Terrain-System

Its mostly a matter of generating/loading tiles of mesh information, and flattening tiles of meshes at different LODs, and transitioning between LODs.

[Anon]

The problem here is, flattening is a one way operation, correct?

[Anon]

Ive put some base code for this we can work from.
The thing is I havent actually posted any opensource project before for others to be able to download/ modify. Where would you suggest me to host the project?

[rdb]

You can always host it at panda3dprojects.com/ .

[Anon]

I havent gotten an email. I think its about a month now. The problem is I changed my email and Im thinking the confirmation wasnt resend to the new one, and the old one doesnt exist any more.

[rdb]

I’m sure I approved it. I’ll resend the information in PM.

You should log in to your p3dp account and change your e-mail address, though.

[Anon]

Thanks.

Okay, back on topic. Right now for positioning the trees Im thinking of using an image. I couldnt find any info about this on the web, so if anyone has any tips/suggestions, please post it here.
Right now Im thinking of something lik this: basically Red, Blue and Green pixels mark trees on the vegetation map. The value of each determines the size of the tree. Thus there can be 3 types of trees. Yep, not very smart.

[rdb]

You have four channels; (assuming you don’t use 16-bits images) you have 4294967296 possibilities per pixel. They can surely be used more efficiently than that.

In your approach, there can be multiple trees of different types per pixel. Let’s assume that that’s impossible - there cannot be two trees at the same location.

With that in mind, you could do something like this: have the Red channel represent the type of the tree, and have the Green channel represent the size of the tree. Blue could be the rotation, and Alpha perhaps a colour tint or position randomisation so.

This would allow for 256 types of trees. That’s a lot better than 3.

[Anon]

Cool.
maybe I should make a Blender3d script which will export a png from the scene.
I think it would give more control than making one in a image editor.

[Anon]

Okay, another thing, a detail map is just a texture which is applied on top of the original terrain texture? But wont simply tiling a small detail texture over a huge terrain be a preformance issue? Or is there some other way of doing it?

EDIT: btw, anyone know how to convert Ogre .mesh files to something usable?
Wont look good without those trees

[Craig]

Okay, another thing, a detail map is just a texture which is applied on top of the original terrain texture? But wont simply tiling a small detail texture over a huge terrain be a preformance issue? Or is there some other way of doing it?

Not an issue. The texture sampling time does not depend on texture scale (except for a bit of cache performance). Having a highly tiled texture adds a multiply to the UV before sampling, and thats it. Sampling is done per pixel on screen, not per pixel on the texture. Adding a detail texture does add one more texture sample per pixel per frame, but thats not too bad.

[Anon]

OK then.
I still cant find a way to convert the .meshes to something else. What should I do?

[tah]

Hi

About converting Ogre mesh files to egg.
There is a addon for blender called “blenderimport” located in the ogre3d web site addons page:

http://www.ogre3d.org/developers/addons

This adds an Ogre import to the list of importers in the Blender import drop down menu.

Then you can use the Chicken addon to export the egg file.

http://chicken-export.sourceforge.net/

I’ve used it on several .mesh files; I use to play with Ogre3d some months ago before I got started on Panda3d…

tim

[Anon]

Yeah Ive read about it, but that site sucks. The page you linked just has the description of that importer. Theres no link, which is weird, Ive also found this page: ogre3d.org/tikiwiki/BlenderI … ture=Tools but again I dont see any download

And the svn link is dead: ogreaddons.svn.sourceforge.net/v … _import.py

[tah]

try this
it worked for me:

https://ogreaddons.svn.sourceforge.net/svnroot/ogreaddons/trunk/blenderimport/

it’s on the site under the title:
How to obtain the add-ons
and it’s really small print, for what ever reason ???