Infinite Procedural Terrain Engine

Heh… well the water shader and skybox come from the YARR demo mentioned in my first post, and the textures mostly came from google image search. The shader is mine though, and the math I used to generate realistic heightmaps is worth at least looking at.

Anyways, I’m definitely liking the improvements I see in your screenshots. I can hardly wait to see what it looks like with prettier textures and shader.

I’ve updated the shader in my project to process two different sets of height and slope boundries for the same rock texture. The code got uglier, but the results got prettier. Obviously my shader and its python counterpart need more flexibility so that this kind of thing doesn’t require hackish code. I’ll do a rewrite soon.
Edit: I’ve realized that the shader’s inability to store global variables means there is no way to make the shader more flexible without making it do possibly needless calculations on every pixel. The only good solution would be a shader generator like yours. Something much simpler would suffice for the purposes of customizing my terrain shader though.

There is now a simple shader generator to gracefully handle a variable number of textures and a variable number of slope/altitude boundaries per texture. This obviates the need to modify multiple lines in both the shader and python side to achieve such changes.

Three significant updates.

  1. Performance improved greatly.

  2. My terrain shader got a major overhaul as I finally noticed proper documentation here. I had previously assumed input in the vertex shader came from the semantics, but it turns out they’re totally irrelevant and only the variables’ names seem to matter in practice. :unamused: Anyways, I’ve combined hdr per-pixel lighting and bloom with my terrain shader, despite my lack of shader expertise. This still works with my shader generator

To do: Make these new effects configurable by the generator. Also I would like to switch my detail texture from an ugly blend to a normal map.

  1. There is a simple GUI control in the demo that can be used to alter the terrain shader parameters on the fly. Of course, these shifts in textures can be achieved programmatically as well and smoothly transition a landscape to a totally different climatic appearance.

To do: Expand the gui and engine to allow new textures and regions to be added during run-time.

Just for fun… some screen shots of the latter feature at work.

http://stephenlujan.com/panda3d/terrain/menu1.jpg

http://stephenlujan.com/panda3d/terrain/menu2.jpg

http://stephenlujan.com/panda3d/terrain/menu3.jpg

Oh yeah, and here’s a Ralph on a mountain I liked. Forgive the jpeg artifacts/blurriness. I didn’t want this thread to drain everyone’s bandwidth. :wink:

http://stephenlujan.com/panda3d/terrain/mountain1.jpg

I’ve had a couple dozen commits since my last post here. It was mostly minor improvements and organization, but a few obvious differences are:

  • All shader issues are now resolved.
  • Normal mapping is working beautifully.
  • The time of day changes.
  • There is a new sky made to be less static.
  • A Ralph clone follows you around.
  • Some loading screen feedback appears.

Check out the browser demo, or a couple of screen shots.

Sunset
Valley

The browser demo appears to be built against Panda3D 1.8, which isn’t released yet, and doesn’t run everywhere. It doesn’t run on my Mac, for instance.

You should build p3d files meant for public consumption only against the standard Panda3D releases.

David

Now those terrain pictures look awesome!
Keep up the good work.

Sorry, I forgot to mention I’ve been building with 1.8.

My shader fails to work using either of these.
runtime.panda3d.org/packp3d.p3d
runtime-dev.panda3d.org/packp3d.p3d

It only works properly with this one.
runtime-dev.panda3d.org/packp3d_dev.p3d

Oh, thanks Nemesis!

I’ve added exponential fogging to the shader and optimized it by removing all conditional statements. The terrain engine has almost all of the features I have intended for it at this point. I’ve finally made a proper web page for it here. You can go there for more details, screen shots, videos, and a browser demo.

Tried the latest ver crashes right after the loading card Idle spews a lot of stuff about invalid shader params. The previous one which had a gui and would allow adjustment of the shader heights and slopes wouldn’t run right either, stayed in monotexture shader, with NO terrain lighting, and with no error messages about not being able to load the slope texturing shaders.
I also cannot run Craig Macomber’s terrain demo with the geo clip grass enabled, I get NO terrain whatsoever. I have a two year old HP DV7 1200 series with DX11 but an ATI Radeon HD 3200 GPU which seems to have a serious “mental handicap” and also seems to only want to run in single precision mode.
This is the second time I’ve posted in a long time … after being roundly castigated by a bunch of “Ivy League” “professional developers” over my favorite choice of text colors ( 1.0, 0.0, 1.0, 1.0 ) over black and my stunning lack of professionalism… what I’m trying to do isn’t about money to me its about supporting creativity which moneygrubbing VR sites like Second Life care little about.
Your slope based texturing system was of great interest to me as was Craig’s infinite terrain system. Also having problems as my app needs functional collision detection which only seems to work with EGG based terrains and objects not geomip or procedural plants created on the fly.
At this point I’m about to chuck my whole project and this “brain dead” computer in the dumpster, don’t tell me to just buy a newer one theres NO funds for that. Ready to throw in the towel and crawl back into my foxhole.

Had an instance of Alice beach house as a background piece to take snapshots in front of so it looked like I was farther along with meshes ( I kno cheating ). Went back and opened the EGG with TextPad, guess what I discover… the magic incantation { Polyset keep descend } is not present. added that by cut and paste fronm the level file in Treeform’s FPS demo and BINGO!!! We have contact. height setting code I ripped off from original Roaming Ralph still not working like it should but I’m showing collisions now at least with this one and only test object. Panda manual talks like the magick spell is not really needed but only to make collisions more efficient. BUT this appears to be an inaccuracy as IT seems to be REQUIRED. And NONE of the Alice EGGs I’ve looked at so far seem to have it but its easy enough to cut and paste. I really liked Craig’s procedural trees but I will probably have to go back to my original concept of randomly placing EGGs for the trees and such. Still wish I could collide the terrains tho as editor would be so much cooler and more like what folks would expect it to do. Now I just wish I could figure how to do something like your slope based texturing shaders that could run on this machine. My code at this point is more Craig’s than yours but I’d really like to make my hills rocky instead of grassy and be able to add a few more height levels. Craig’s shader chain I still don’t really understand well enough even to add in a shoreline texture. Actually at this point I wish I knew why your slope based shaders wont’run on this ATI HD3200 and why Craig’s GEOCLIP won’t either.

I’m afraid I’ve been using a 1.8 beta build of panda3d. I had some problems with 1.7x versions with my shaders. Try the online demo I listed on my website. My card is an ATI HD 3870 so its the same generation as yours.

My terrain is infinite and persistant as well. It is for the moment 100% procedural, though I have plans to allow height map editing later. Craig's system is supposed to be able to use modular tile bakeries and renderers interchangeably, but at the time I had a very hard time figuring out how to plug my system into it. 

As for terrain collisions I haven’t really put any thought into it yet, but its something I would like to figure out as well. It can’t be that hard to feed the mesh into a physics engine… I hope.
By the way Craig’s geoclip engine is I believe still unfinished, but you might want to ask him yourself.

They’re right its a bit unstable, and the new ability to split the threads DOES NOT work 4 me at all… just terminates itself at startup.

But now I successfully can run the earlier version of your slope shaders now, will be trying to integrate your renderer into Craig’s bakeries as I really like the way his code creates those mountains, Yes I decided NOT to trash over a year of study and effort quite yet, still cannot get ALL visible geometry respected by Panda as collideable.

I had fun before, trying to get Craig’s trees to accept alpha on leaf textures had to dig way down inside that code to find nice place to manually create the needed transparency attribute, suspect this will involve a similar “hands and knees crawl thru the ductwork”.

Wish I could fix this ATI so 64 bits for coordinates isn’t a total waste of half the space. I have supposedly recent drivers and DX 11 so I dunno, also wishing there was a better way to debug shaders as Panda’s “so called Graceful” behavior of not even giving a meaningful error message about why a shader fails is just awful, how does anybody even debug them??

I had decided to work on heightmap editing and adding database chunks for mesh placement and editor which is how I ended up in the debacle of how to make terrain collidable. I could restrict meshes to only EGG’s with the “Magick Tags” in them ( aka the polyset keep descend stuff ) but I also have need for procedurals for roads, rivers, railroads, etc.

What I had been planning was to associate a SQL Lite DB to each tile holding the meshes and edited terrain maps for a Sim City 4 style editor, that and click new location style avatar control both need clickable terrains to work otherwise stuck with a crummy outdated Alice99 style editor that just throws the mesh on the ground in front of you and you must move it manually with something akin to the old Dtools “placer panel”. I’d like for users to just be able to say select a pine tree and click click click the terrain where they want multiple instances, they can always go back for scales and rotates or I’d like to have a “randomize” option for multiple placements

Will re DL the newer ver to see if that now runs as well, wish there was better tech docs on ATI chips somewhere ( gamer’s sites say its braindead buy a new one, and ATI does the same extolling virtues of new cards but providing no useful tech docs on old P O S chip I can find anywhere. The 64 bit precision issue was big on these forums over a year ago, never found out if there was any resolution tho.

You and Craig bout the only ones not buying into the “ivy leaguer’s” rag on this guy till he gives up thing.

New ver of your terrain works well.

Also DRWR posted some useful info on how Panda really works internally re: 32 and 64 bits. Seems there aren’t really ANY true 64 bit GPU’s and Panda cuts things down to 32 bits internally, this answered my questions regarding wasted space, seems Panda does not store vertices and other data sent to the GPU as 64 bits and that length is only used by python for intermediate calculations. The whole 32/64 problem was just the FPU getting mis-set by the graphics driver causing precision problems in Python, first really good explanation I’ve heard. Still, I cannot run geometry shaders like Craig’s GEOCLIP grass on this thing but I CAN do my grass clumps like Demomaster does in his grass demo 3.

Your terrain baking and texturing system is a LOT easier to understand than Craig’s and will make what I’m trying to implement simpler, with multiple height level texturing and slope texturing that will allow me to do cliff faces and a nice shoreline for the waterplane. I had almost turned Craig’s code literally inside out so as to get a splash screen and GUI up first, I’ve got a somewhat stable three viewpoint orbitable camera and an avatar with overhead text labels. As soon as I rework and objectify my X input joystick and GUI code, I’m going to try integrating it into your framework and start the process of adding the SQL databases for mesh placements.

P.s. Do you know if anyone has a version of Psyco for Python 2.7 available, I use it heavily and miss it now that I’ve upgraded to Panda 1.8

Best regards from the “dirty old fox”.

Hey kitsune, I’m very glad you were able to get my terrain engine working for you. If you don’t like the way my engine generates mountains, it should be very easy to alter. I’ve separated out a “HeightMap” class that is soley resonsible for generating that geometry. There are a bunch of variables you can manipulate, or it should be easy to plug your own code in there too.

I’ve been using Craig’s code so I’ve gotten familiar with it, but I personally think yours will work better for me… and its easier to understand and add in the tile cacheing and SQL Lite database mesh placement I need. Also I like the height and slope based texturing as it has more regions and is a lot easier to understand than Craig’s approach.

Now I just wish I could find a version of Psyco that works with the Python 2.7 underlying Panda 1.8 which I’m now using exclusively. I hope to have version 0.4 up on Google code sooon so youse guys can see what I’ve been up to. I haven’t tried your demo code with the new multithreading but I have with Craig’s code and it does not work like that as I think app/cull and draw end up fighting over posession of the GPU and it ALWAYS crashes befroe even drawing anything. Gotta try with your code next as being able to use both of my cores could give me a speedup even without Psyco.

Just tried yours with two threads… and IT WORKS, and it is faster. YIP! YIP!

Best regards Raven Gengoro Foxfire-Ninetails

Craig has posted new terrain code that CAN create collision geometry for Geomip. Looking forward to getting my code, his new collideable ground and your shaders all working together.

Then I will put what I’ve got up somewhere. I currently have a world with a tracking waterplane two moons a sun and a two layer cloudy sky with nighttime stars and a multi viewpoint camera, I’ve replaced Ralph with Demomaster’s Fleur and she swims in the water, considering replacing some elements of my sky with yours as they look far better.

YIP! YIP!

Sounds like you have made great progress lately. I can’t wait to see what you’ve done.

For my clouds I just found the highest res tileable image of clouds I could, converted it to a grayscale, then I use it as an alpha map and color it the same value as the sun light. Its alright but there’s really no depth to them. I’m thinking about hacking up a way to approximate directional lighting even though its just a 2d image.

The sun is just a tweaked clone of the one from Naith.

My clouds are basically taken from the day night cycle demo but I changed the texture and implemented it as two rotating domes ( at differing speeds )instead of just one, I also replaced the cheezy night stars with a Hubble image also replaced the mopath sun with the same sort of “orrery” like spinners most folks use for their Dlights as the mopaths didn’t scale properly when I resized the skydomes. My clouds still have seaming problems which is one reason I was looking at yours, I also spun up two moons, I’m still not done fixing my multiviewpoint camera or the joystick code on this one and now am seriously considering dropping the whole framework based on Craig’s terrain demo in favor of one based on yours but adding back in MY move task and cam code plus Craig’s new version of the ground factory as I want something like that but running your renderer.Syill I probly should post "ol 0.3 just so u can see how awfull and un OO my Python is. LOL Also I have my waterplane rigged to track the avatar around and have taken many long walks out to the point where 32 bit FP precision issues begin. But I’m working on a fix.

regards,
“the dirty old fox”

Well, this may go without saying but to fix the floating point issue with my engine I would wait until the player hits some threshold. Then just subtract the vector position of the player from everything, but add it to a variable worldPositionOffset or something. Then add that offset to the terrain’s getHeight function so terrain continues uneffected by the shift.

Thats pretty much my plan, add an offset that periodically changes. This will keep everything positioned within the 32 bit FPU sweet spot and since all object positions will be relative to the cached terrain tiles this will have the side effect of making my databases more storage efficient as well and also allow me to wrap coordinates for round or toriodal worlds without the complication of having objects crossing signed boundaries at the wraparound point.

YIP! YIP!