As gyedo’s request, I’m posting my idea for a terrain editor/generator here. I believe it could potentially be integrated into the general editor.
My work with procedural planet generation ( f-g.wikidot.com/ ) has given me some ideas for terrain design and rendering
Really there are 3 parts, specifying/designing/editing the terrain, baking the terrain into the various things needed to render, and actually getting it onscreen with proper LODs and such (The renderer).
Generally for a map, we have a height map, with some detail textures alpha mapped, maybe triplanar projected, decals, vegetation, and various placed/designed art such as buildings. Then there are also caves, overhangs, and the stuff that pisses of heigh map users to no end, and generally gets added much like the buildings. These items, their locations, and the various height maps, and detail texture masks would come from the baker, and be passed off to the renderer.
For now, I shall ignore the renderer to a major extent, but lets assume it uses some sort of large tile based height maps (maybe used via geo mip map).
My procedural system resembles this process, but has no editor. There is a baker that generates information handed off to the rendering system. Adding an editor does not have to fully overwrite this.
The editor can apply things along the lines of alpha masked height maps to areas. The alpha channel allows it to fade off to procedural terrain. This means you can stick flat spots in procedural terrain where you need them. There could also be alpha masked vegetation maps. This would allow the editor to specify areas which should be devoid of procedurally placed trees, or even bias specific types of trees (so you could make some areas creepy with dark trees or what ever). Of course, where the artists/level designers want full control, they can simply mask out all procedural content, set a custom height map and ground texture, and place all vegetation with the editor.
The procedural content would be generated by the baker by using a collection of shaders that generate maps for each of the parameters. The shaders will procedurally generate maps, then overlay any corresponding maps specified in the editor (which remember, are alpha masked). An example is a height map, but they have some specific issues I will discuss below. A common complex example would be a forest density map. The editor could specify forest density in different areas, which would over write the procedural values, or it could leave the procedural one. This map would then be fed into a shader to generate maps for the distribution of different tree types, which, like all maps, could be controlled through the editor, or not. These maps could then be used to place the corresponding tree models. This whole process could be adjusted through adding/removing and editing the various map generation shaders. When procedural content is not desired, the shaders could simply output a constant (say, 0 for no forest) which would be overwritten as needed by maps set in the editor.
The baker would run all the shaders, and reduce the whole set of maps the maps for the various terrain tiles which are actually needed for model placement and rendering.
The editor would thus need to be able to take in artist created alpha masked map images, and place, scale, rotate and specify what type of map they are (forest type, height, water, grassland, what ever).
The issue with height maps is that suppose you wish to simply make a flat spot. You would need to make a heigh map that was flat, and at the right elevation. To solve this, there could be some sort of offset tool that would allow you to offset the heigh map to the correct heigh value easily.
The system for processing all the maps in the baker would have some complexities, but it would basically be a bunch of shaders that optionally took in maps, and each produced a single map. Thus each shader would just be named after the map it makes, and include a list of the ones in needs as input. The bulk of the shaders would be fixed, and thus could be auto generated, so all they would need is a list of inputs and one expression to do math with the input values, and produce and output value, and this would be turned into a shader.
Edit: I’ve started work on the terrain bakery which should generate procedural terrain until it is hooked up to an editor of some sort. I guess I better lay out a rendering system for testing it too. When I have something interesting, I’ll upload it somewhere.