Parallel Split Shadow Mapping using PSSMCameraRig

Code Snippets
1

Here is a small example on how to use the PSSMCameraRig class found in Panda3D rplight contrib. It is heavily based on the shader terrain sample and the PSSM plugin of RenderPipeline by @tobspr.

With that, you can easily add nice looking shadows in your outdoor scene :slight_smile:

pssm
pssm1596Ă—1200 519 KB

You can use the mouse to move around and see how the PSSM reacts with the camera movements. There is also a couple of shortcuts available :

Shortcuts :

  • s : Switch between traditional shadow mapping and PSSM
  • f : Freeze PSSM update
  • g : Toggle fog
  • F3 : Toggle wireframe mode
  • F5 : Toggle buffer viewer
  • Escape : Exit the sample

Note that only the basic features of PSSCameraRig are demonstrated, there are many other useful features available.

6 Likes
2

Thanks so much for bringing out this snippet! Very useful!

One question: the variable “pssm_nearfar[]” seems not be used in the terrain fragment shader: is there any reason?

thanks!

3

The pssm_nearfar array contains the distance of the near and far planes of each split frustum. It’s not really useful for basic shadowing, I guess it could be needed for more advanced techniques (though even RP does not use it).

4

I tried this by adding this code in my application, however, the program crashes during the updating of the camera_rig:

light_dir = self.directionalLightNP.get_mat().xform(-self.directionalLight.get_direction()).xyz
self.camera_rig.update(base.camera, light_dir)

AssertionError: max_distance <= 1.0 at line 272 of c:\buildslave\sdk-windows-amd64\build\contrib\src\rplight\pssmCameraRig.cxx

I cannot solve that without more knowledge of how this all is implemented, for example, what max_distance <= 1.0 is this referring to?

5

It appears that the PSSM distance needs to be greater than the camera far plane. Either lower the far distance on the camera lens or increase the PSSM distance.

6

far distance on the camera lens is 750, pssm distance on the camera_rig is 2048 and sun_distance in the camera_rig is 1024. But even if I reduce the far distance on the camera to 10 or if I increase the PSSM distance, the same exception still occurs.

7

Actually it is the opposite, the max pssm distance must be shorter than (or equal to) the far plane of the camera. As PSSM splits up the camera frustum into sections, no section can overlap or be farther than the far plane of the camera frustum, hence the ratio must be lower or equal to 1.0.

The sun distante on the other hand controls the size of the shadow frustums.

8

Thank you. Now it does not give an exception anymore. However, I still do not get PSSM shadows in my application, so I probably did something wrong. I integrated your code in the Roaming Ralph example to see how it works in a simple application, but I also could not get it to work in Roaming Ralph, see this:

#!/usr/bin/env python

# Author: Ryan Myers
# Models: Jeff Styers, Reagan Heller
#
# Last Updated: 2015-03-13
#
# This tutorial provides an example of creating a character
# and having it walk around on uneven terrain, as well
# as implementing a fully rotatable camera.

from direct.showbase.ShowBase import ShowBase
from panda3d.core import CollisionTraverser, CollisionNode
from panda3d.core import CollisionHandlerQueue, CollisionRay
from panda3d.core import Filename, AmbientLight, DirectionalLight
from panda3d.core import PandaNode, NodePath, Camera, TextNode
from panda3d.core import CollideMask
from panda3d.core import SamplerState, Texture
from panda3d.core import WindowProperties, FrameBufferProperties, GraphicsPipe, GraphicsOutput

from panda3d._rplight import PSSMCameraRig

from direct.gui.OnscreenText import OnscreenText
from direct.actor.Actor import Actor
import random
import sys
import os
import math

# Function to put instructions on the screen.
def addInstructions(pos, msg):
    return OnscreenText(text=msg, style=1, fg=(1, 1, 1, 1), scale=.05,
                        shadow=(0, 0, 0, 1), parent=base.a2dTopLeft,
                        pos=(0.08, -pos - 0.04), align=TextNode.ALeft)

# Function to put title on the screen.
def addTitle(text):
    return OnscreenText(text=text, style=1, fg=(1, 1, 1, 1), scale=.07,
                        parent=base.a2dBottomRight, align=TextNode.ARight,
                        pos=(-0.1, 0.09), shadow=(0, 0, 0, 1))


class RoamingRalphDemo(ShowBase):
    def __init__(self):
        # Set up the window, camera, etc.
        ShowBase.__init__(self)

        # Set the background color to black
        self.win.setClearColor((0, 0, 0, 1))

        # This is used to store which keys are currently pressed.
        self.keyMap = {
            "left": 0, "right": 0, "forward": 0, "cam-left": 0, "cam-right": 0}

        # Post the instructions
        self.title = addTitle(
            "Panda3D Tutorial: Roaming Ralph (Walking on Uneven Terrain)")
        self.inst1 = addInstructions(0.06, "[ESC]: Quit")
        self.inst2 = addInstructions(0.12, "[Left Arrow]: Rotate Ralph Left")
        self.inst3 = addInstructions(0.18, "[Right Arrow]: Rotate Ralph Right")
        self.inst4 = addInstructions(0.24, "[Up Arrow]: Run Ralph Forward")
        self.inst6 = addInstructions(0.30, "[A]: Rotate Camera Left")
        self.inst7 = addInstructions(0.36, "[S]: Rotate Camera Right")

        # Set up the environment
        #
        # This environment model contains collision meshes.  If you look
        # in the egg file, you will see the following:
        #
        #    <Collide> { Polyset keep descend }
        #
        # This tag causes the following mesh to be converted to a collision
        # mesh -- a mesh which is optimized for collision, not rendering.
        # It also keeps the original mesh, so there are now two copies ---
        # one optimized for rendering, one for collisions.

        self.environ = loader.loadModel("models/world")
        self.environ.reparentTo(render)

        # Create the main character, Ralph

        ralphStartPos = self.environ.find("**/start_point").getPos()
        self.ralph = Actor("models/ralph",
                           {"run": "models/ralph-run",
                            "walk": "models/ralph-walk"})
        self.ralph.reparentTo(render)
        self.ralph.setScale(.2)
        self.ralph.setPos(ralphStartPos + (0, 0, 0.5))

        # Create a floater object, which floats 2 units above ralph.  We
        # use this as a target for the camera to look at.

        self.floater = NodePath(PandaNode("floater"))
        self.floater.reparentTo(self.ralph)
        self.floater.setZ(2.0)

        # Accept the control keys for movement and rotation

        self.accept("escape", sys.exit)
        self.accept("arrow_left", self.setKey, ["left", True])
        self.accept("arrow_right", self.setKey, ["right", True])
        self.accept("arrow_up", self.setKey, ["forward", True])
        self.accept("a", self.setKey, ["cam-left", True])
        self.accept("s", self.setKey, ["cam-right", True])
        self.accept("arrow_left-up", self.setKey, ["left", False])
        self.accept("arrow_right-up", self.setKey, ["right", False])
        self.accept("arrow_up-up", self.setKey, ["forward", False])
        self.accept("a-up", self.setKey, ["cam-left", False])
        self.accept("s-up", self.setKey, ["cam-right", False])

        taskMgr.add(self.move, "moveTask")

        # Game state variables
        self.isMoving = False

        # Set up the camera
        self.disableMouse()
        self.camera.setPos(self.ralph.getX(), self.ralph.getY() + 10, 2)

        # We will detect the height of the terrain by creating a collision
        # ray and casting it downward toward the terrain.  One ray will
        # start above ralph's head, and the other will start above the camera.
        # A ray may hit the terrain, or it may hit a rock or a tree.  If it
        # hits the terrain, we can detect the height.  If it hits anything
        # else, we rule that the move is illegal.
        self.cTrav = CollisionTraverser()

        self.ralphGroundRay = CollisionRay()
        self.ralphGroundRay.setOrigin(0, 0, 9)
        self.ralphGroundRay.setDirection(0, 0, -1)
        self.ralphGroundCol = CollisionNode('ralphRay')
        self.ralphGroundCol.addSolid(self.ralphGroundRay)
        self.ralphGroundCol.setFromCollideMask(CollideMask.bit(0))
        self.ralphGroundCol.setIntoCollideMask(CollideMask.allOff())
        self.ralphGroundColNp = self.ralph.attachNewNode(self.ralphGroundCol)
        self.ralphGroundHandler = CollisionHandlerQueue()
        self.cTrav.addCollider(self.ralphGroundColNp, self.ralphGroundHandler)

        self.camGroundRay = CollisionRay()
        self.camGroundRay.setOrigin(0, 0, 9)
        self.camGroundRay.setDirection(0, 0, -1)
        self.camGroundCol = CollisionNode('camRay')
        self.camGroundCol.addSolid(self.camGroundRay)
        self.camGroundCol.setFromCollideMask(CollideMask.bit(0))
        self.camGroundCol.setIntoCollideMask(CollideMask.allOff())
        self.camGroundColNp = self.camera.attachNewNode(self.camGroundCol)
        self.camGroundHandler = CollisionHandlerQueue()
        self.cTrav.addCollider(self.camGroundColNp, self.camGroundHandler)

        # Uncomment this line to see the collision rays
        #self.ralphGroundColNp.show()
        #self.camGroundColNp.show()

        # Uncomment this line to show a visual representation of the
        # collisions occuring
        #self.cTrav.showCollisions(render)

        # Create some lighting
        ambientLight = AmbientLight("ambientLight")
        ambientLight.setColor((.3, .3, .3, 1))
        self.directionalLight = DirectionalLight("directionalLight")
        self.directionalLight.setDirection((-5, -5, -5))
        self.directionalLight.setColor((1, 1, 1, 1))
        self.directionalLight.setSpecularColor((1, 1, 1, 1))
        # Configure the directional light to cast shadows
        self.directionalLight.set_shadow_caster(True, 1024, 1024)
        self.directionalLight.get_lens().set_near_far(0, 1024)
        self.directionalLight.get_lens().set_film_size(1024, 1024)

        render.setLight(render.attachNewNode(ambientLight))
        self.directionalLightNP = render.attachNewNode(self.directionalLight)
        render.setLight(self.directionalLightNP)

        # Basic PSSM configuration
        self.camera_rig = None
        self.split_regions = []
        self.num_splits = 5
        self.split_resolution = 1024
        self.border_bias = 0.058
        self.fixed_bias = 0.5
        self.use_pssm = True
        self.freeze_pssm = False
        self.fog = True
        self.last_cache_reset = globalClock.get_frame_time()
        # Create the PSSM
        self.create_pssm_camera_rig()
        self.create_pssm_buffer()
        self.attach_pssm_camera_rig()
        self.set_shader_inputs(self.environ)

    # Records the state of the arrow keys
    def setKey(self, key, value):
        self.keyMap[key] = value

    # Accepts arrow keys to move either the player or the menu cursor,
    # Also deals with grid checking and collision detection
    def move(self, task):

        # Get the time that elapsed since last frame.  We multiply this with
        # the desired speed in order to find out with which distance to move
        # in order to achieve that desired speed.
        dt = globalClock.getDt()

        # If the camera-left key is pressed, move camera left.
        # If the camera-right key is pressed, move camera right.

        if self.keyMap["cam-left"]:
            self.camera.setX(self.camera, -20 * dt)
        if self.keyMap["cam-right"]:
            self.camera.setX(self.camera, +20 * dt)

        # save ralph's initial position so that we can restore it,
        # in case he falls off the map or runs into something.

        startpos = self.ralph.getPos()

        # If a move-key is pressed, move ralph in the specified direction.

        if self.keyMap["left"]:
            self.ralph.setH(self.ralph.getH() + 300 * dt)
        if self.keyMap["right"]:
            self.ralph.setH(self.ralph.getH() - 300 * dt)
        if self.keyMap["forward"]:
            self.ralph.setY(self.ralph, -25 * dt)

        # If ralph is moving, loop the run animation.
        # If he is standing still, stop the animation.

        if self.keyMap["forward"] or self.keyMap["left"] or self.keyMap["right"]:
            if self.isMoving is False:
                self.ralph.loop("run")
                self.isMoving = True
        else:
            if self.isMoving:
                self.ralph.stop()
                self.ralph.pose("walk", 5)
                self.isMoving = False

        # If the camera is too far from ralph, move it closer.
        # If the camera is too close to ralph, move it farther.

        camvec = self.ralph.getPos() - self.camera.getPos()
        camvec.setZ(0)
        camdist = camvec.length()
        camvec.normalize()
        if camdist > 10.0:
            self.camera.setPos(self.camera.getPos() + camvec * (camdist - 10))
            camdist = 10.0
        if camdist < 5.0:
            self.camera.setPos(self.camera.getPos() - camvec * (5 - camdist))
            camdist = 5.0

        # Normally, we would have to call traverse() to check for collisions.
        # However, the class ShowBase that we inherit from has a task to do
        # this for us, if we assign a CollisionTraverser to self.cTrav.
        #self.cTrav.traverse(render)

        # Adjust ralph's Z coordinate.  If ralph's ray hit terrain,
        # update his Z. If it hit anything else, or didn't hit anything, put
        # him back where he was last frame.

        entries = list(self.ralphGroundHandler.getEntries())
        entries.sort(key=lambda x: x.getSurfacePoint(render).getZ())

        if len(entries) > 0 and entries[0].getIntoNode().getName() == "terrain":
            self.ralph.setZ(entries[0].getSurfacePoint(render).getZ())
        else:
            self.ralph.setPos(startpos)

        # Keep the camera at one foot above the terrain,
        # or two feet above ralph, whichever is greater.

        entries = list(self.camGroundHandler.getEntries())
        entries.sort(key=lambda x: x.getSurfacePoint(render).getZ())

        if len(entries) > 0 and entries[0].getIntoNode().getName() == "terrain":
            self.camera.setZ(entries[0].getSurfacePoint(render).getZ() + 1.0)
        if self.camera.getZ() < self.ralph.getZ() + 2.0:
            self.camera.setZ(self.ralph.getZ() + 2.0)

        # The camera should look in ralph's direction,
        # but it should also try to stay horizontal, so look at
        # a floater which hovers above ralph's head.
        self.camera.lookAt(self.floater)
        # Update the camera position and the light direction
        light_dir = self.directionalLightNP.get_mat().xform(-self.directionalLight.get_direction()).xyz
        self.camera_rig.update(self.camera, light_dir)
        cache_diff = globalClock.get_frame_time() - self.last_cache_reset
        if cache_diff > 5.0:
            self.last_cache_reset = globalClock.get_frame_time()
            self.camera_rig.reset_film_size_cache()

        return task.cont

    def create_pssm_camera_rig(self):
        # Construct the actual PSSM rig
        self.camera_rig = PSSMCameraRig(self.num_splits)
        # Set the max distance from the camera where shadows are rendered
        self.camera_rig.set_pssm_distance(2048)
        # Set the distance between the far plane of the frustum and the sun, objects farther do not cas shadows
        self.camera_rig.set_sun_distance(1024)
        # Set the logarithmic factor that defines the splits
        self.camera_rig.set_logarithmic_factor(2.4)
        
        self.camera_rig.set_border_bias(self.border_bias)
        # Enable CSM splits snapping to avoid shadows flickering when moving
        self.camera_rig.set_use_stable_csm(True)
        # Keep the film size roughly constant to avoid flickering when moving
        self.camera_rig.set_use_fixed_film_size(True)
        # Set the resolution of each split shadow map
        self.camera_rig.set_resolution(self.split_resolution)
        self.camera_rig.reparent_to(self.render)

    def create_pssm_buffer(self):
        # Create the depth buffer
        # The depth buffer is the concatenation of num_splits shadow maps
        self.depth_tex = Texture("PSSMShadowMap")
        self.buffer = self.create_render_buffer(
            self.split_resolution * self.num_splits, self.split_resolution,
            32,
            self.depth_tex)

        # Remove all unused display regions
        self.buffer.remove_all_display_regions()
        self.buffer.get_display_region(0).set_active(False)
        self.buffer.disable_clears()

        # Set a clear on the buffer instead on all regions
        self.buffer.set_clear_depth(1)
        self.buffer.set_clear_depth_active(True)

        # Prepare the display regions, one for each split
        for i in range(self.num_splits):
            region = self.buffer.make_display_region(
                i / self.num_splits,
                i / self.num_splits + 1 / self.num_splits, 0, 1)
            region.set_sort(25 + i)
            # Clears are done on the buffer
            region.disable_clears()
            region.set_active(True)
            self.split_regions.append(region)

    def attach_pssm_camera_rig(self):
        # Attach the cameras to the shadow stage
        for i in range(5):
            camera_np = self.camera_rig.get_camera(i)
            camera_np.node().set_scene(self.render)
            self.split_regions[i].set_camera(camera_np)

    def set_shader_inputs(self, target):
        # Configure the parameters for the PSSM Shader
        target.set_shader_inputs(PSSMShadowAtlas=self.depth_tex,
                                 pssm_mvps=self.camera_rig.get_mvp_array(),
                                 pssm_nearfar=self.camera_rig.get_nearfar_array(),
                                 border_bias=self.border_bias,
                                 fixed_bias=self.fixed_bias,
                                 use_pssm=self.use_pssm,
                                 fog=self.fog)

    def create_render_buffer(self, size_x, size_y, depth_bits, depth_tex):
        # Boilerplate code to create a render buffer producing only a depth texture
        window_props = WindowProperties.size(size_x, size_y)
        buffer_props = FrameBufferProperties()

        buffer_props.set_rgba_bits(0, 0, 0, 0)
        buffer_props.set_accum_bits(0)
        buffer_props.set_stencil_bits(0)
        buffer_props.set_back_buffers(0)
        buffer_props.set_coverage_samples(0)
        buffer_props.set_depth_bits(depth_bits)

        if depth_bits == 32:
            buffer_props.set_float_depth(True)

        buffer_props.set_force_hardware(True)
        buffer_props.set_multisamples(0)
        buffer_props.set_srgb_color(False)
        buffer_props.set_stereo(False)
        buffer_props.set_stencil_bits(0)

        buffer = self.graphics_engine.make_output(
            self.win.get_pipe(), "pssm_buffer", 1,
            buffer_props, window_props, GraphicsPipe.BF_refuse_window,
            self.win.gsg, self.win)

        if buffer is None:
            print("Failed to create buffer")
            return

        buffer.add_render_texture(
            self.depth_tex, GraphicsOutput.RTM_bind_or_copy,
            GraphicsOutput.RTP_depth)

        buffer.set_sort(-1000)
        buffer.disable_clears()
        buffer.get_display_region(0).disable_clears()
        buffer.get_overlay_display_region().disable_clears()
        buffer.get_overlay_display_region().set_active(False)

        return buffer

demo = RoamingRalphDemo()
demo.run()
9

I’m afraid PSSMCameraRig requires that you use your own custom shader, it can not work with the default shader generator (nor the fixed pipeline).

So you need to create and assign on each object in your scene that could be shadowed by the PSSM a shader that will take care of the colour, lighting and shadowing of your object. In my example I implemented a basic shader which supports Lambert diffusion and multiple lights. But there are many examples available around that are much more complete.

10

Okay I see. But this example only works with a ShaderTerrainMesh because the shaders require that. I am using a different type of environment a bam file with other 3D objects added to it, so I will probably have to look for other shaders.

11

Thanks! Then we can safely remove it.

On an other side, it seems that using:

threading-model Cull/Draw

with PSSM makes the shadows flicker as soon as the camera moves.

Is it only on my (old GPU) or may it be an issue with the threading model?

12

Indeed I see the same issue on when I activate the multithreaded render pipeline.

As it happens only when you mode the camera, I would guess this is perhaps due to an inconsistency between the MVPs matrices set as shader inputs in the App thread and the internal calculation of PSSMCameraRig, I assume done in the Draw thread (otherwise running App and Cull in the same thread would not show the flickering)
.

13

That’s indeed the problem, if I add the following code in the update task of the sample, the flickering disappear :

        src_mvp_array = self.camera_rig.get_mvp_array()
        mvp_array = PTA_LMatrix4()
        for array in src_mvp_array:
            mvp_array.push_back(array)
        self.terrain.set_shader_inputs(pssm_mvps=mvp_array)

(There’s probably a better way to copy a PTA though)

A better fix would be that PSSMCameraRig is aware of the threaded pipeline and return the correct map array for the rendered frame, but I don’t know enough about the internals of Panda3D to do that :slight_smile:

14

The issue is that PTA updates aren’t pipeline-cycled. This means that updates to a PTA may already affect the previous frame that is still being rendered, making it possible for updates to shader inputs to appear out of chronological order. However, PTAs are the fastest way (aside from textures, etc.) to get generic data to shader uniforms right now. Cycling between multiple PTAs (or creating a new one for every frame) would be the way to fix this.

We ought to have a new class that’s as fast as PTA but handles pipeline cycling automatically.

15

I see, I thought the object owning the data ought to be responsible to manage the data across the different stage of the pipeline, it’s much easier if it’s the data itself. I’ll add a feature request in the GitHub tracker.

16

Here is the feature request : Add pipeline cycled variant of PTA · Issue #1149 · panda3d/panda3d · GitHub

1 Like
17

@eldee: Thanks for the workaround. This works as expected with the multi-threaded pipeline.
The PSSM algo provides:

  • a real visual quality improvement for the dir light shadows (even with filtering methods)…
  • …without requiring (very) large shadowmap texture!

@rdb: thanks, and good to know for the PTAs, I was not aware there were not pipeline-cycled.

One small suggestion if I may (please tell me if that would make sense):

pssmCameraRig::update() requires to have a camera built on the same way as the base.camera from Showbase (see pssmCameraRig.cxx l. 372)

Camera* cam = DCAST(Camera, cam_node.get_child(0).node());

When you use a camera NP alone directly parented to render (as the one generated by a call to base.make_camera()), there is an assert error (which is normal considering the test done just after the DCAST() to Camera). A possible workaround is 1/ to update base.camera with the transform/lens of the current camera and 2/ pass base.camera as the PSSM camera in update()

However, would it possible/suitable to add a test prior to ensure we have a proper NP that could be downcasted to a camera but without using a prior call to get_child(0)?

18

Yes, that is quite fragile code. I think we could instead check whether the node is a Camera, and if not, look for a child that is a Camera. In the long run I think people ought to really pass the node that is a Camera, not a parent node. I’ll make that change.

19

Thanks!