[UPDATE] - Fur - Deferred shading


Deferred shading & Fancy effects

VIDEO :
http://youtu.be/4nEV3nK7INs

[color=green]WORKING

  • deferred shading : point and directional lights in view space, phong equation, bounding volumes for lights,
  • doesn’t rely on autoshader,
  • custom g-buffers shader for normal maps,
  • FXAA : Fast Approximate Anti-Aliasing,
  • depth of field bokeh : one-pass effect based on Martins Upitis code,
  • bleach bypass : b&w merged with color to enhance contrasts and desaturate colors,
  • bloom,
  • vignetting.

[color=darkblue]TODO

  • material specular information written in g-buffer,
  • linear depth buffer,
  • try other light models,
  • working shadows,
  • sprite-based bokeh dof,
  • fxaa last version,
  • anamorphic lens flare,
  • transparency,
  • HDR tonemaping,
  • speed.

[color=red]PROBLEMS

  • projection of shadows in the light bounding volumes
  • change texture format

CODE INCOMING !

Hello all,
here a simple deferred shading system, only point lights atm, more to come.

You need panda buildbot version.

The panda app :

#####################################################
##  DEFERRED LIGHTING - POINT LIGHT IN VIEW SPACE  ##
#####################################################

### Imports ###
from panda3d.core import *
loadPrcFileData('', 'show-buffers 0')
from direct.showbase.ShowBase import ShowBase
from panda3d.egg import *
from direct.filter.FilterManager import *
from pandac.PandaModules import Vec3
from math import pi,sin,cos
import random

### Class constants ###
LIGHTS_NUMBER = 15

### Class Application ###
class Application(ShowBase):

    ### Constructor ###
    def __init__(self):
        ShowBase.__init__(self)

        self.setup_scene()

        taskMgr.add(self.update,"update")

        #disable mouse#
        base.disableMouse()

    ### Function setup ###
    def setup_scene(self):

        ### Camera setup ###
        self.light_cam = self.makeCamera(self.win)
        self.light_cam.reparentTo(self.cam)

        scene_mask = BitMask32(1)
        light_mask = BitMask32(2)

        self.cam.node().setCameraMask(scene_mask)
        self.light_cam.node().setCameraMask(light_mask)

        base.camera.setPos(0,-200,115)
        self.camera.setP(-30)

        self.cam.node().getDisplayRegion(0).setSort(1)
        self.light_cam.node().getDisplayRegion(0).setSort(2)
        self.win.setSort(3)

        self.light_cam.node().getDisplayRegion(0).setClearColor(Vec4(.0, .0, .0, 1))
        self.light_cam.node().getDisplayRegion(0).setClearColorActive(1)


        ### Scene creation ###
        self.scene = render.attachNewNode("scene")
        self.scene.hide(light_mask)
        self.scene.setShaderAuto()

        #create spheres
        for i in range(10):
            angleDegrees = i * 36
            angleRadians = angleDegrees * (pi / 180.0)

            sphere = loader.loadModel("test")
            sphere.setPos( sin(angleRadians)*40,cos(angleRadians)*40,0)
            sphere.setScale(10)
            sphere.reparentTo(self.scene)
            sphere.setAttrib(ShadeModelAttrib.make(ShadeModelAttrib.MSmooth))

        #create floor
        cm = CardMaker("floor")
        cm.setFrame(-100, 100, -100, 100)

        plane = render.attachNewNode(cm.generate())
        plane.setP(270)
        plane.setZ(-10)
        plane.reparentTo(self.scene)

        self.lights = []
        self.bulbs = []

        self.abs = []
        self.ords = []

        ### Buffers creation ###
        self.gbuffer_man = FilterManager(self.win, self.cam)

        depth = Texture()
        albedo = Texture()
        normal = Texture()

        self.gbuffer_man.renderSceneInto(colortex = albedo, depthtex = depth, auxtex = normal,auxbits = AuxBitplaneAttrib.ABOAuxNormal)

        ### Lights creation ###
        for i in range(LIGHTS_NUMBER):
            #light geometry
            light = loader.loadModel("misc/sphere")
            light.setPos(random.uniform(-40,40),random.uniform(-40,40),random.uniform(0,15))
            light.setColor(random.uniform(0,1),random.uniform(0,1),random.uniform(0,1))
            light.setScale(100)
            light.reparentTo(render)
            light.setAttrib(DepthTestAttrib.make(RenderAttrib.MLess))
            light.setAttrib(CullFaceAttrib.make(CullFaceAttrib.MCullClockwise))
            #self.light.setAttrib(CullFaceAttrib.make(CullFaceAttrib.MCullCounterClockwise)
            light.setAttrib(ColorBlendAttrib.make(ColorBlendAttrib.MAdd, ColorBlendAttrib.OOne, ColorBlendAttrib.OOne))
            light.setAttrib(DepthWriteAttrib.make(DepthWriteAttrib.MOff))

            #light shader
            light.setShader(loader.loadShader("def_light.cg"))
            light.setShaderInput("albedo", albedo)
            light.setShaderInput("depth", depth)
            light.setShaderInput("normal", normal)
            light.setShaderInput("Kd", Vec3(1,1,1))
            light.setShaderInput("Ks",Vec3(.8,.8,.8))
            light.setShaderInput("att_params",Vec3(0.1,0.2,0.003))
            light.setShaderInput("light_radius", light.getScale())
            light.setShaderInput("camera",base.camera)

            self.abs.append(light.getX())
            self.ords.append(light.getY())

            bulb = loader.loadModel("sphere")
            bulb.reparentTo(render)
            bulb.setPos(light.getPos())
            bulb.setColorScale(light.getColor())

            bulb.hide(light_mask)
            light.hide(scene_mask)

            self.bulbs.append(bulb)
            self.lights.append(light)

    ### Update per frame ###
    def update(self,task):
 
        for i in range(LIGHTS_NUMBER):
            
            self.lights[i].setZ(self.lights[i].getZ()+ sin(task.time)/20)

            self.bulbs[i].setZ(self.bulbs[i].getZ()+ sin(task.time)/20)
            angleDegrees = task.time * 6.0
            angleRadians = angleDegrees * (pi / 180.0)
            self.lights[i].setPos(self.abs[i] * sin(angleRadians), self.ords[i] * cos(angleRadians), 3)
            self.bulbs[i].setPos(self.abs[i] * sin(angleRadians), self.ords[i] * cos(angleRadians), 3)

        return task.cont


application = Application()
application.run()

The point light shader :

//Cg

void vshader(
             float4 vtx_position : POSITION,
             out float4 l_position : POSITION,
             out float4 l_pos : TEXCOORD0,
             uniform float4x4 mat_modelproj)
{
    l_position = mul(mat_modelproj, vtx_position); 
    l_pos = l_position;
   
}

void fshader(
             float4 l_position : TEXCOORD0,
             float4 l_normal    : TEXCOORD1,
             uniform sampler2D k_albedo : TEXUNIT0, // color map
             uniform sampler2D k_depth : TEXUNIT1, // depth map
             uniform sampler2D k_normal : TEXUNIT2, // normal map
             uniform float4 texpad_albedo,
             uniform float4 texpad_normal,
             uniform float4 texpad_depth,
             uniform float4 attr_color,
             uniform float3 att_params, // attenuation parameters
             uniform float3 light_radius,
             uniform float4 vspos_camera, // position of camera in view space
             uniform float3 Kd, // diffuse coef.
             uniform float3 Ks, // specular coef.
             uniform float4 vspos_model, // position of model in view space
             uniform float4x4 trans_clip_to_view,
             out float4 o_color0 : COLOR0)
{
    ////////////////////////////////////////////
    //   DEFERRED POINT LIGHT IN VIEW SPACE   //
    ////////////////////////////////////////////

    // fragment position in clip space
    l_position.xy /= l_position.w; 

    // textures input
    float2 texcoords = float2(l_position.xy) * texpad_normal.xy + texpad_normal.xy;
    float4 albedo = tex2D(k_albedo, texcoords);
    float4 normal = tex2D(k_normal, texcoords);
    float4 depth = tex2D(k_depth, texcoords);

    // point illuminated
    float4 P;
    P.xy = l_position.xy;
    P.z = depth;
    P.w = 1;
    P = mul(trans_clip_to_view, P);
    P /= P.w*2;

    // normal
    float3 N;
    N = normal;
    N.xyz = N.xyz - float3(0.5,0.5,0.5);
    N.xyz = 2 * N.xyz;
    N = normalize(N);

    // light vector
    float3 L = (vspos_model - P);

    // ray length
    float l = length(L);

    // light attenuation
    float constant_att = att_params.x;
    float linear_att = att_params.y;
    float quadra_att = att_params.z;
    float attenuation = 1.0f / (constant_att + linear_att*l + quadra_att*l*l);

    // normalize light vector
    L = normalize(L);

    // compute diffuse light
    float diffuseLight = max(dot(L, N),0);
    float3 diffuse = diffuseLight * albedo * attr_color;

    // compute specular term
    float3 V = normalize(vspos_camera - P);
    float3 H = normalize(L + V);
    float specularLight = pow(max(dot(N, H), 0), 180);

    if (diffuseLight <= 0) specularLight = 0;
    float3 specular = Ks * attr_color * specularLight* albedo;

    // output
    o_color0.xyz = attenuation * diffuse + specular ;
    o_color0.w = 1;
 
}

Feel free to discuss, there may be errors or bad comments.

Panda 1.7.2 Win XP
GF 8600GT

Known pipe types:
  wglGraphicsPipe
(all display modules loaded.)
:gobj(error): /d/work/panda3d/samples/diff_lights/def_light.cg: unrecognized par
ameter name (uniform in float3 att_params)
:gobj(error): /d/work/panda3d/samples/diff_lights/def_light.cg: unrecognized par
ameter name (uniform in float3 light_radius)
:gobj(error): /d/work/panda3d/samples/diff_lights/def_light.cg: invalid paramete
r name (uniform in float3 Kd)
:gobj(error): /d/work/panda3d/samples/diff_lights/def_light.cg: invalid paramete
r name (uniform in float3 Ks)
:gobj(error): Shader encountered an error.
Traceback (most recent call last):
  File "main.py", line 151, in <module>
    application = Application()
  File "main.py", line 25, in __init__
    self.setup_scene()
  File "main.py", line 115, in setup_scene
    light.setShaderInput("Kd", Vec3(1,1,1))
TypeError: Arguments must match one of:
setShaderInput(non-const NodePath this, const ShaderInput inp)
setShaderInput(non-const NodePath this, non-const InternalName id)
setShaderInput(non-const NodePath this, string id)
setShaderInput(non-const NodePath this, non-const InternalName id, const Vec4 v)

setShaderInput(non-const NodePath this, non-const InternalName id, const NodePat
h np)
setShaderInput(non-const NodePath this, non-const InternalName id, non-const Tex
ture tex)
setShaderInput(non-const NodePath this, string id, const Vec4 v)
setShaderInput(non-const NodePath this, string id, const NodePath np)
setShaderInput(non-const NodePath this, string id, non-const Texture tex)
setShaderInput(non-const NodePath this, non-const InternalName id, float n1)
setShaderInput(non-const NodePath this, string id, float n1)
setShaderInput(non-const NodePath this, non-const InternalName id, const Vec4 v,
 int priority)
setShaderInput(non-const NodePath this, non-const InternalName id, const NodePat
h np, int priority)
setShaderInput(non-const NodePath this, non-const InternalName id, non-const Tex
ture tex, int priority)
setShaderInput(non-const NodePath this, string id, const Vec4 v, int priority)
setShaderInput(non-const NodePath this, string id, const NodePath np, int priori
ty)
setShaderInput(non-const NodePath this, string id, non-const Texture tex, int pr
iority)
setShaderInput(non-const NodePath this, non-const InternalName id, float n1, flo
at n2)
setShaderInput(non-const NodePath this, string id, float n1, float n2)
setShaderInput(non-const NodePath this, non-const InternalName id, float n1, flo
at n2, float n3)
setShaderInput(non-const NodePath this, string id, float n1, float n2, float n3)

setShaderInput(non-const NodePath this, non-const InternalName id, float n1, flo
at n2, float n3, float n4)
setShaderInput(non-const NodePath this, string id, float n1, float n2, float n3,
 float n4)
setShaderInput(non-const NodePath this, non-const InternalName id, float n1, flo
at n2, float n3, float n4, int priority)
setShaderInput(non-const NodePath this, string id, float n1, float n2, float n3,
 float n4, int priority)

Looks like the original code was written against Panda 1.8 (buildbot version), not 1.7.

Deferred normal mapping has arrived !


Code incoming.

Some news,

I’m implementing various effects found on the web; I have a question about shader performances : I have a very simple scene with 10 spheres, I have the following effects on it :

  • 2 deferred point lights (no shadows),
  • 1 deferred dir light (no shadows),
  • deferred normal maps,
  • dof,
  • color correction pass.

I get about 250 fps with a gtx 260 + e8400, do you think it will be ok with a real game or is it too slow already, (the dof shader being the most intensive) ?
How do I get the rendering time of frames with cg shaders on linux ?

Nice effects you have there.

Hey, I had to compute normals with trans-egg to get them display properly, try this sphere :
http://www.mediafire.com/?zf9bood0sb91oz0

What does the buffer display?

The first version relies on autoshader to get normals buffer.

hey, is the dof shader available too? Did you use filtermanager?

Hello,

I’m implementing more effects atm; I’ll post the code when it will be cleaner and faster.

The dof shader is the one Martins Upitis did : http://artmartinsh.blogspot.com/2010/02/glsl-lens-blur-filter-with-bokeh.html It’s not the best method available (search for sprite based bokeh dof) but it’s working and the effect looks better than simple fullscreen blur.

I’m using filter manager atm because it’s the last effect applied and is fullscreen

The thing is Panda3d doesn’t have a built-in DOF filter (in the commonfilters), though I think it’s used more ofthen then few other’s which are included. It would be a nice addition to Panda3d, if you’d want to contribute, though I’m not sure if the devs would add a GLSL version.

Fxaa looks ok !
Doesn’t work with basic shaders.

Performance hit : about 20 fps with gtx260 @1024x768

with :

without :

shader (ported from geeks3D glsl) :

//Cg

varying vec4 posPos;
uniform float FXAA_SUBPIX_SHIFT = 1.0/4.0;
const float rt_w = 1024; // resolution width
const float rt_h = 768; // resolution height

void vshader(float4 vtx_position : POSITION,
             out float4 l_position : POSITION,
             out float2 l_texcoord : TEXCOORD0,
             out float4 l_posPos : TEXCOORD1,
             uniform float4 texpad_color,
             uniform float4x4 mat_modelproj)
{
  l_position = mul(mat_modelproj, vtx_position);
  l_texcoord = (vtx_position.xz * texpad_color.xy) + texpad_color.xy;
 
  vec2 rcpFrame = vec2(1.0/rt_w, 1.0/rt_h);
  posPos.xy = l_texcoord.xy;
  posPos.zw = l_texcoord.xy - (rcpFrame * (0.5 + FXAA_SUBPIX_SHIFT));

  l_posPos = posPos;
}


uniform float vx_offset;
uniform float FXAA_SPAN_MAX = 8.0;
uniform float FXAA_REDUCE_MUL = 1.0/8.0;

#define FxaaInt2 ivec2
#define FxaaFloat2 vec2
#define FxaaTexLod0(t, p) texture2DLod(t, p, 0.0)
#define FxaaTexOff(t, p, o, r) tex2Dlod(t, float4(p.xy + (o * rcpFrame), 0, 0))

vec3 FxaaPixelShader(
  vec4 posPos, // Output of FxaaVertexShader interpolated across screen.
  sampler2D tex, // Input texture.
  vec2 rcpFrame) // Constant {1.0/frameWidth, 1.0/frameHeight}.
{
/*---------------------------------------------------------*/
    #define FXAA_REDUCE_MIN   (1.0/128.0)
    //#define FXAA_REDUCE_MUL   (1.0/8.0)
    //#define FXAA_SPAN_MAX     8.0
/*---------------------------------------------------------*/
    vec3 rgbNW = FxaaTexLod0(tex, posPos.zw).xyz;
    vec3 rgbNE = FxaaTexOff(tex, posPos.zw, FxaaInt2(1,0), rcpFrame.xy).xyz;
    vec3 rgbSW = FxaaTexOff(tex, posPos.zw, FxaaInt2(0,1), rcpFrame.xy).xyz;
    vec3 rgbSE = FxaaTexOff(tex, posPos.zw, FxaaInt2(1,1), rcpFrame.xy).xyz;
    vec3 rgbM  = FxaaTexLod0(tex, posPos.xy).xyz;
/*---------------------------------------------------------*/
    vec3 luma = vec3(0.299, 0.587, 0.114);
    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbM,  luma);
/*---------------------------------------------------------*/
    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
/*---------------------------------------------------------*/
    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));
/*---------------------------------------------------------*/
    float dirReduce = max(
        (lumaNW + lumaNE + lumaSW + lumaSE) * (0.25 * FXAA_REDUCE_MUL),
        FXAA_REDUCE_MIN);
    float rcpDirMin = 1.0/(min(abs(dir.x), abs(dir.y)) + dirReduce);
    dir = min(FxaaFloat2( FXAA_SPAN_MAX,  FXAA_SPAN_MAX),
          max(FxaaFloat2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),
          dir * rcpDirMin)) * rcpFrame.xy;
/*--------------------------------------------------------*/
    vec3 rgbA = (1.0/2.0) * (
        FxaaTexLod0(tex, posPos.xy + dir * (1.0/3.0 - 0.5)).xyz +
        FxaaTexLod0(tex, posPos.xy + dir * (2.0/3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA * (1.0/2.0) + (1.0/4.0) * (
        FxaaTexLod0(tex, posPos.xy + dir * (0.0/3.0 - 0.5)).xyz +
        FxaaTexLod0(tex, posPos.xy + dir * (3.0/3.0 - 0.5)).xyz);
    float lumaB = dot(rgbB, luma);
    if((lumaB < lumaMin) || (lumaB > lumaMax)) return rgbA;
    return rgbB; }

vec4 PostFX(float4 pos, sampler2D tex, vec2 uv, float time)
{
  vec4 c = vec4(0.0);
  vec2 rcpFrame = vec2(1.0/rt_w, 1.0/rt_h);
  c.rgb = FxaaPixelShader(pos, tex, rcpFrame);
  //c.rgb = 1.0 - texture2D(tex, pos.xy).rgb;
  c.a = 1.0;
  return c;
}

void fshader(float2 l_texcoord : TEXCOORD0,
             float4 l_posPos : TEXCOORD1,
             uniform sampler2D k_color : TEXUNIT0,
             uniform float k_active,
             uniform float4 texpix_color,
             uniform float4 texpad_color,
             out float4 o_color : COLOR)
{
  float4 without_fxaa = tex2D(k_color, l_texcoord);

  vec2 uv = l_texcoord.st;
  o_color = PostFX(l_posPos, k_color, uv, 0.0);

  if(k_active == 0) o_color = without_fxaa;
}

To use it, render whole scene to a fullscreen quad and just pass your colormap (linearly filtered) as input :

quad.setShader(loader.loadShader("fxaa.cg"))
quad.setShaderInput("color", color_map)
quad.setShaderInput("active", 1)

Still needs more testing with proper scene, I don’t know why the texture2DLod function doesn’t throw error (should be tex2Dlod in cg).

Anon : I don’t know how to do this atm, I’d prefer a real programmer to do it :stuck_out_tongue:

Uh, do what?

PS. Youre not a real programmer? Off-topic, but how did you learn shader programming, Ive tried the GPU gems book, the Orange book and nothing. Or youre a maths person?

To implement the dof code into panda source, I don’t know C++ :frowning:

I’m hybrid art/coder in web applications, I have AS3 background.
About shader programming the nvidia Cg tutorial is very good. : http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter01.html

Programmer or not, the work you’re going is pretty bad ass. Keep it up! I’d love to see this integrated into Panda at some point.

If I may have a suggestion, it would be great if you kept a change log in the first post. Otherwise, once this thread grows, it will be difficult to keep up with what’s available and what’s not.

Also, I’d like to know if transparency will be supported in any way? I have little to no background on the subject, but I’ve read that deferred lighting doesn’t work with transparent models well.

thanks man!

I’d like to integrate this as well if it’s possible in python, but I don’t know how contributing to panda project works and if this code worth it. Can anybody add code to panda source, how does it work in general?

the changelog is good idea, I’m on it.

Speaking about transparency, you are right, this is the main limitation of the deferred shading, the solution is to use fixed pipeline to render transparent objects.

i think commonfilters is a python module. You can find it in “C:\Panda3D-1.X.X\direct\filter\CommonFilters.py”. I didnt mean for you to do that though, you could just give an example of using the shader with filtermanager.

The bloom code is working (2-pass blur + HDR), I don’t know if the HDR tone mapping is really working, needs more testing.

I’ve added vignetting effect to the camera, the vignetting effect is function of the aperture size.

The quantity of light received by the lens is function of the aperture size.

I’ve simplified the dof code, it needs 24 texture lookups now.

I might make a video to show it in motion.

Cool, you should join forces with Craig.

Hello all,

I’m trying Strauss lighting model (http://wiki.gamedev.net/index.php/D3DBook:%28Lighting%29_Strauss), and I’m not so happy about the result, this should be a metal material :

Also the color diffusion is not as smooth as in the phong model (look at red lighting):
:laughing:

Anyone got experience with this model ? and what lighting model do you use in your own applications ?

Phong model :

Btw, I can’t set my texture format, setFormat(Texture.Format) doesn’t have any incidence, I want to try fp16 textures for hdr, I’ve searched on the forums, someone asked for 128 bits textures format but he couldn’t change the format of texture as well. What is the current state on this ? I’ve tried to include setFormat() in the filtermanager panda class but no success :frowning:.

Thanks.

EDIT :

Looks ok now.

I’m trying to import custom models but I have still some uv mapping problems :imp: :

I have a more serious question btw and I don’t want to create new thread for this, do you still need samples or not ([url]Better models in samples]) ?