Shadows: direction, parameters, and inputs in GLSL

Thaumaturge · February 11, 2016, 5:15pm

I’ve been working on a simple GLSL implementation of shadow-mapping, and have a handful of small (I think) questions. Since none of these seems worth a thread in and of itself, I’d like to ask them all here, if I may:

When getting the (model-space) direction of a directional light, is “row2__to_model” the correct value to examine? It looks more or less correct, but if so then I seem to have a problem somewhere else: its x/“left”- and y/“forward”- values seem to point in the opposite direction to that which I expect, an observation that might be borne out by the fact that the shadows appear to be cast in the opposing direction. Note that both the shadows and the lighting algorithm appear to operate as expected in the z/“up”-direction.

It’s possible, of course, that there’s a problem with my shadow code, but that’s adapted from the “advanced shadows” tutorial, so I’m more inclined to suspect my choice of source for my light direction.

What’s the recommended method of passing in other light parameters–colour, for example–in Panda 1.9? I’m aware of p3d_LightSourceParameters, but my understanding is that it’s first available in 1.10. Should I simply pass my parameters in as ordinary shader inputs?
Speaking of shader inputs, should I attempt to reduce their number, or is it safe to add as many as I feel called for?

tobspr · February 11, 2016, 6:32pm

I can’t answer you the first question, but for your last question:

Each shader input adds a bit of overhead. You should in general attempt to keep your calls to set_shader_input as low as possible. A simple optimization is to use PTA_float for example (instaed of float), and only call setShaderInput() once. You can then update the data using my_pta[0] = new_value. You can construct a new pta storing only a single value e.g. with: PTA_float.empty_array(1).

There is also experimental support for UBO’s in the ubo branch. These are blocks of uniforms which should speed up the shader input handling, however I’m not sure if thats required for you.

Thaumaturge · February 12, 2016, 12:57pm

Ah, thank you–I’ll try to keep that in mind once I start adding parameters!

rdb · February 12, 2016, 10:58pm

For what it’s worth, here is a sample program showing how to use shadows with the built-in parameters in Panda:

from panda3d.core import *
load_prc_file_data("", "framebuffer-srgb #t")
#load_prc_file_data("", "gl-support-shadow-filter false")
load_prc_file_data("", "default-fov 90")
import math

from direct.showbase.ShowBase import ShowBase
from direct.interval.IntervalGlobal import Parallel, Sequence
base = ShowBase()

EXPONENT = 128
BRIGHTNESS = 1
DIST = 0.5

shader = Shader.make(Shader.SL_GLSL, """
#version 330

uniform mat4 p3d_ModelViewProjectionMatrix;
uniform mat3 p3d_NormalMatrix;

in vec4 vertex;
in vec3 normal;

out vec3 vpos;
out vec3 norm;
out vec4 shad[3];

uniform struct {
  vec4 position;
  vec4 diffuse;
  vec4 specular;
  vec3 attenuation;
  vec3 spotDirection;
  float spotCosCutoff;
  float spotExponent;
  sampler2DShadow shadowMap;
  mat4 shadowMatrix;
} p3d_LightSource[3];

void main() {
  gl_Position = p3d_ModelViewProjectionMatrix * vertex;
  vpos = vec3(gl_ModelViewMatrix * vertex);
  norm = normalize(p3d_NormalMatrix * normal);
  shad[0] = p3d_LightSource[0].shadowMatrix * vertex;
  shad[1] = p3d_LightSource[1].shadowMatrix * vertex;
  shad[2] = p3d_LightSource[2].shadowMatrix * vertex;
}
""", """
#version 330

uniform sampler2D p3d_Texture0;

uniform struct {
  vec4 ambient;
} p3d_LightModel;

uniform struct {
  vec4 ambient;
  vec4 diffuse;
  vec3 specular;
  float shininess;
} p3d_Material;

uniform struct {
  vec4 position;
  vec4 diffuse;
  vec4 specular;
  vec3 attenuation;
  vec3 spotDirection;
  float spotCosCutoff;
  float spotExponent;
  sampler2DShadow shadowMap;
  mat4 shadowMatrix;
} p3d_LightSource[3];

in vec3 vpos;
in vec3 norm;
in vec4 shad[3];

void main() {
  gl_FragColor = p3d_LightModel.ambient * p3d_Material.ambient;

  for (int i = 0; i < p3d_LightSource.length(); ++i) {
    vec3 diff = p3d_LightSource[i].position.xyz - vpos * p3d_LightSource[i].position.w;
    vec3 L = normalize(diff);
    vec3 E = normalize(-vpos);
    vec3 R = normalize(-reflect(L, norm));
    vec4 diffuse = clamp(p3d_Material.diffuse * p3d_LightSource[i].diffuse * max(dot(norm, L), 0), 0, 1);
    vec4 specular = vec4(p3d_Material.specular, 1) * p3d_LightSource[i].specular * pow(max(dot(R, E), 0), gl_FrontMaterial.shininess);

    float spotEffect = dot(normalize(p3d_LightSource[i].spotDirection), -L);

    if (spotEffect > p3d_LightSource[i].spotCosCutoff) {
      diffuse *= pow(spotEffect, p3d_LightSource[i].spotExponent);
      diffuse *= textureProj(p3d_LightSource[i].shadowMap, shad[i]);
      gl_FragColor += diffuse / dot(p3d_LightSource[i].attenuation, vec3(1, length(diff), length(diff) * length(diff)));
    }
  }

  gl_FragColor.a = 1;
}
""")

cm = CardMaker("card")
cm.setFrame(-5, 5, -5, 5)
card = render.attachNewNode(cm.generate())
m = Material()
m.set_diffuse((1, 1, 1, 1))
card.setMaterial(m)

red = Spotlight("red")
red.setColor((BRIGHTNESS, 0, 0, 1))
red.setExponent(EXPONENT)
red.getLens().setNearFar(5, 11)
red.getLens().setFov(30)
red_path = render.attachNewNode(red)
red_path.set_pos(1, -10, 0)
red_path.look_at(DIST * math.cos(math.pi * 1.333), 0, DIST * math.sin(math.pi * 1.3333))

green = Spotlight("green")
green.setColor((0, BRIGHTNESS, 0, 1))
green.setExponent(EXPONENT)
green.getLens().setNearFar(5, 11)
green.getLens().setFov(30)
green_path = render.attach_new_node(green)
green_path.set_pos(-1, -10, 0)
green_path.look_at(DIST * math.cos(math.pi * 0.666), 0, DIST * math.sin(math.pi * 0.6666))
green_path.set_x(green_path.get_x() + 1)

blue = Spotlight("blue")
blue.setColor((0, 0, BRIGHTNESS, 1))
blue.setExponent(EXPONENT)
blue.getLens().setNearFar(5, 11)
blue.getLens().setFov(30)
blue_path = render.attach_new_node(blue)
blue_path.set_pos(0, -10, 1)
blue_path.look_at(DIST * math.cos(0), DIST * math.sin(0), 0)
blue_path.set_y(green_path.get_y() + 1)

red.setShadowCaster(True, 4096, 4096)
green.setShadowCaster(True, 4096, 4096)
blue.setShadowCaster(True, 4096, 4096)

#red.show_frustum()
#green.show_frustum()
#blue.show_frustum()

render.set_light(red_path)
render.set_light(green_path)
render.set_light(blue_path)


sphere = loader.loadModel("models/sphere")

spheres = render.attach_new_node("spheres")
sphere.copy_to(spheres).set_pos(1.5, 0, 0)
sphere.copy_to(spheres).set_pos(0, 1.5, 0)
sphere.copy_to(spheres).set_pos(-0.75, -0.75, 1.5)
spheres.set_scale(0.5)
spheres.set_y(-1)
Sequence(
    spheres.posInterval(0.7, (0, -3, 0)),
    spheres.posInterval(0.7, (0, -1, 0))
  ).loop()

spheres.hprInterval(1, (360, 360, 360)).loop()

render.set_shader(shader)

base.trackball.node().set_pos(0, 6, 0)
base.trackball.node().set_hpr(0, -60, 30)

base.accept('a', lambda: render.set_shader_auto(True))
base.accept('s', lambda: render.set_shader(shader))
base.accept('n', lambda: render.clear_shader())
base.accept('tab', base.bufferViewer.toggleEnable)

base.run()

Sorry that it’s a bit messy and unoptimized - I used this for testing the built-in GLSL shadow inputs and comparing the results against the equivalent autoshader (enabled using A).

In 1.9, there is still p3d_LightSourceParameters, but not p3d_LightSource[n]. You would have to declare a custom input as p3d_LightSourceParameters and then you can pass a NodePath pointing to a light to setShaderInput. [see] However, the shadowMatrix and shadowMap members are missing in 1.9.

As for setShaderInput being slow, I wouldn’t worry about it. Since you can pass a NodePath to setShaderInput, there shouldn’t be a need to call it every frame. tobspr’s use cases typically involve tens of thousands of shader input changes in a frame, where the overhead starts to become noticeable.

Thaumaturge · February 13, 2016, 2:08pm

Interesting, and thank you.

I actually have my shadows more or less working now, I believe–I’m only held back by the first issue mentioned above, the odd matter of my directional light shading backwards in x and y, but not in z.

Looking through the code that you posted, you seem to use the difference between the light and the current vertex as the light’s direction; while that makes sense for a spot- or point- light (and indeed, I have that working for my player’s “lantern” light, which is a pseudo-spotlight pretending to be a point-light), it doesn’t seem quite right for a directional light.

Ah, I see.

Hmm… I do notice that your light-source struct includes a variable named “spotDirection”–is that available for directional lights, and if so, is it what it seems to be: the light’s direction as controlled via calls to “setHpr”?

Ah, fair enough, and thank you for clarifying! I’ll switch back to passing in my handful of variables individually, I think: it produces code that’s a little clearer than that produced when separating them out from an array of floats, I feel.

rdb · February 13, 2016, 3:13pm

My code should work regardless of whether it is a spot light, directional light or point light. As explained in the linked thread, the direction of a directional light is stored in the “position” field like so, as is the OpenGL convention:

vec4(-direction, 0)

This allows us to calculate the light vector as such:

vec3 diff = light.position.xyz - vpos * light.position.w;

As you can see, the vpos is only subtracted when the w is 1 (as is the case with punctual lights) but it is ignored when the w is 0 (ie. directional light). This means the same code works for both spotlights and directional lights.

spotDirection is always defined as the light’s view vector (in camera view space, of course). In my shader it is effectively ignored for directional lights since spotCosCutoff is -1 for non-spotlights (meaning the if cannot evaluate to true). I think it would work fine for directional lights, but I suggest just using “position.xyz” instead.

As for your directional light shading backwards in X and Y: I wonder if this might be a coordinate conversion issue. Panda sets the coordinate system to Y-up in OpenGL for historical reasons; in a modern shader-based pipeline there is no reason for this. You may make things easier on yourself if you tell Panda to always work in its default coordinate system instead, by setting this in Config.prc:

gl-coordinate-system default

This may not be the cause of the issue at hand, but it may be a good idea to do this anyway.

Thaumaturge · February 13, 2016, 4:07pm

Ah, I see, thank you. I wasn’t aware of that, and had missed the use of “position” in the linked thread.

Hum. I seem to be having some trouble merging the code that I have, which was based on the “advanced” shadow tutorial, with yours above. I’ve added the following to my shaders, both fragment and vertex:

struct p3d_LightSourceParameters {
  vec4 position;
  vec4 diffuse;
  vec4 specular;
  vec3 attenuation;
  vec3 spotDirection;
  float spotCosCutoff;
  float spotExponent;
  sampler2DShadow shadowMap;
  mat4 shadowMatrix;
};

uniform p3d_LightSourceParameters mainLight;

Along with that, my game-code now creates a normal DirectionalLight, set to be a shadow-caster and applied to render via “render.setLight”; this light is then passed to the shader via “render.setShaderInput”.

However, I’m seeing some odd results:

Attempting to simply apply “mainLight.position.xyz” seems to produce a direction that’s affected by the camera’s orientation.
The shadow pixel produced by “shadow2D(mainLight.shadowMap, mainLightShadowCoord.xyz).x” appears to be drawn from the object’s main texture: I get that image smeared at an angle over the object, rather than shadows.
The shader doesn’t seem to be getting the “shadowMatrix” input–in running the following line:

mainLightShadowCoord = mainLight.shadowMatrix*gl_Vertex;

I’m getting the following error:

Assertion failed: Shader input mainLight.shadowMatrix is not present.
 at line 494 of panda/src/pgraph/shaderAttrib.cxx

“mainLightShadowCoord” is a “varying vec4”.

I note that the tutorial doesn’t actually create lights, as such; it simply creates cameras and buffers, then passes those into the shaders. However, I’m guessing that “p3d_LightSourceParameters” relies on the input being a light, not just a camera.

In case it’s relevant, I’m using GLSL version 120 at the moment.

Hmm, adding “loadPrcFileData(”“, “gl-coordinate-system default #t”)” to the start of my program doesn’t seem to change the result.

Your mention of the coordinate system being “y-up” prompts me to clarify something: when I’ve spoken of the x- and y- directions above, I’ve meant the two horizontal axes, and used “z” to refer to the “up” axis. My problem is thus that objects are lit as expected with respect to their tops and bottoms, but the horizontal direction of the main light seems to be reversed.

rdb · February 13, 2016, 9:15pm

It is affected by the camera’s orientation. It is in view space, as is the OpenGL convention. The shader I gave does view-space lighting.
shadowMap input is only available in Panda 1.10, as stated earlier.
shadowMatrix input is only available in Panda 1.10, as stated earlier.

Sorry, I did not realise you were targeting such an ancient version of OpenGL. If you are using GLSL 1.20, you can just use the deprecated gl_LightSourceParameters and gl_LightSource[n] inputs, which index into the lights set using setLight. Note that using GLSL versions before 1.30 limits portability on some platforms, since modern OpenGL implementations are not required to support GLSL 1.20 and earlier.

In general, the p3d_LightSourceParameters set of inputs try to emulate the deprecated gl_LightSourceParameters that were available in GLSL 1.20. In Panda 1.9, we only supported the p3d_LightSourceParameters structure members that were available in GLSL 1.20. You can check this page for an overview:
opengl.org/sdk/docs/tutoria … ghting.php

The setting is “gl-coordinate-system default”, not “gl-coordinate-system default #t”. The variable is “gl-coordinate-system”, and the value is “default”.

OK, an overview of what’s available and what isn’t, for clarity:

// GLSL 110/120  (implicitly declared)
struct gl_LightSourceParameters {
   vec4 ambient;
   vec4 diffuse;
   vec4 specular;
   vec4 position;
   vec4 halfVector;
   vec3 spotDirection;
   float spotExponent;
   float spotCutoff;
   float spotCosCutoff;
   float constantAttenuation;
   float linearAttenuation;
   float quadraticAttenuation;
};
uniform gl_LightSourceParameters gl_LightSource[n];

// Panda 1.9+
struct p3d_LightSourceParameters {
   vec4 ambient;
   vec4 diffuse;
   vec4 specular;
   vec4 position;
   vec4 halfVector;
   vec3 spotDirection;
   float spotExponent;
   float spotCutoff;
   float spotCosCutoff;
   float constantAttenuation;
   float linearAttenuation;
   float quadraticAttenuation;
};
uniform p3d_LightSourceParameters customLight;

// Panda 1.10+
struct p3d_LightSourceParameters {
   vec4 ambient;
   vec4 diffuse;
   vec4 specular;
   vec4 position;
   vec4 halfVector;
   vec3 spotDirection;
   float spotExponent;
   float spotCutoff;
   float spotCosCutoff;
   float constantAttenuation;
   float linearAttenuation;
   float quadraticAttenuation;

   vec3 attenuation;
   sampler2DShadow shadowMap;
   mat4 shadowMatrix;
};
uniform p3d_LightSourceParameters p3d_LightSource[n];
uniform p3d_LightSourceParameters customLight;

Thaumaturge · February 15, 2016, 12:44pm

[edit 2]

For now I think that I’ve found a solution that works for me:

I’m honestly not sure of how to properly integrate Panda lights with a custom shader in 1.9: While 1.10 provides direct access to the shadow-buffer in the light-properties struct, I’m not finding an obvious way to access it in 1.9. While LightLensNode does have the “getShadowBuffer” method, it description indicates that it’s not intended for non-debugging use, and when I attempted to use it anyway I believe that it returned “None”. I could use the buffers that I’ve been creating under the tutelage of the “advanced shadows” tutorial, but I don’t see a means of connecting them to a light.

On top of that, my current lighting algorithm doesn’t use most of the parameters given by the light-properties struct.

As a result, I’ve returned to the method provided by the “advanced shadows” tutorial, simply creating cameras and then using “setShadowInput” to providing those parameters that I’m using.

I solved the issue of the light-direction by transforming the vector (0, 0, -1, 0) via the matrix “trans_clip_of_mainLight_to_model”–that is, if I’m not much mistaken, taking a vector defined as pointing in the negative z-direction in the light’s space and transforming it into model-space. This seems to work as expected.

[/edit 2]

… But a light’s apparent direction shouldn’t be affected by the camera’s orientation unless that light is intended to be attached to the camera, surely?

[edit]
Argh, right, I see what you’re saying. Sorry, I fear that lack of sleep might be getting to me a bit. >_<;

All right, I’m doing my lighting in model-space, so I’ll want to transform the “position” value of my directional light–easily-enough done, I believe.
[/edit]

Ah, right–sorry, I seem to have missed or forgotten that somehow. ^^;

Hmm… That’s a concern–I wasn’t aware of the portability issue. I think that I chose 1.20 simply because it was the version that I was familiar with, and the version that I found least difficult to adapt my extant shaders to when I started in on these shadow-shaders. I also don’t want to push the version too high for fear of limiting my potential user-base. In that case, would you recommend 1.30, or even higher?

Ah, I feel silly for having missed that–sorry! ^^;

Right, that does seem to have an effect. Noting that I haven’t yet had another shot at integrating your code above, and am thus still using my own (which uses “row2_mainLight_to_model.xyz” for the light’s direction), it it doesn’t seem to fix the problem: a light with HPR of (-45, -45, 0) now produces light on the left- and front- sides of a cube (as expected), but also on the underside, with a self-shadow partially obscuring it. The top, which I would expect to be lit, isn’t.

Changing the pitch to 45 causes the back- and right- sides to be lit, with the bottom still lit, but no longer self-shadowed (presumably a result of the self-shadow falling onto the already-dark upper side).

Given your explanation above, however, I intend to try again at using your code given above, along with a move to GLSL 1.30 (I can port to a higher version again if called for; porting to 1.30 should, from what you’ve said, at least get me the functionality that you’ve described).

Ah, thank you–that looks like a useful overview!

rdb · February 22, 2016, 2:27pm

I can’t make that decision for you - it depends on your target audience. Below are some considerations.

Let’s first establish what this means; GLSL 1.30 corresponds to OpenGL 3.0, GLSL 1.40 to OpenGL 3.1, and GLSL 1.50 to OpenGL 3.2. (GLSL versions after that correspond 1:1 to OpenGL versions, ie. GLSL 3.30 corresponds to OpenGL 3.3.)

OpenGL 3 corresponds to DirectX 10-level cards. For AMD, this means Radeon HD 4000 series and up. For NVIDIA, this means GeForce 8000 series and up. NVIDIA released their first DX10 cards in late 2006. This means that hardware incapable of running OpenGL 3 applications covers only a tiny portion of the gaming market nowadays, given that most computers don’t last as long as 10 years, certainly not as a gaming computer.

So, if you’re targeting the high-end gaming market, you can target OpenGL 4 safely. If you’re targeting the low/mid-end gaming market, you can target OpenGL 3 safely. If you’re not targeting the gaming market, targeting children using hand-me-down computers or people playing games on 10-year-old laptops, then perhaps you could target OpenGL 2.1 hardware, but this also comes at costs to available functionality and performance budget, which could severely limit your options.

Also to keep in mind is that all hardware capable of OpenGL 3.0 is also capable of OpenGL 3.3. The only difference is driver support. In this respect, I believe that the open-source drivers on Linux have been stuck to older GL 3 versions for a while, only recently having advanced to GL 4 level support, so this might be something to consider - but otherwise this is generally a matter of telling people to make sure their drivers are up-to-date.

Thaumaturge · March 1, 2016, 4:01pm

Fair enough, and a good point.

Thank you for the information.

Honestly, I don’t have a terribly solid idea of my target audience. That said, I don’t want to limit my audience without good reason, and I don’t believe that I have anything in mind that’s terribly advanced.

As a result, and given that GLSL seems to have changed a little across the versions, I think that I’m inclined to stick with GLSL 1.30: it would seem to be widely supported, and remains fairly close to the version of the language that I (more or less) learned.

rdb · April 3, 2017, 12:19pm

Update: shadowMatrix has now been deprecated; please use shadowViewMatrix instead to transform from view-space coordinates, which is more efficient.