samplerCubeShadow doesn't work, "Sampler object 1 does not have depth compare enabled"

I want to have soft shadows on my point lights. They work when I used the normal samplerCube shadowMap in my light source. However, as soon I switch to samplerCubeShadow shadowMap I get the following error every frame:

:display:gsg:glgsg(warning): Program undefined behavior warning: Sampler object 1 does not have depth compare enabled. It is being used with depth texture PointLight, by a program that samples it with a shadow sampler. This is undefined behavior.

I doubt this is a shader problem, it sounds like I need to set something differently on the Panda side?
Still, here’s the relevant part of the fragment shader:


float NEAR = 0.05;
float FAR = 20;
in vec4 shadow_coords;
uniform float use_shadows = 1.0;

uniform struct p3d_LightSourceParameters {
  // Primary light color.
  vec4 color;

  // Light color broken up into components, for compatibility with legacy
  // shaders.  These are now deprecated.
  vec4 ambient;
  vec4 diffuse;
  vec4 specular;

  // View-space position.  If w=0, this is a directional light, with the xyz
  // being -direction.
  vec4 position;

  // Spotlight-only settings
  vec3 spotDirection;
  float spotExponent;
  float spotCutoff;
  float spotCosCutoff;

  // Individual attenuation constants
  float constantAttenuation;
  float linearAttenuation;
  float quadraticAttenuation;

  // constant, linear, quadratic attenuation in one vector
  vec3 attenuation;

  // Shadow map for this light source
  //samplerCube shadowMap;
  samplerCubeShadow shadowMap;

  // Transforms view-space coordinates to shadow map coordinates
  mat4 shadowViewMatrix;
} p3d_LightSource[1];

float calcShadow( vec4 shadow_coords )
{
	vec4 coords;
	coords.xyz = shadow_coords.xyz / shadow_coords.w;
	float dist = max(abs(coords.x), max(abs(coords.y), abs(coords.z)));
	coords.w = (NEAR + FAR) / (FAR - NEAR) + ((-2 * FAR * NEAR) / (dist * (FAR - NEAR)));
	coords.w = coords.w * 0.5 + 0.5;

	// Works, with "samplerCube shadowMap":
	// float shadow = (texture( p3d_LightSource[0].shadowMap, coords.xyz, 1e-4 ).r > coords.w) ? 1.0 : 0.0;
	// Should work with "samplerCubeShadow shadowMap"
	float shadow = texture( p3d_LightSource[0].shadowMap, coords );
	return shadow;
}

Has anyone seen this error? Could someone tell me hot to “enable depth compare” for sampler 1? Has anyone used samplerCubeShadow’s for point lights in Panda3D?

After some brief tests, I don’t seem to get the depth comparison error when instantiating samplerCubeShadow.

You seem to be using samplerCubeShadow with a vec4 like the GLSL spec 4.40 describes. My next guess would be that you need some more information from the vertex shader.

I can try around some more, but did you throw together a test that I could run here? Maybe I’m doing something else wrong?

It might also be something with my somewhat outdated hardware - I just thought that shouldn’t be the problem because the compiler doesn’t complain…

It sounds like the depth texture does not have the FT_shadow filter mode enabled.

Can you share more code?

Sorry it took me half a year to reply, I was focusing on other parts of the project.

I tried putting together a minimal example. However, I can no longer reproduce the error I got above, and I don’t know why - maybe some graphic driver update fixed it?

Still, I cannot get the samplerCubeShadow to work. I get “zeros” everywhere in the texture lookup (i.e. no samples pass the depth comparison) but when switching back to samplerCube, everything works as expected. The only thing I changed was a line in the two definitions of p3d_LightSourceParameters and the texture lookup in calcShadow.
I do not get any errors, and I tried different glsl versions…

This sample should be self-contained (save as main.py and run python3 main.py).
Any help appreciated!

What I get with samplerCube:


And with samplerCubeShadow:

Note that the render buffers are almost the same in both cases (press “v” to enable).

from direct.showbase.ShowBase import ShowBase
from panda3d.core import PointLight, AmbientLight
from panda3d.core import Vec4
from panda3d.core import Shader
from direct.showbase.BufferViewer import BufferViewer


# Shaders adapted from examples by @wezu:
# https://discourse.panda3d.org/t/shadows-made-easy/15399
v_shader='''
#version 330
uniform struct p3d_LightSourceParameters {
    // Primary light color.
    vec4 color;

    // Light color broken up into components, for compatibility with legacy
    // shaders.  These are now deprecated.
    vec4 ambient;
    vec4 diffuse;
    vec4 specular;

    // View-space position.  If w=0, this is a directional light, with the xyz
    // being -direction.
    vec4 position;

    // Spotlight-only settings
    vec3 spotDirection;
    float spotExponent;
    float spotCutoff;
    float spotCosCutoff;

    // Individual attenuation constants
    float constantAttenuation;
    float linearAttenuation;
    float quadraticAttenuation;

    // constant, linear, quadratic attenuation in one vector
    vec3 attenuation;

    // Shadow map for this light source
    // ENABLE FOR HARD SHADOWS:
    //samplerCube shadowMap;

    // ENABLE FOR SOFT SHADOWS:
    samplerCubeShadow shadowMap;

    // Transforms view-space coordinates to shadow map coordinates
    mat4 shadowViewMatrix;
} p3d_LightSource[1];

uniform mat4 p3d_ModelViewProjectionMatrix;
uniform mat3 p3d_NormalMatrix;
uniform mat4 p3d_ModelViewMatrix;

in vec4 p3d_Vertex;
in vec3 p3d_Normal;
in vec2 p3d_MultiTexCoord0;

out vec2 uv;
out vec4 shadow_uv;
out vec3 normal;

out vec3 vertex_viewspace;
out vec3 normal_viewspace;

void main()
    {
    //position    
    gl_Position = p3d_ModelViewProjectionMatrix * p3d_Vertex;      
    //normal      
    normal = p3d_NormalMatrix * p3d_Normal;
    //uv
    uv = p3d_MultiTexCoord0;

    normal_viewspace = normalize( p3d_NormalMatrix * p3d_Normal );
    vertex_viewspace = vec3(p3d_ModelViewMatrix * p3d_Vertex);
    //shadows
    //shadow_uv = p3d_LightSource[0].shadowViewMatrix * p3d_Vertex;
    shadow_uv = p3d_LightSource[0].shadowViewMatrix * vec4(vertex_viewspace, 1);
    //shadow_uv = p3d_LightSource[0].shadowViewMatrix * (p3d_ModelViewMatrix * p3d_Vertex);

}
'''

#fragment shader
f_shader='''
#version 330

uniform sampler2D p3d_Texture0;
uniform vec3 camera_pos;
uniform float shadow_blur;

in vec2 uv;
in vec4 shadow_uv;
in vec3 normal;
in vec3 vertex_viewspace;
in vec3 normal_viewspace;

out vec4 out_color;


float NEAR = 1;
float FAR = 500;
in vec4 shadow_coords;

uniform struct p3d_LightSourceParameters {
    // Primary light color.
    vec4 color;

    // Light color broken up into components, for compatibility with legacy
    // shaders.  These are now deprecated.
    vec4 ambient;
    vec4 diffuse;
    vec4 specular;

    // View-space position.  If w=0, this is a directional light, with the xyz
    // being -direction.
    vec4 position;

    // Spotlight-only settings
    vec3 spotDirection;
    float spotExponent;
    float spotCutoff;
    float spotCosCutoff;

    // Individual attenuation constants
    float constantAttenuation;
    float linearAttenuation;
    float quadraticAttenuation;

    // constant, linear, quadratic attenuation in one vector
    vec3 attenuation;

    // Shadow map for this light source
    // ENABLE FOR HARD SHADOWS:
    //samplerCube shadowMap;

    // ENABLE FOR SOFT SHADOWS:
    samplerCubeShadow shadowMap;

    // Transforms view-space coordinates to shadow map coordinates
    mat4 shadowViewMatrix;
} p3d_LightSource[1];

float hardShadow(samplerCube tex, vec3 uv, float bias, float depth)
{
	float closestDepth = texture(tex, uv).r; 

	float shadow = 0.0;
	if(depth - bias < closestDepth)
		shadow += 1.0;
	return shadow;
}

float calcShadow( vec4 shadow_coords )
{
	vec4 coords;
	coords.xyz = shadow_coords.xyz / shadow_coords.w;
	float dist = max(abs(coords.x), max(abs(coords.y), abs(coords.z)));
	coords.w = (NEAR + FAR) / (FAR - NEAR) + ((-2 * FAR * NEAR) / (dist * (FAR - NEAR)));
	coords.w = coords.w * 0.5 + 0.5;

        // ENABLE FOR HARD SHADOWS:
        //float shadow = hardShadow( p3d_LightSource[0].shadowMap, coords.xyz, 1e-4, coords.w );

        // ENABLE FOR SOFT SHADOWS:
	float shadow = texture( p3d_LightSource[0].shadowMap, coords );

	return shadow;
}

void main()
{
    //base color
    vec3 ambient=vec3(0.03, 0.03, 0.03);    
    //texture        
    vec4 tex=texture(p3d_Texture0, uv);        
    //light ..sort of, not important
    vec3 light=p3d_LightSource[0].color.rgb*max(dot(normalize(normal),-p3d_LightSource[0].spotDirection), 0.0);

    vec3 diffuseCol = tex.rgb;
    float shininess = 1;

    vec3 color = vec3(0,0,0);
    vec3 diff = p3d_LightSource[0].position.xyz - vertex_viewspace * p3d_LightSource[0].position.w;
    vec3 L = normalize(diff);
    vec3 E = normalize(-vertex_viewspace);
    vec3 R = normalize(-reflect(L, normal_viewspace));
    float diffusePower = max(dot(normal_viewspace, L), 0) * 0.5;

    float specularPower = pow(diffuseCol.r*max(dot(R, E), 0), shininess);

    float len = length(diff);
    float attenuation = 1 / dot(p3d_LightSource[0].attenuation, vec3(1, len, len*len));

    float specular = 0.1;

    color += diffuseCol.rgb * p3d_LightSource[0].color.rgb * diffusePower * attenuation;
    color += p3d_LightSource[0].color.rgb * specularPower * specular * attenuation;

    // calculate shadow
    float shadow = calcShadow( shadow_uv );
    //make the shadow brighter
    shadow=0.5+shadow*0.5;

    //out_color=vec4(shadow,shadow,shadow,1);//, tex.a);
    out_color = vec4(ambient+color*shadow, tex.a);

}'''                    

class MyApp(ShowBase):

    def __init__(self):

        ShowBase.__init__(self)

        # Load the environment model.
        self.scene = self.loader.loadModel("environment.egg")

        # Reparent the model to render.
        self.scene.reparentTo(self.render)

        # Apply scale and position transforms on the model.
        self.scene.setScale(0.25, 0.25, 0.25)
        self.scene.setPos(-8, 42, 0)

        # Set up point light that will cast shadows:
        plight = PointLight('plight')
        plight.setColorTemperature( 3600 )
        plight.setShadowCaster(True, 1024, 1024)
        plnp = render.attachNewNode(plight)
        plnp.setPos(0, 0, 5)
        render.setLight(plnp)

        for i in range(6):
            plight.getLens(i).setNear(1)
            plight.getLens(i).setFar(500)

        # Let environment receive shadows:
        self.init_shaders()

        self.accept("v", base.bufferViewer.toggleEnable)

    def init_shaders(self):

        shader = Shader.make(Shader.SL_GLSL, v_shader, f_shader)
        self.scene.setShader(shader)
        self.scene.set_two_sided(True)
        #self.scene.ls()
 

app = MyApp()
app.run()

I tested the code above on another PC, I get the same results:
Hard shadows work, but when I switch to samplerCubeShadow, I don’t get any shadows any more…

Apparently, we don’t enable FT_shadow on the auto-generated shadow maps from cube maps because Cg doesn’t support shadow-filtered cube maps, and we use Cg in the automatic shader generator. This will certainly be remedied in Panda 1.11, but in the meantime, perhaps we can find a workaround (via a PRC variable or otherwise).

1 Like

Thanks a lot for looking into this!

By “we can find a workaround”, you mean in the Panda codebase? Is there, alternatively, a way to change the filter mode after creating the light?

I tried the following, but it just seems to disable the shadows altogether:

        sBuffer = self.lightNode.node().getShadowBuffer( GraphicsStateGuardianBase.getGsg(0) )
        if sBuffer is not None:
            tex = sBuffer.getTexture()
            print(tex.default_sampler.getMinfilter())   # outputs "1"
            print(tex.default_sampler.getMagfilter())   # outputs "1"

            state = tex.default_sampler
            # Create mutable copy:
            state = SamplerState( state )
            state.setMinfilter( SamplerState.FT_shadow )
            state.setMagfilter( SamplerState.FT_shadow )
            tex.setDefaultSampler( state )
            print(tex.default_sampler.getMinfilter())   # outputs "6"
            print(tex.default_sampler.getMagfilter())   # outputs "6"

That should do it, if anything.

If you can’t get it to work, please post the issue on GitHub with a self-contained simple program.

It looks like I got it to work after all, with the manual workaround in my previous post.
Maybe the issue was that I didn’t call that code “late” enough, i.e. it seems that the pipeline needs some time to initialize textures etc. For example, right after creating the point light, self.lightNode.node().getShadowBuffer() returns None…
image

The effect is more subtle than I had thought, but at least now I’ve got hardware-enabled soft shadows!

Note however that it’s still weird that this workaround is required - it looks like out of the box, shadowSamplerCube does not work. If no one has done this already, I’ll open an issue on github.

The shadow buffer gets created as needed on the fly by the shader generator. You can force the shader generator to do its thing by premunging the scene.