First off, boilerplate apology/disclaimer, I am new to shaders as of yesterday.
I’m trying, ultimately, to get an ink-drawing effect, as though the game world were drawn on paper with simple ink outlines. I thought the way to accomplish this would be to use almost the same set-up as the “Tut-Cartoon-Advanced” sample included with panda3d (one shader generates a 2d image of the scene in which surface normals are rendered as colors, then another shader runs edge-detection over that to draw the ink outlines), but to hit all the models with .setAlphaScale(0), so you only see the outlines. And it works, kind of. A problem I ran into was that parallel planes facing the camera but at different distances have their normals pointing in the same direction, and thus the ink shader cannot differentiate between them and does not draw a complete outline. Screenshot of what I’m talking about below, you can see the issue in the normals buffer at the bottom-left:
I thought a solution might be to modify the normalGen.sha shader that generates the normals buffer to take depth/distance into account. I thought the way to do that was to parent the “light” node that the shader uses to calculate the normals to the camera (so that it will move with the player), and then calculate how “bright” the normals-color should be based on the distance between the vertices and the light node. I attempted to do this with mixed results… Here’s the shader code:
//Cg
//
//Cg profile arbvp1 arbfp1
void vshader(float4 vtx_position : POSITION,
float4 vtx_normal : NORMAL,
out float4 l_position : POSITION,
out float3 l_color : TEXCOORD0,
out float l_bright: COLOR,
uniform float4x4 mat_modelproj,
uniform float4 mspos_light,
uniform float4x4 itp_modelview)
{
l_position=mul(mat_modelproj, vtx_position);
l_color=(float3)mul(itp_modelview, vtx_normal);
float distance = length(mspos_light - vtx_position);
l_bright = (100.0 / distance);
}
void fshader(float3 l_color: TEXCOORD0,
in float l_bright: COLOR,
out float4 o_color: COLOR)
{
l_color = normalize(l_color);
l_color = l_color/2;
o_color.rgb = ( l_color + float4(0.5, 0.5, 0.5, 0.5) ) * l_bright;
o_color.a = 1;
}And here’s what happens on the screen:
The second image is the same scene from the first image with the camera moved forward, into the corner formed by the right-hand wall. If you compare the normals buffers in the bottom left of each image, you’ll see that the wall facing the camera is much lighter in the second image as compared to the first, but the wall on the right remains at the same brightness no matter how far away it is.
I’m sure there’s something wrong with the way I’m calculating the brightness/distance or the way I’m applying it to the out color in the fragment shader, but my brain is fuzzled by the shader code. A big part of this, I suspect, has to do with how little I paid attention in math class back in high school. It would also be nice if there were some way to get some kind of console output from the shaders…
Anyway, any shader experts want to help me out?
edit: here’s some direct links to the images since it looks like the right-hand edges got cut off. I think you can still see what I’m talking about, though.
http://i.imgur.com/V6I0655.png
http://i.imgur.com/gMh8YJW.png
http://i.imgur.com/q490bXR.png
