applying texture in cartoon shader(SOLVED)

when i saw the cartoon shader example, i really liked how it worked, finally i got to putting in my model in the shader to see how it would look like

well the model’s texture was completely white although i had a PNG image texture that was supposed to be on, heres the models

the top row is how it should look like with texture
the bottom row is the result of what i saw when i tried to apply my model and texture in the cartoon shader example

is there something i did wrong when trying to apply my own model into the cartoon shader?

Hello, you must be using custom shader(advanced cartoon sample), otherwise the texture would show. That´s a problem everyone interested in custom cartoon shader have(and i think the advanced sample need to be better)

In your custom shader(in the sample “lightgen.sha”) you need in vertex shader start:

in float2 vtx_texcoord0 : TEXCOORD0,
out float2 l_texcoord0 : TEXCOORD0,

between vertex and fragment shader

l_texcoord0 = vtx_texcoord0;

in fragment shader start

in float2 l_texcoord0 : TEXCOORD0,
uniform sampler2D tex_0 : TEXUNIT0,

and at the end of the the cg file

float4 texcolor = tex2D( tex_0, float2( l_texcoord0 ) );
o_color = texcolor;

Then you will have texture colors, but if you´re going to use vertex colors you´ll need another shader file for that.

Custom cg shaders are a bit tough, but you can do whateaver you imagine graphically speaking and which the end-user videocard can handle. I would advise you to use them only if the automatic one don´t give you what you want. If that´s the case you can read the following and/or PM me.
http://www.panda3d.org/manual/index.php/Shaders_and_Coordinate_Spaces
http://www.panda3d.org/manual/index.php/List_of_Possible_Cg_Shader_Inputs
http://www.panda3d.org/manual/index.php/Shader_Basics
http://code.google.com/p/p3dst/source/browse/trunk
http://http.developer.nvidia.com/CgTutorial/cg_tutorial_appendix_e.html

im not sure of what you mean by the terms “vertex shader” or “fragment shader”

i suppose they are parts fo the file “lightingGen” but im not sure which part you refer to at the mention of those terms

please clarify

EDIT:
i think i just realised that you meant the following terms within the lightingGen file, is it vshader and fshader?

i gave your solution a try and i got this

this is no different from if i just applied the model and texture in the basic shader

i seem to have lost the shading i wanted. is the shading what you refered to as vertex colors?

I dont know GPU programming myself, but I can guess that in the shader the shading isnt applied after applying the texture. Or the texture is applied on top of it.

That is if you have lightning set up correctly.

o_color = texcolor;

that only applies texture color.

change it to :

o_color = texcolor * l_brite;

no change, another thing i didnt mention is this error i get even with the change suggested by ynjh_jo

:util(warning): Adjusting global clock's real time by 4.72739e-006 seconds.
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0000: syntax error, unexpected integer constant, expecting identifier or reserved word "operator" or template identifier or type identifier at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0501: type name expected at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0000: syntax error, unexpected integer constant, expecting identifier or reserved word "operator" or template identifier or type identifier at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0501: type name expected at token "<int-const>"
:Actorwarning: arrow3.egg is not a character!
:util(warning): Adjusting global clock's real time by -0.308374 seconds.
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0000: syntax error, unexpected integer constant, expecting identifier or reserved word "operator" or template identifier or type identifier at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0501: type name expected at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0000: syntax error, unexpected integer constant, expecting identifier or reserved word "operator" or template identifier or type identifier at token "<int-const>"
:gobj(error): /e/yr3/my stuff/Oban Star Racers game proposal/Panda3D_model_turntable/lightingGen.sha: (34) : error C0501: type name expected at token "<int-const>"

is there soemthing i didnt do right in changing the lightingGen file? like wrong format or soemthing like that, that would give such an rror?

post your shader.

//Cg
//
//Cg profile arbvp1 arbfp1

void vshader(in float2 vtx_texcoord0 : TEXCOORD0,
             out float2 l_texcoord0 : TEXCOORD0,
             float4 vtx_position   : POSITION,
             float3 vtx_normal     : NORMAL,
             float4 vtx_color      : COLOR,
             out float4 l_position : POSITION,
             out float4 l_brite    : TEXCOORD0,
             out float4 l_color    : COLOR,
             uniform float4 mspos_light,
             uniform float4x4 mat_modelproj)
{
  l_position = mul(mat_modelproj, vtx_position);
  float3 N = normalize(vtx_normal);
  float3 lightVector = normalize(mspos_light - vtx_position);
  l_brite = max(dot(N,lightVector), 0);
  l_color = vtx_color;
  l_texcoord0 = vtx_texcoord0;
}


void fshader(in float2 l_texcoord0 : TEXCOORD0,
             uniform sampler2D tex_0 : TEXUNIT0,
             float4 l_brite     : TEXCOORD0, 
             float4 l_color     : COLOR,
             out float4 o_color : COLOR)
{
  if (l_brite.x<0.5) l_brite=0.8;
  else l_brite=1.2;
  o_color=l_brite * l_color;
  float 4 texcolor = tex2D( tex_0, float2( l_texcoord0) );
  o_color = texcolor * l_brite;
}
float 4 texcolor = tex2D( tex_0, float2( l_texcoord0) );

should be float4.

You’re still binding l_brite to TEXCOORD0, which is also bound to the texture coordinates. Not good, remove one.

//Cg
//Cg profile arbvp1 arbfp1

void vshader(in float2 vtx_texcoord0 : TEXCOORD0,
             out float2 l_texcoord0 : TEXCOORD0,
             float4 vtx_position   : POSITION,
             float3 vtx_normal     : NORMAL,
             float4 vtx_color      : COLOR,
             out float4 l_position : POSITION,
             out float4 l_brite,
             out float4 l_color    : COLOR,
             uniform float4 mspos_light,
             uniform float4x4 mat_modelproj)
{
  l_position = mul(mat_modelproj, vtx_position);
  float3 N = normalize(vtx_normal);
  float3 lightVector = normalize(mspos_light - vtx_position);
  l_brite = max(dot(N,lightVector), 0);
  l_color = vtx_color;
  l_texcoord0 = vtx_texcoord0;
}


void fshader(in float2 l_texcoord0 : TEXCOORD0,
             uniform sampler2D tex_0 : TEXUNIT0,
             float4 l_brite,
             float4 l_color     : COLOR,
             out float4 o_color : COLOR)
{
  if (l_brite.x<0.5) l_brite=0.8;
  else l_brite=1.2;
  float4 texcolor = tex2D( tex_0, float2( l_texcoord0) ); 
  o_color = texcolor * l_brite; 
}

For lighting in your custom shader, being it vertex color shader or texture colors shader you´ll need to add:
At your game code:

        	plight = self.render.attachNewNode("point light")
        	plight.setPos(30,-40,0)

		#ambient light 
        	alight = 1
                
        	self.render.setShaderInput("alight", Vec4(alight,alight,alight,0))
        	self.render.setShaderInput("plight", plight)

The set shader input send the variables above which the shader will need(if you want ambient color and/or point light); then at vertex shader start:

	in float4 vtx_position : POSITION,
	in float3 vtx_normal : NORMAL,
	out float3 l_myposition : TEXCOORD2,
	out float3 l_mynormal : TEXCOORD3,
	uniform float4x4 mat_modelproj

Between vertex shader and fragment shader you´ll need:

	l_position = mul( mat_modelproj, vtx_position );
	l_myposition = vtx_position.xyz;
	l_mynormal = normalize( vtx_normal );

“mul” means you will multiply mat_modelproj with every vertex position, to have no distortions in your graphic; any time you can use a CG function like mul, max and etc instead of doing the calculations the usual way, you should do so. At fragment shader start:

	in float3 l_myposition : TEXCOORD2,
	in float3 l_mynormal : TEXCOORD3,
	uniform float4 mspos_plight,
        uniform float3 k_alight

The special point above is mspos_plight, which is the variable we passed earlier from python, were saying it is the light position. k_alight is the other variable sent earlier, we will make it act as a ambient light. Then:

	float4 result;
	float3 lightpos = mspos_plight.xyz;
	float3 modelpos = l_myposition;
	float3 normal = normalize( l_mynormal );
	float3 direction = normalize( lightpos - modelpos );
	float light = max( dot( normal, direction ), 0 ) ;
	//here is where the light gets "Cartonized"
	if ( light < 0.5 ) light = 0.7;
	else light = 1;
	result = texcolor;
	result.rgb *= k_alight.rgb;
	result *= light;
	o_color = result;

If in cg you don´t code something, it will not exist in your program graphic, here we build our lighting. As far as I understand normalize(l_mynormal) turn whateaver value given into -1,0,1 range. normalize(lightpos - modelpos) gives you a vector, a vector is a point with a direction; then at “max” you get the greatest value from the ones passed to “max”; dot is scalar dot product, which means if normal is (1,0,-1) and direction(0,1,1) then the result will be calculated as result = (10)+(01)+(-1*1). Dot product is used to discover the value between two vectors. Then you get that and simplify, making light have only 2 values with an if. Then you just multiply the ambient light with the result(which already contains texcolor), and do the same for the point light.

At least thats how I understood it, finding info about cg is a bit hard without asking someone who knows about it to teach you. I myself I´m still learning.

that is alot to digest, sorry to say but that is an overwhelming amount of information

anyway, with ynjh_jo’s latest solution, the shader kinda works, but it still looks kinda weird, not at all what i expected of a merger between the shader and texture

if all else fails i guess ill just stick with the basic cartoon line shader, although it would be best to have the advanced cartoon shader

It doesn’t look correct at all. :laughing:
How if you try it with Ralph ? Looks OK here.

Also, try this :
instead of unbinding l_brite from TEXCOORD0, try bind it to TEXCOORD1 or any unused slot.

ill revive the topic next when im free

just a small question, in a game, which do you find more important, even if its just a small near insignificant percentage of a percentage:

which do you feel in a game is MORE important?
gameplay(fun game mechanics)
control(smoothe natural feel)
aesthetic(looks)

in order of significance :
control
gameplay
arts

Iaperen, if you really want to understand custom shaders, you will need to learn a lot more than what I wrote, but you will eventally discover that by yourself.

About the three stated game aspects(you should do another topic for it), I agree with ynjh_jo.

thank you all for your contributions in helping me

and i am very glad to say that all that you have done was actally enough to get it to work

it was an EXTREMELY stupid typo on my part in editing the “lightingGen.sha” file which caused the shader to screw up

anyway, here is the final result, it works great, although the texture seems kind of see-through, but i think that is the faut of my texture, you all encountered this before although it wasn’t the fault of the texture? is it something i should be worried about?

and for anyone else interested in applying texture while using the cartoon shader, heres is the file to get you there, replace the stuff in “lightingGen.sha” with it, although its virtually the same as what ynjh_jo provided for me in his final solution, just for people who skip too fast ahead i guess.

//Cg
//
//Cg profile arbvp1 arbfp1

void vshader(in float2 vtx_texcoord0 : TEXCOORD0,
             out float2 l_texcoord0 : TEXCOORD0,
             float4 vtx_position   : POSITION,
             float3 vtx_normal     : NORMAL,
             float4 vtx_color      : COLOR,
             out float4 l_position : POSITION,
             out float4 l_brite,
             out float4 l_color    : COLOR,
             uniform float4 mspos_light,
             uniform float4x4 mat_modelproj)
{
  l_position = mul(mat_modelproj, vtx_position);
  float3 N = normalize(vtx_normal);
  float3 lightVector = normalize(mspos_light - vtx_position);
  l_brite = max(dot(N,lightVector), 0);
  l_color = vtx_color;
  l_texcoord0 = vtx_texcoord0;
}


void fshader(in float2 l_texcoord0 : TEXCOORD0,
             uniform sampler2D tex_0 : TEXUNIT0,
             float4 l_brite, 
             float4 l_color     : COLOR,
             out float4 o_color : COLOR)
{
  if (l_brite.x<0.5) l_brite=0.8;
  else l_brite=1.2;
  o_color=l_brite * l_color;
  float4 texcolor = tex2D( tex_0, float2( l_texcoord0) );
  o_color = texcolor * l_brite;
}
1 Like

I always thought the custom shader of Cartoon sample should be able to do exactly the same as the non shader one.
I would vote to update the sample with the shader from here.