[SOLVED] CG - Geometry Instancing - Normals look wrong?


I’m still on a quest to get instancing working nicely, but noticed when I applied some lighting to the instances it didn’t look quite correct. (So close yet so far!)

Taking a look at a normal buffer, the normals of instanced smileys look different from a non-instanced smiley.

Example image:

As you can see in the normal buffer, theres a bit of a difference. (You can see the weird lighting effects on the instanced geometry as well)

The shader code that governs this seems to be this line:

// Change the coordinate system of IN.vtx_normal from the object (model) space to clipping space.
OUT.o_normal    = mul(mul(to_apiview,inverse_transposed(offset[IN.l_id])),float4(IN.vtx_normal,0)).xyz;

I’ve had quite a search and couldn’t find concrete example of calculating normals for geometry instances, and the shader maths above is a bit above my skills. I’ve tried some things like using the built in transpose, inverse functions instead of using inverse_transposed, but that didn’t seem to help much, I’ve also tried calculating the normals as per the GPU gems page here: http.developer.nvidia.com/GPUGem … ter03.html but couldn’t get it working either :frowning:

Any tips or pointers would be great :slight_smile:

Heres some example cutdown code that shows the normal buffers and the shader I’m using:

# Authors: Federico Perazzi, Deepak Murali
# Last Updated: 28/08/2013
# This tutorial will demonstrate how to use Hardware Based Instancing
# Requires advancedInputs.cg shader

from pandac.PandaModules import loadPrcFileData

loadPrcFileData('', 'sync-video 1')
loadPrcFileData('', 'show-frame-rate-meter 1')
loadPrcFileData('', 'show-buffers 1')
loadPrcFileData('', 'basic-shaders-only 0')

import random, sys, os, math
import direct.directbase.DirectStart
from   direct.showbase.DirectObject import DirectObject
from pandac.PandaModules  import *
from direct.gui.DirectGui import OnscreenText
from direct.task.Task     import Task
from pandac.PandaModules import Vec3,Vec4,Point3
from direct.filter.CommonFilters import CommonFilters

class World(DirectObject):

  def __init__(self):

   # Check if the Graphics Hardware supports this program
    assert base.win.getGsg().getSupportsCgProfile("gp4vp"),\
      "Profile gp4vp is not supported by the hardware."
    assert base.win.getGsg().getSupportsCgProfile("gp4fp"),\
      "Profile gp4fp is not supported by the hardware."
    assert base.win.getGsg().getSupports2dTextureArray(),\
        "2D-Texture Array is not supported by the hardware."
    assert base.win.getGsg().getSupportsGeometryInstancing(),\
        "Geometry Instancing is not supported by the hardware."

    # This creates the on screen title
    self.title = OnscreenText(text="Panda3D: Simple Instancing",
                              style=1, fg=(1,1,1,1),
                              pos=(0.57,0.90), scale = .07)

    # Set the background color
    base.setBackgroundColor(.66, .714, 1)

    # Hit ESC to exit

    # Setup buffers
    filters = CommonFilters(base.win, base.cam)
    # How many instances should be shown?
    # Limited to 255
    # If you want to change this, remember to also change the number in the shader!
    # TODO: See if we can add / remove instances without changing the size of the arrays
    self.instanceNum = 200

    # Create the dummy NodePath root. This NodePath is passed to the PandAI
    # Library that will update the position of the nodes. The model matrices
    # of each node of this graph, will be used to feed the shader, that, through
    # Hardware Based Instancing, will render each node.
    self.dummyNodePathRoot = NodePath('dummy-path')
    self.dummyNodePath = []

    # Attach nodes to the dummy path. No models are loaded.
    for i in range(self.instanceNum):
      index = int(i)
    # Create another Nodepath. This nodePath contains the model that will
    # be instatiated multiple times directly inside the shader at the positions
    # defined by the dummy NodePath's nodes.
    self.orginalNode = loader.loadModel('smiley.egg.pz')

    self.testNode = loader.loadModel('smiley.egg.pz')

    # Send array of textures to the shader.
    # Use this to set a different texture for each instance
    # self.orginalNode.setShaderInput('mtex_0',self.ralphTextures)

    # Custom bounds, as we don't want to cull all the instances if the original
    # scrolls out of view.
    # We should cull the instances properly though based on if the instance
    # is visible or not.

    # Add task to update the shader
    # This means you can update the dummyNodePath postions and the
    # instances will move as well
    taskMgr.add(self.IssueShaderParameters, "ShaderParameters")

  def IssueShaderParameters(self, task):

    # Set the instance count based on the current number of instances

    # Retrieve the view matrix needed in the shader.
    # Need to update this every frame to allow for camera movement
    self.viewMatrix = self.orginalNode.getMat(base.camera)

    # Retrieve model matrices from the dummy NodePath
    self.modelMatrices = [nodePath.getMat(self.dummyNodePathRoot) for nodePath in self.dummyNodePath]
    # Compute the modelview matrix for each node
    self.modelViewMatrices = [UnalignedLMatrix4f(modelMatrix * self.viewMatrix) for modelMatrix in self.modelMatrices]

    # Send array of 4x4 matrices to the shader.
    self.orginalNode.setShaderInput('offset', self.modelViewMatrices)

    return Task.cont
w = World()


    //Cg profile arbvp1 arbfp1

    // Total number of instances
    #define NUMBER_OF_INSTANCES 200

    // Number of textures we pass in the texture array
    #define NUMBER_OF_TEXTURES 15

    // This matrix represent a ninty degrees rotation around the X axis.
    // It will used to transform a vertex from the Panda3D coordinate system
    // to the OpenGL coordinate system (right handed - Y Up)
    const float4x4 to_apiview = {{1.0, 0.0, 0.0, 0.0},
                  {0.0, 0.0, 1.0, 0.0},
                       {0.0,-1.0, 0.0, 0.0},
                       {0.0, 0.0, 0.0, 1.0}};

    // Compute the inverse of the transpose of an affine
    // matrix composed by a rotation and a translation.
    float4x4 inverse_transposed(float4x4 matrix) {
      float4x4 R = matrix;
      float4x4 T = {{1.0, 0.0, 0.0, 0.0},
               {0.0, 1.0, 1.0, 0.0},
               {0.0, 0.0, 1.0, 0.0},
               {-R[0].w, -R[1].w, -R[2].w, 1.0}};
      R[0].w = 0;
      R[1].w = 0;
      R[2].w = 0;

      return mul(T, R);

    // Vertex data entering the Vertex Shader
    struct VertexDataIN {
      float4 vtx_position  :POSITION;  // object-space
      float3 vtx_normal    :NORMAL;    // object-space
      float4 vtx_color     :COLOR0;
      float3 vtx_texcoord0 :TEXCOORD0;
      int l_id             :INSTANCEID;

    // Vertex data coming out of the Vertex Shader
    struct VertexDataOUT{
      float4 o_position :POSITION;  // clip-space
      float3 o_normal   :TEXCOORD1; // eye-space
      float3 o_texcoord :TEXCOORD3;

    // Vertex Shader
    void vshader(in  VertexDataIN IN,
                 out VertexDataOUT OUT,
                 uniform float4x4 mat_projection,
            uniform float4x4 offset[NUMBER_OF_INSTANCES])
      // Change the coordinate system of IN.vtx_position from the object (model) space to clipping space.
      OUT.o_position  = mul(mul(mul(mat_projection, to_apiview),offset[IN.l_id]),IN.vtx_position);
      // Change the coordinate system of IN.vtx_normal from the object (model) space to clipping space.
      OUT.o_normal    = mul(mul(to_apiview,inverse_transposed(offset[IN.l_id])),float4(IN.vtx_normal,0)).xyz;

      // Set the Z coordinates depending on the 2D texture, stored in
      // the 2D texture array, that should be binded to the model.
      OUT.o_texcoord = IN.vtx_texcoord0;

    // Fragment Shader
    void fshader(in VertexDataOUT vIN,
            sampler2D tex_0,
            sampler2DArray mtex_0,
            out float4 o_color :COLOR,
			out float4 o_normal :COLOR1)
       // Fetch Texture Color from a texture array
       o_color= tex2DARRAY(mtex_0,vIN.o_texcoord);
       // Or use the existing texture
       o_color= tex2D(tex_0,vIN.o_texcoord);
	   o_normal.rgb = vIN.o_normal;
	   o_normal.a = 1;

Should you really be applying the offset to your normal transformation matrix? The clip-space normals are expected to be the same for each instance, aren’t they?

Ok, well removing the offset and replacing the line with this worked much better:

OUT.o_normal = mul((float3x3)tpose_view_to_model, IN.vtx_normal.xyz).xyz;

But the normals don’t really face the camera properly like the non-instanced geometry. Hmm.

Does removing the random H and/or scale fix that? If so, you’ll have to find a way to integrate that into the calculation of OUT.o_normal, since the shader has no way of knowing the transformation properties of all your instances. In the worst case, you’ll have to pass a per-instance version of tpose_view_to_model using an array.

Indeed, removing the random H makes the instanced normals look the same as the non-instanced one.

Soooo, I had a look at panda3d.org/manual/index.php … ate_Spaces and used the tpos_world_to_model, and multplied that by the transformations. Which seems to have worked.

(Ignore the bad compression)

My lighting still looks a bit off compared to the non-instanced one, but the normal buffer looks great, so I think thats an issue with another part of my code. :slight_smile:

So heres the output with a random set of 50 lights generated, which works much better then the original. Thanks for the help!


Great, glad you got it working. Looks nice!