Procedurally Generating animated 3D Models (and textures)?

Forgot to update the code, i dont assign the r,g,b and a separately anymore. I dont know why, but with PIL its still faster, well ‘comparably’.

Well, this sure was the most difficult thing I had done before.
But it wasnt impossible. I got most of the stuff figured out and working, only things are perhaps assigning the bones to vertices and setting weights part, and setting the animation transforms for each frame. that part is a bit weird.
[Procedural character skeleton hierarchy generation?)
drwr posted a snippet there:

from direct.directbase.DirectStart import *
from pandac.PandaModules import *
from direct.actor.Actor import Actor

# Create a character.
ch = Character('simplechar')
bundle = ch.getBundle(0)
skeleton = PartGroup(bundle, '<skeleton>')

# Create the joint hierarchy.
root = CharacterJoint(ch, bundle, skeleton, 'root',
                      Mat4.identMat())
hjoint = CharacterJoint(ch, bundle, root, 'hjoint',
                        Mat4.translateMat(Vec3(10, 0, 0)))
vjoint = CharacterJoint(ch, bundle, hjoint, 'vjoint',
                        Mat4.translateMat(Vec3(0, 0, 10)))

# Create a TransformBlendTable, listing all the different combinations
# of joint assignments we will require for our vertices.
root_trans = JointVertexTransform(root)
hjoint_trans = JointVertexTransform(hjoint)
vjoint_trans = JointVertexTransform(vjoint)

tbtable = TransformBlendTable()
t0 = tbtable.addBlend(TransformBlend())
t1 = tbtable.addBlend(TransformBlend(root_trans, 1.0))
t2 = tbtable.addBlend(TransformBlend(hjoint_trans, 1.0))
t3 = tbtable.addBlend(TransformBlend(vjoint_trans, 1.0))
t4 = tbtable.addBlend(TransformBlend(hjoint_trans, 0.7, vjoint_trans, 0.3))

# Create a GeomVertexFormat to represent the vertices.  We can store
# the regular vertex data in the first array, but we also need a
# second array to hold the transform blend index, which associates
# each vertex with one row in the above tbtable, to give the joint
# assignments for that vertex.
array1 = GeomVertexArrayFormat()
array1.addColumn(InternalName.make('vertex'),
                3, Geom.NTFloat32, Geom.CPoint)
array2 = GeomVertexArrayFormat()
array2.addColumn(InternalName.make('transform_blend'),
                 1, Geom.NTUint16, Geom.CIndex)
format = GeomVertexFormat()
format.addArray(array1)
format.addArray(array2)
aspec = GeomVertexAnimationSpec()
aspec.setPanda()
format.setAnimation(aspec)
format = GeomVertexFormat.registerFormat(format)

# Create a GeomVertexData and populate it with vertices.
vdata = GeomVertexData('vdata', format, Geom.UHStatic)
vdata.setTransformBlendTable(tbtable)
vwriter = GeomVertexWriter(vdata, 'vertex')
twriter = GeomVertexWriter(vdata, 'transform_blend')

vwriter.addData3f(0, 0, 0)
twriter.addData1i(t1)

vwriter.addData3f(10, 0, 0)
twriter.addData1i(t2)

vwriter.addData3f(10, 0, 10)
twriter.addData1i(t3)

vwriter.addData3f(8, 0, 2)
twriter.addData1i(t4)

# Be sure to tell the tbtable which of those vertices it will be
# animating (in this example, all of them).
tbtable.setRows(SparseArray.lowerOn(vdata.getNumRows()))

# Create a GeomTriangles to render the geometry
tris = GeomTriangles(Geom.UHStatic)
tris.addVertices(2, 3, 1)
tris.closePrimitive()
tris.addVertices(1, 3, 0)
tris.closePrimitive()

# Create a Geom and a GeomNode to store that in the scene graph.
geom = Geom(vdata)
geom.addPrimitive(tris)
gnode = GeomNode('gnode')
gnode.addGeom(geom)
ch.addChild(gnode)

# Now create the animation tables.  (We could also load just this part
# from an egg file, if we already have a compatible table ready.)
bundle = AnimBundle('simplechar', 5.0, 10)
skeleton = AnimGroup(bundle, '<skeleton>')
root = AnimChannelMatrixXfmTable(skeleton, 'root')

hjoint = AnimChannelMatrixXfmTable(root, 'hjoint')
table = [10, 11, 12, 13, 14, 15, 14, 13, 12, 11]
data = PTAFloat.emptyArray(len(table))
for i in range(len(table)):
    data.setElement(i, table[i])
hjoint.setTable(ord('x'), CPTAFloat(data))

vjoint = AnimChannelMatrixXfmTable(hjoint, 'vjoint')
table = [10, 9, 8, 7, 6, 5, 6, 7, 8, 9]
data = PTAFloat.emptyArray(len(table))
for i in range(len(table)):
    data.setElement(i, table[i])
vjoint.setTable(ord('z'), CPTAFloat(data))

wiggle = AnimBundleNode('wiggle', bundle)

# Finally, wrap the whole thing in a NodePath and pass it to the
# Actor.
np = NodePath(ch)
anim = NodePath(wiggle)
a = Actor(np, {'simplechar' : anim})
a.reparentTo(render)
a.setPos(0, 50, 0)
a.loop('simplechar')

Look at this part:

# Now create the animation tables.  (We could also load just this part
# from an egg file, if we already have a compatible table ready.)
bundle = AnimBundle('simplechar', 5.0, 10)
skeleton = AnimGroup(bundle, '<skeleton>')
root = AnimChannelMatrixXfmTable(skeleton, 'root')

hjoint = AnimChannelMatrixXfmTable(root, 'hjoint')
table = [10, 11, 12, 13, 14, 15, 14, 13, 12, 11]
data = PTAFloat.emptyArray(len(table))
for i in range(len(table)):
    data.setElement(i, table[i])
hjoint.setTable(ord('x'), CPTAFloat(data))

Now I would expect to animate the joints for each frame with a Mat4, not a Python list converted to some kind of Array object.
I would like to know what the values in the table are. I would guess scalex,scaley,scalez, roth,rotp,rotr, posx,posy,posz, but there are like 10 members. What are they?
I think completely don’t understand what those objects are.

At least I managed to make the procedural geometry, textures and bones, though.

I still dont really undertsand how to assign bones and weights to vertices…
Im just guessing these functions and classes are for that:

transformblendtable.addBlend(TransformBlend(trans1, weight1, trans2, weight2, ...)) 

Should I just create a trans (JointVertexTransform) for each bone? is there some kind of limit to the amount of bones (blends) per vertice?

bump…

The different channels that may be set on an AnimChannelMatrixXfmTable are the same as allowed in an egg file, and documented in eggSyntax.txt:

      i, j, k - scale in x, y, z directions, respectively
      a, b, c - shear in xy, xz, and yz planes, respectively
      r, p, h - rotate by roll, pitch, heading
      x, y, z - translate in x, y, z directions

You would use this structure (and a JointVertexTransform) only if you want to create animation tables for pre-canned animation sequences. If you’re animating your actors dynamically, you would create a UserVertexTransform instead. Both can be used interchangeably.

Yes, you need a different transform object for each bone. There’s no hard limit to the number of bones per vertex, but there could be performance implications, of course. It just means more math per frame.

David

OK.
The animations arent really dynamically generated as the geometry.

Im having problems assigning weights to vertices, I’ll make an example code and post here.

OK, the whole code is a bit long and complex, so I’ll just tell the parts where Im not sure what is being done/ if Im doing it right.

The ‘Actors’ are composed of multiple parts.
each part is turned into a geom, then geomnode, then nodepath (so I can have nodepath functions like setTexture for each part).

I create a Character object, get a handle to its ‘bundle’ (which im not sure what is for), ‘PartGroup’ and create a ‘TransformBlendTable’.

ch = Character('character')
bundle = ch.getBundle(0)
skeleton = PartGroup(bundle, '<skeleton>') 
tbtable = TransformBlendTable() 

I then make a nodepath from the Character object:

nodepath =  NodePath(ch)

then

for i in parts:
    i.reparentTo(nodepath)

where “parts” is a list of the nodepaths i generated previously.
or can I only parent the nodes (geomnodes), not nodepaths?

for i in parts:
    ch.addChild(i.node())

and finally

actor = Actor(nodepath)

to be able to play animations on it.

joints (bones) are created like this:

joint = CharacterJoint(ch, bundle, skeleton, name, matrix)
jointtrans = JointVertexTransform(joint)
jointtranslist.append(jointtrans)

Now Im pretty sure I assign the “weights” incorrectly, or the geomnodes altogether, as moving the actor doesnt move the joints and transforming a joint doesnt affect the vertices.
So how do you assign weights?
I do this for every vertice:

table = tbtable.addBlend(TransformBlend(jointtranslist[0], weight1, jointtranslist[1], weigth2...))

Oh and I have 2 arrays like in your sample code:

array = GeomVertexArrayFormat() #vertex, normal, etc
array2 = GeomVertexArrayFormat()
array2.addColumn(InternalName.make('transform_blend'), 1, Geom.NTUint16, Geom.CIndex) 

and

format = GeomVertexFormat()
format.addArray(array)
format.addArray(array2)

# this object describes how the vertex animation, if any, represented in a GeomVertexData is encoded.Vertex animation includes soft-skinned skeleton animation and morphs, and might be performed on the CPU by Panda, or passed down to the graphics backed to be performed on the hardware (depending on the hardware's advertised capabilities). Changing this setting doesn't by itself change the way the animation is actually performed, this just specifies how the vertices are set up to be animated. 
aspec = GeomVertexAnimationSpec()
aspec.setPanda() # specifies that vertex animation is to be performed by Panda. 
format.setAnimation(aspec)
	
# Finally, before you can use your new format, you must register it:
format = GeomVertexFormat.registerFormat(format)
	
# Once you have a GeomVertexFormat, registered and ready to use, you can use it to create a GeomVertexData.
vdata = GeomVertexData('vdata', format, Geom.UHStatic)

Please tell me if Im doing something the wrong way, Im pretty sure I am.
I think I posted the necessary code, I didnt post how I assign vertices, normals and such, and create primitives which I assign to geoms, because they all work perfectly fine and I have no issues there.

The PartBundle is the root of the PartGroup hierarchy; which is to say, the root of the skeleton. That’s all it means.

A NodePath is a handle to its underlying node. Reparenting a NodePath means (almost) the same thing as reparenting its node. The difference between your two versions of reparentTo() vs. addChild() is what happens to the NodePath you created: in the former, it becomes the handle to the new path you created; in the latter, it is unchanged.

This doesn’t sound like a weighting issue to me. You’re not seeing any animation at all? It sounds like it’s not seeing the updates to your transforms.

Try using a UserVertexTransform for now, just to eliminate the complexity of the animation tables. Try setting the transform explicitly on the UserVertexTransform. It should move the character’s vertices. If it doesn’t, then something’s wrong with the TransformBlendTable or the GeomVertexData–somehow the vertices aren’t associated with your transform the way you think they should be.

Also, just use on transform per blend for now, with a weight of 1.0. That’s the simplest relationship, and it means the transform exactly operates on the vertices assigned to the blend.

Be sure you put the blend index in the appropriate vertex column; that’s how the vertices are matched to the blend.

David

I dont have animations yet, I just got a handle to a joint (exposeJoint())and transformed a joint like that and then moved the Actor, in the latter the bones didnt move with the actor. What I mean by that is the visual representations of the bones didnt move: [Visualise actor bones (joints))

Did you mean controlJoint() instead of exposeJoint()? controlJoint() is used to get a node that you can transform to move the joint. exposeJoint() is the opposite, it gets a node that moves when the joint moves due to its own animation, but transforming the node that exposeJoint() returns will have no effect on anything.

David

Yeah, I got the names wrong in this topic.
Anyway, Ill try and post

okay, coming back to this, the difference in the code is only in following:

gnode = GeomNode('gnode')
gnode.addGeom(geom)
ch.addChild(gnode)

I, to be able to assign textures, have to make a nodepath from the geomnode:

gnode = GeomNode('gnode')
gnode.addGeom(geom)
np = NodePath(gnode)
np.setTexture(tex)
ch.addChild(np.node())

I don’t think theres a difference here, right?
I’m rewriting the rest of the code, but just in case.

EDIT: The bones still remain in the same place after I rotate the Actor. I’ll try to assign the weigths properly and see if it persists, but I think this isnt related.

You’re right, there’s no difference between those two code blocks, other than the fact that the texture gets applied in the second. You’re saying that one of them works and the other one doesn’t?

I don’t know what you mean when you say the bones “remain in the same place”. Are they not parented to the Actor? All nodes should move with their parent, regardless of any transform weighting.

David

Actually, looks like they both don’t.
Here is your code with the joint renderer snippet:

from direct.directbase.DirectStart import *
from pandac.PandaModules import *
from direct.actor.Actor import Actor

from random import *

# Create a character.
ch = Character('simplechar')
bundle = ch.getBundle(0)
skeleton = PartGroup(bundle, '<skeleton>')

# Create the joint hierarchy.
root = CharacterJoint(ch, bundle, skeleton, 'root',
                      Mat4.identMat())
hjoint = CharacterJoint(ch, bundle, root, 'hjoint',
                        Mat4.translateMat(Vec3(10, 0, 0)))
vjoint = CharacterJoint(ch, bundle, hjoint, 'vjoint',
                        Mat4.translateMat(Vec3(0, 0, 10)))

# Create a TransformBlendTable, listing all the different combinations
# of joint assignments we will require for our vertices.
root_trans = JointVertexTransform(root)
hjoint_trans = JointVertexTransform(hjoint)
vjoint_trans = JointVertexTransform(vjoint)

tbtable = TransformBlendTable()
t0 = tbtable.addBlend(TransformBlend())
t1 = tbtable.addBlend(TransformBlend(root_trans, 1.0))
t2 = tbtable.addBlend(TransformBlend(hjoint_trans, 1.0))
t3 = tbtable.addBlend(TransformBlend(vjoint_trans, 1.0))
t4 = tbtable.addBlend(TransformBlend(hjoint_trans, 0.7, vjoint_trans, 0.3))

# Create a GeomVertexFormat to represent the vertices.  We can store
# the regular vertex data in the first array, but we also need a
# second array to hold the transform blend index, which associates
# each vertex with one row in the above tbtable, to give the joint
# assignments for that vertex.
array1 = GeomVertexArrayFormat()
array1.addColumn(InternalName.make('vertex'),
                3, Geom.NTFloat32, Geom.CPoint)
array2 = GeomVertexArrayFormat()
array2.addColumn(InternalName.make('transform_blend'),
                 1, Geom.NTUint16, Geom.CIndex)
format = GeomVertexFormat()
format.addArray(array1)
format.addArray(array2)
aspec = GeomVertexAnimationSpec()
aspec.setPanda()
format.setAnimation(aspec)
format = GeomVertexFormat.registerFormat(format)

# Create a GeomVertexData and populate it with vertices.
vdata = GeomVertexData('vdata', format, Geom.UHStatic)
vdata.setTransformBlendTable(tbtable)
vwriter = GeomVertexWriter(vdata, 'vertex')
twriter = GeomVertexWriter(vdata, 'transform_blend')

vwriter.addData3f(0, 0, 0)
twriter.addData1i(t1)

vwriter.addData3f(10, 0, 0)
twriter.addData1i(t2)

vwriter.addData3f(10, 0, 10)
twriter.addData1i(t3)

vwriter.addData3f(8, 0, 2)
twriter.addData1i(t4)

# Be sure to tell the tbtable which of those vertices it will be
# animating (in this example, all of them).
tbtable.setRows(SparseArray.lowerOn(vdata.getNumRows()))

# Create a GeomTriangles to render the geometry
tris = GeomTriangles(Geom.UHStatic)
tris.addVertices(2, 3, 1)
tris.closePrimitive()
tris.addVertices(1, 3, 0)
tris.closePrimitive()

# Create a Geom and a GeomNode to store that in the scene graph.
geom = Geom(vdata)
geom.addPrimitive(tris)
gnode = GeomNode('gnode')
gnode.addGeom(geom)
ch.addChild(gnode)

# Now create the animation tables.  (We could also load just this part
# from an egg file, if we already have a compatible table ready.)
bundle = AnimBundle('simplechar', 5.0, 10)
skeleton = AnimGroup(bundle, '<skeleton>')
root = AnimChannelMatrixXfmTable(skeleton, 'root')

hjoint = AnimChannelMatrixXfmTable(root, 'hjoint')
table = [10, 11, 12, 13, 14, 15, 14, 13, 12, 11]
data = PTAFloat.emptyArray(len(table))
for i in range(len(table)):
    data.setElement(i, table[i])
hjoint.setTable(ord('x'), CPTAFloat(data))

vjoint = AnimChannelMatrixXfmTable(hjoint, 'vjoint')
table = [10, 9, 8, 7, 6, 5, 6, 7, 8, 9]
data = PTAFloat.emptyArray(len(table))
for i in range(len(table)):
    data.setElement(i, table[i])
vjoint.setTable(ord('z'), CPTAFloat(data))

wiggle = AnimBundleNode('wiggle', bundle)

# Finally, wrap the whole thing in a NodePath and pass it to the
# Actor.
np = NodePath(ch)
anim = NodePath(wiggle)
a = Actor(np, {'simplechar' : anim})
a.reparentTo(render)
a.setTwoSided(True)
a.setPos(0, 50, 0)
a.setP(-45)
a.loop('simplechar')

def renderBones(actor, part, parentNode = None, indent = ""):
    if isinstance(part, CharacterJoint):
        np = actor.exposeJoint(None, 'modelRoot', part.getName())

        if parentNode and parentNode.getName() != "root":
            lines = LineSegs()
            lines.setThickness(3.0)
            lines.setColor(random(), random(), random())
            lines.moveTo(0, 0, 0)
            lines.drawTo(np.getPos(parentNode))
            lnp = parentNode.attachNewNode(lines.create())
            lnp.setBin("fixed", 40)
            lnp.setDepthWrite(False)
            lnp.setDepthTest(False)

        parentNode = np

    for child in part.getChildren():
        renderBones(actor, child, parentNode, indent + "  ")

renderBones(a, a.getPartBundle('modelRoot'), None) 


run()

Now it works here, I rotated the actor and the bones rotated with it. But they don’t in my case.

So, you’re saying my code works and yours doesn’t. What do you want me to do?

David

Well I found that out in my last post, so nothing.

OK, i think i found the difference and the problem.

My actor consists of multiple pieces (geom(nodes)), unlike in the sample.
I did

actor.clearModelNodes()
actor.flattenStrong()

and the bones now follow the actor. But I’m afraid if flattening an actor like that will bring up other problems.

Right now I have geoms, which I assign to geomNodes and then make a nodePath to set a texture on it, I then add each GeomNode to the Character object with

character.addChild(nodePath.node())

one at a time. Might there be a problem with adding more than 1 geomNode to a Character like that? Ive read that a geomNode can have more than 1 geoms, maybe I am only allowed to add all the geoms to a single geomNode and add it as child to the Character?

Also, this is the code which sets the weights and assign bones to vertices, right?

vdata = GeomVertexData('data', format, Geom.UHStatic)
transform = GeomVertexWriter(vdata, 'transform_blend')
tbtable = TransformBlendTable()

transform.addData1i(tbtable.addBlend(TransformBlend(bonename_trans, weight)))

assuming I have made a JointVertexTransform for each bone and pass it to “bonename_trans” and weigth is the weight value from 0.0 to 1.0

There’s no problem with having multiple GeomNodes and multiple Geoms under a Character node. I still don’t see anything wrong in the code you’re posting, and I have difficulty conceiving of a situation in which a character doesn’t animate, but flattening it works.

But, you are using the Actor interface, which is designed for models loaded from egg files, not for models generated at runtime. Could be something in that interface is not aware of all of the joints you’ve added or something? I still don’t understand what you mean about the bones not following the actor.

David

but Im using animations loaded from a file, I thought I needed the Actor interface to be able to set an animation. …youre using an Actor too in the snippets, right?

Well, I’m just grasping at straws here. Unless you show me a complete (but small) program that demonstrates the problem, I have nothing really to go on.

You never have to use Actor. That’s all just a convenience layer around the lower-level Character interface, which provides all the functionality you need.

David