Text to Speech Talking Model

Finally have the model rigged for handling dynamic tts:
youtube.com/watch?v=YX5ZGQuraSE

I only know the two free tts engines with viseme support, Microsoft and Espeak. I use Microsoft one. Festival voices are much better but it does not come with viseme support by default.

very good job cheung - coupling this to a speech-to-text program we could code a tool like the one in the Valve’s Source engine 8)

Hey clcheung,

That is really cool. Can I ask how u exported the face deformations? I am keyframing and trying to export (from maya) and nothing is deforming!

Thanks,

Doros

I add some shape keys to a blender human head model and export it using a version of Chicken (with shape key support added). I then drive the shape keys accordingly in the tts program.

I have no idea about maya.

since you’re talking about and I’m actually interested in: do you mean to release the shapekey exporter code Cheung?

I am busy with something else. The shape key support is not yet finished and it only export the shape key information (but not the animation part). So one may need to control it by playing with the exported joints.

All my demos that uses shape keys does not use the Actor interface to do the animation. It is done through my own animation script (as in demomaster).

If you are interested, I can send to you or post it.

I’d already dug into - I like that choice

indeed I am - I’m actually studying a way to dynamically add some expression to my actors and I found the shapekey way the best - I’ll appreciate to have your chicken mod, at least to study it and I’ll possibly try to test and finish it either, but no guarantee to succeed though.

I’ve just updated it:

code.google.com/p/pandademomaster/downloads/list

It is based on R66. Search for “clcheung” in the script to locate my changes.

got it - thank you pal!

Ok, i am here to stop spamming blender2panda video tuts thread.

I know how to make animations with shapekeys in blender (and how to get makehuman model in blender), and now i will be able to export them.

I am very thankfull for this.
Could we get it merged into chicken?

Export the shapekeys in animation is not yet done in my provided code. May be someone can complete it and merge with chicken later.

Cheung I’m in need of your help cos I’m quite stuck
what I’m doing is studying the whole stuff and trying to port your additions regarding the shapekeys to a newer chicken version - I guess I got that but launching the exporter I got a console error. Before to dig into I need to know: do I gotta make some more prep before to export may actor (having 4 sk assigned) or is just enough to select it?

Are you able to export with my version before porting it ?
I will send you a simple blender file for your test.

it hangs as well:

[Object "Mesh"]                              
Shape keys information                       
Shape Key: 1 : wink : length 3797 : pos 0.100000 : vgroup Head
Traceback (most recent call last):                            
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 932, in button
    self.buttonFuncs[evt]()                                                                                
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 789, in fExport
    self.doExport()                                                                                         
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 446, in doExport
    character = Character(self.armature, self.meshes['armature'], self.mExtract and self.srb, self.animOnly, addTangents=self.doTangents)
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 1368, in __init__                           
    self.children = [Group(mesh=m, anim = True) for m in meshes]                                                                         
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 1277, in __init__                           
    self.shapeKeys = ShapeKeys(self.mesh, mesh.matrix)                                                                                   
  File "/home2/kartoffeln/Dloz/blender/.blender/scripts/chicken_exportR66_shapekey.py", line 142, in __init__                            
    vertexes = mesh.getVertsFromGroup(bKey.blocks[i].vgroup)                                                                             
AttributeError: group does not exist! 

I’m using the latest blender (2.49b)
I guess it is because you used a later version less picky but anyhow lemme know what you think looking the error log, if you got a quick idea, so I’ll keep investigate less in the dark

Can you email me the blender file ?

you may download it from my website here

“group does not exist!” seems referring to your shapekey definition. The vgroup in the wink shapekey is set to “Head”. But my code is case sensitive.

Change that to “head” then the problem gone.

But another script issue happens. Do you confirm if this blender file can be exported with original R66 ?

no I don’t - i found the R66 broken for blender249b (this is why I started porting it)
but the good news is that the Head vgroup label really was the matter (junk leftover I forgot to clean) and deleting it now it works in chicken R71 that I used for the porting - grab it here if you want
my concern is now that the exporter added kinda unexpected code:

<Dxyz> raged { -0.001147 0.000893 0.009561 }
<Dxyz> wink { -0.000686 0.000893 0.024460 }
<Dxyz> frown { -0.001470 0.000893 0.017983 }

I was expecting a tag not that Dxyz - is that ok for you or what?

I forget to mention there is a global flag:
dNormal = False

You shall set it to true if you want DNormal tag to be output. I set it to False for performance reason. For facial animation, the normal delta is actually not noticeable in general. I was planning to make this flag configurable during export.

You shall be able to play with this model with the exported joints already.

excuse my ignorance but what is “viseme”