Panda3, video and motion capture

Hi Everybody,

I am a newby to panda3. My area is computer vision and machine learning.

I have several newby questions:

Is there a way to pipe video / audio into the panda3 virtual reality?

Is there a way to use motion capture from video to control character movement?

To what degree can you control the facial expressions on the avatars?

Can panda3 take advantage of speed benefits of GPUs?

Thanks for any answers. Exact pointers to literature (URLs) are most appreciated.


about pipeing… no idea but it should be possible. and even if it’s not…it’s possible to add :stuck_out_tongue:

about motion capturing… well panda cant capture the motion BUT it can display the motion you capture. so if you have a programm which can capture the data then panda can display it since panda can manipulate the joints in a character. check the sample called “looking and gripping” where joints are manipulated based on your mousemovement.
of you wanna use non-realtime motion capturing… even easyer. did it myself.

about facial expressions. panda supports motiontargets afaik. aswell as joint-animation. basically thats all you need to get facial animations working, once it works you can controll it as you want. thought setting it up might not be that easy.

about speed. panda uses DX/openGL for rendering so it utilizes the gpu to it’s best. despite the fact that you code your stuff in python the engine itself is still c++ and surprisingly fast. it’s very fast and offers lots of functions to detect and fix bottlenecks. really great stuff.

By “pipe audio and video data,” do you mean like reading from a webcam and a microphone? If so, then the answer is — it’s not implemented yet, but I just wrote a big part of the code. There’s already an API to hook into, it’s merely a question of hooking in the appropriate drivers.

I’m just looking for some webcam capture, computer vision and augmented reality.
This thread was going the right way into that direction in 2007. We’re almost in 2012… My question is what’s happened since?

Is there today a simple (and efficient) way to this?

thank you and congratulations for sharing your work,

webcams are now pretty well supported, i have no information on the audio part tho.

Yeah, WebcamVideo works fine on both Linux and Windows now.