This is a partial port of work I presented as a SIGGRAPH 97 Technical Sketch . I have also hooked this graphical puppet up to Improv to give it directibly autonomous behavior. That combination has been presented as part of a travelling museum exhibit by the American Museum of the Moving Image. But even with what is shown here, you can begin to see the model begin to come to life.
In our Improv work, we showed how to make an embodied agent react with responsive facial expression, without using repetitive prebuilt animations, and how to mix those facial expressions to simulate shifting moods and attitudes. The result is real-time interactive facial animation with convincing emotive expressiveness.
The eventual goal of this research is to give computer/human interfaces the ability to represent the subtleties we take for granted in face to face communication, so that they can function as agents for an emotional point of view.
The demo you are seeing now is mostly an experiment to isolate the minimal number of facial expression elements that will produce a "convincing" impression of character and personality. Of course this is just a subset of the full range of expression the human face is capable of. Paul Ekman's pioneering work on the Facial Action Coding System gives a functional description of the full range of expression of which the human face is capable.
This was also a personal challenge I set for myself to see whether I could implement a 3D character in Java without using any 3D plug-ins (ie: doing the 3D rendering entirely in the Java applet itself). I've set it up with an interactive interface that's designed to be fun to play with - you can hit various hot keys to "play" the shifting facial expressions, much as one might play a piano.
NEW!! I now have a later version which has a much more complete 3D rendering, more options, and an interface that lets you do more. This later version was first shown in a public virtual reality exhibition in Rio de Janeiro, Brazil, in June 1999. You can try it in either English or the original Portuguese.
- Ken Perlin