- Talking Heads:
My earlier work on analysis of
speaking lips started in
Alex Waibel's group
and continued
with
Steve Omohundro ,
Yochai Konig,
and others in
ICSI's Realization Group.
We were one of the first who demonstrated
significant performance improvements by
speechreading
over pure acoustic
recognition. With
Steve
I also demonstrated how to learn complex lip configurations
with a new technique called "Manifold Learning".
Building on these speech processing and lip-tracking ideas, I became
interested in new methods that use such analysis techniques to
synthesize lip and facial animations.
This resulted in a project
called
Video Rewrite,
that I developed with
Michele Covell,
Malcolm Slaney
and others at
Interval Research Corp.
A "video model" of a specific person is built automatically by processing
large amounts of example data. We demonstrated the photorealistic
synthesis of several people including
John F. Kennedy
saying sentences
he never said before.
-
Full Body Movements
:
More recent work is done on new visual learning and motion tracking
techniques and their application to recognition and animation of
full body movements (with
Jitendra Malik
and
Jerome A. Feldman
).
One important
contribution is a multi-level probabilistic architecture of human body
movements. We demonstrated how to learn hybrid dynamical
systems representation and recognize complex human gait movements in
unconstrained video sequences.
We also found a new technique which
draws on kinematic model constraints, and allows us to recover high
degree-of-freedom body configurations in complex video sequences. We
introduced the use of a novel mathematical technique, the product of exponential
maps and the twist representation.
This enables a differential method based on solving
linear systems that seeks to recover the kinematic degrees-of-freedom in
noise and complex self occluded configurations.
Besides lab sequences we are also able to recover and animate in 3D the
famous movements of Edweard Muybridge's motion studies from the last
century.
- Digital Humans: [COMING SOON]
- Learning in Vision:
Statistical Learning plays a central role in most of my research.
I explored several learning architectures including Neural Networks trained
for
lip-reading,
manifold learning for
visual tracking and interpolation,
hierarchical mixtures of experts applied to object recognition,
and
hybrid dynamical systems
applied to human gait classification.
[MORE TEXT COMES HERE]
Research Projects
Chris Bregler (bregler@cs.berkeley.edu)
Digital Muybridge: Annotation and Recognition of Human Movements