Animated Heads: From 3D Motion Fields to Action Descriptions
Published 1 January 2001Open access
Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos
Citations6
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The ability to estimate 3D motion fields points to a new framework for the study of action, which can serve as an intermediate representation which can be analyzed using geometrical and statisti cal tools for the purpose of extracting representations of generic human actions.
Abstract
We demonstrate a method to compute three-dimensional (3D) motion fields on a face. Twelve synchronized and calibrated cameras are positioned around a talking person, and observe its head in motion. We represent the head as a deformable mesh, which is fitted in a global optimization step to silhouette-contour and multi-camera stereo data derived from all images. The non-rigid displacement of the mesh from frame to frame, the 3D motion field, is determined from the spatiotemporal derivatives in all the images. We integrate these cues over time, thus producing an animated representation of the talking head. Our ability to estimate 3D motion fields points to a new framework for the study of action. The 3D motion fields can serve as an intermediate representation, which can be analyzed using geometrical and statistical tools for the purpose of extracting representations of generic human actions.
Keywords
Computer Science
Robot Vision
3,632 Citations1986Berthold K. P. Horn
This book presents a coherent approach to the fast-moving field of computer vision, using a consistent notation based on a detailed understanding of the image formation process, and will provide a useful and current reference for professionals working in the fields of machine vision, image processing, and pattern recognition.
Computer Vision and Image UnderstandingThe Visual Analysis of Human Movement: A Survey
1,767 Citations1999Dariu M. Gavrila
A number of promising applications are identified and an overview of recent developments in this domain is provided, including work on whole-body or hand motion and the various methodologies.
IEEE Transactions on Pattern Analysis and Machine IntelligenceThe visual hull concept for silhouette-based image understanding
1,438 Citations1994Aldo Laurentini
This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding and introduces the geometric concept of visual hull of a 3-D object, which is the maximal object silhouette-equivalent to S.
Human motion analysis: a review
1,408 Citations2002J.K. Aggarwal, Q. Cai
The paper gives an overview of the various tasks involved in motion analysis of the human body, and focuses on three major areas related to interpreting human motion: motion analysis involving human body parts, tracking of human motion using single or multiple cameras, and recognizing human activities from image sequences.
Computer Vision and Image UnderstandingHuman Motion Analysis: A Review
1,145 Citations1999J.K. Aggarwal, Q. Cai
IEEE Transactions on Pattern Analysis and Machine IntelligenceCoding, analysis, interpretation, and recognition of facial expressions
914 Citations1997Irfan Essa, Alex Pentland
Synthesizing realistic facial expressions from photographs
724 Citations2006Frédéric Pighin, Jamie Hecker +3 more
This work presents new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAnalysis and synthesis of facial image sequences using physical and anatomical models
538 Citations1993Demetri Terzopoulos, K. Waters
An estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images is developed and estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.
International Journal of Computer VisionRecognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion
485 Citations1997Michael J. Black, Yaser Yacoob
This paper explores the use of local parametrized models of image motion for recovering and recognizing the non-rigid and articulated motion of human faces and shows how expressions can be recognized from the local parametric motions in the presence of significant head motion.
Three-dimensional scene flow
433 Citations1999S. Vedula, Simon Baker +3 more
This work presents a framework for the computation of dense, non-rigid scene flow from optical flow and shows that multiple estimates of the normal flow cannot be used to estimate dense scene flow directly without some form of smoothing or regularization.
Making faces
390 Citations2006Brian Guenter, Cindy Grimm +3 more
The primary emphasis of the work is not compression but the use of a novel method to compress the geometric data based on principal components analysis, which eliminates much of the variance in the image data due to motion, which increases compression ratios.
Making faces
257 Citations1998Brian Guenter, Cindy Grimm +3 more
A novel method to compress the geometric data based on principal components analysis is investigated and it is found that an-imations reconstructed from 512x512 pixel textures look good at data rates as low as 240 Kbits per second.
Shape and motion carving in 6D
108 Citations2002S. Vedula, Simon Baker +2 more
Separability of pose and expression in facial tracking and animation
89 Citations2002B. Bascle, Andrew Blake
This work explores the application of facial tracking to automated re-animation in real-time, at video field-rate, using two Indy desktop workstations, and proposes a solution to solve explicitly, at each timestep, for pose and expression variables.
Lecture notes in computer scienceEstimating coloured 3D face models from single images: An example based approach
89 Citations1998Thomas Vetter, Volker Blanz
A method to derive 3D shape and surface texture of a human face from a single image using a general flexible 3D face model which is “learned” from examples of individual 3D-face data (Cyberware-scans).
Lecture notes in computer scienceLearning dynamics of complex motions from image sequences
76 Citations1996D. Reynard, Andrew Wildenberg +2 more
The range of shape and motion models is extended in two significant ways, one to model jointly the random variations in shape arising within an object-class and those occuring during object motion and the second to address the tracking of coupled objects such as head and lips.
Integrated 3D scene flow and structure recovery from multiview image sequences
59 Citations2002Yunhao Zhang, Chandra Kambhamettu
A novel system which integrates 3D scene flow and structure recovery in order to complement each other's performance, and does not assume rigidity of the scene motion, thus allowing for non-rigid motion in the scene.
Computer Vision and Image UnderstandingModel-Based Joint Motion and Structure Estimation from Stereo Images
51 Citations1997Sotiris Malassiotis, M.G. Strintzis
An algorithm is proposed that uses models for object surfaces and their motion and estimates the model parameters using the image intensity matching criterion and a regularization scheme using a coarse to fine strategy is employed.
Vision ResearchThe Ouchi illusion as an artifact of biased flow estimation
41 Citations2000Cornelia Fermüller, Robert Pless +1 more
It is explained that the cause of the illusion lies in the statistical difficulty of integrating local one-dimensional motion signals into two-dimensional image velocity measurements, and the process of optical flow computations should not be carried out in isolation but in conjunction with the higher level processes of 3D motion estimation, segmentation and shape computation.
Computer Vision and Image UnderstandingAnimated Heads from Ordinary Images: A Least-Squares Approach
39 Citations1999Pascal Fua, C. Miccio
This work shows that it can effectively fit arbitrarily complex animation models to noisy data extracted from ordinary face images, using of a set of progressively finer control triangulations and takes advantage of three complementary sources of information: stereo data, silhouette edges, and 2D feature points.
Dense range flow from depth and intensity data
23 Citations2002H. Spies, Bernd Jähne +1 more
The combined use of intensity and depth information greatly helps in the estimation of the local 3D movements (range flow) of moving surfaces in both: a local total least squares algorithm, and an iterative global variational technique.