06-02-2013, 04:17 PM
Non-rigid 3D Face Shape Reconstruction using a Genetic Algorithm
[attachment=49894]
Abstract
This paper proposes a method for reconstructing
non-rigid 3D shapes from noisy 2D shapes. The proposed
method estimates the 3D shape bases and projection matrices,
exploiting low-rank constraints. Then the method finds the
optimal coefficients for linear combinations of 3D shape bases to
represent non-rigid 3D shapes using a genetic algorithm, and
refines the 3D shape bases and the projection matrices using
gradient descent techniques. The method reconstructed correct
non-rigid 3D shapes in the presence of noise. The results can be
used in many areas including animation, motion capture and
non-rigid 3D object tracking.
INTRODUCTION
ODEL-based face tracking is widely used because it
can simultaneously keep track of the positions of faces,
head poses and facial expressions. Many model-based face
tracking algorithms have been proposed, including Active
Shape Model (ASM) [1] and Active Appearance Model
(AAM) [2]. Mattews et al. showed that up to approximately
six times as many shape parameters may be needed by a 2D
model to represent the same phenomenon represented by a
3D model. 3D shape models can also represent self-occlusion
caused by rotation [3]. Therefore 3D model-based face
tracking algorithms are more compact and less sensitive to
variations in shape than 2D model-based face tracking
algorithms.
3D RECONSTRUCTION USING GENETIC ALGORITHM
Generating 2D face shapes
Many algorithms for reconstructing non-rigid 3D shapes
use 2D shapes obtained by 2D model-based object tracking
algorithms, including ASM and AAM. To implement 2D
model-based object tracking algorithms, the 2D shapes of the
object are required to model non-rigid motion. The 2D face
shapes are usually marked by hand because it is the most
accurate method so far. However many objects that have
non-rigid motion, including human faces, have no texture that
specifies the exact position of feature points. Therefore the
positions were guessed by a human and this process added
noise to the 2D shapes.
The objective of our research was to reconstruct non-rigid
3D face shapes for 3D model-based face tracking algorithms
directly from the noisy 2D shapes required to model non-rigid
motion for 2D model-based object tracking algorithms. By
directly using the noisy 2D shapes, we can reconstruct
non-rigid 3D shapes without implementing 2D model-based
object tracking algorithms to obtain 2D shapes.
Analysis of the noise sensitivity
We performed additional experiments to measure the
accuracy of the reconstructed 3D shape in the presence of
noise. In the experiment, we added Gaussian noise with zeromean
to the 2D shapes of the mug while varying standard
deviation σ of the noise. We repeated the 3D reconstruction
process three times, and took the average of the RMS error
between the ground truth and the reconstructed 3D shapes for
each σ . The average RMS error increased almost linearly
over σ when all feature points were corrupted by Gaussian
noise (Fig. 7.).
CONCLUSION
In this study, we reconstructed a non-rigid 3D face shape
model from noisy 2D face shapes using low-rank constraints
and a GA. The 2D face shapes were obtained by manually
marking feature points on 2D pictures of faces with various
facial expressions and head poses. The proposed algorithm
does not require either a 2D model-based tracking algorithm
or an image streams. The 3D shapes reconstructed by the
proposed method had accurate depth information even in the
presence of noise. The resulting 3D shapes can be used in
many areas including animation, motion capture and 3D
object tracking.
[attachment=49894]
Abstract
This paper proposes a method for reconstructing
non-rigid 3D shapes from noisy 2D shapes. The proposed
method estimates the 3D shape bases and projection matrices,
exploiting low-rank constraints. Then the method finds the
optimal coefficients for linear combinations of 3D shape bases to
represent non-rigid 3D shapes using a genetic algorithm, and
refines the 3D shape bases and the projection matrices using
gradient descent techniques. The method reconstructed correct
non-rigid 3D shapes in the presence of noise. The results can be
used in many areas including animation, motion capture and
non-rigid 3D object tracking.
INTRODUCTION
ODEL-based face tracking is widely used because it
can simultaneously keep track of the positions of faces,
head poses and facial expressions. Many model-based face
tracking algorithms have been proposed, including Active
Shape Model (ASM) [1] and Active Appearance Model
(AAM) [2]. Mattews et al. showed that up to approximately
six times as many shape parameters may be needed by a 2D
model to represent the same phenomenon represented by a
3D model. 3D shape models can also represent self-occlusion
caused by rotation [3]. Therefore 3D model-based face
tracking algorithms are more compact and less sensitive to
variations in shape than 2D model-based face tracking
algorithms.
3D RECONSTRUCTION USING GENETIC ALGORITHM
Generating 2D face shapes
Many algorithms for reconstructing non-rigid 3D shapes
use 2D shapes obtained by 2D model-based object tracking
algorithms, including ASM and AAM. To implement 2D
model-based object tracking algorithms, the 2D shapes of the
object are required to model non-rigid motion. The 2D face
shapes are usually marked by hand because it is the most
accurate method so far. However many objects that have
non-rigid motion, including human faces, have no texture that
specifies the exact position of feature points. Therefore the
positions were guessed by a human and this process added
noise to the 2D shapes.
The objective of our research was to reconstruct non-rigid
3D face shapes for 3D model-based face tracking algorithms
directly from the noisy 2D shapes required to model non-rigid
motion for 2D model-based object tracking algorithms. By
directly using the noisy 2D shapes, we can reconstruct
non-rigid 3D shapes without implementing 2D model-based
object tracking algorithms to obtain 2D shapes.
Analysis of the noise sensitivity
We performed additional experiments to measure the
accuracy of the reconstructed 3D shape in the presence of
noise. In the experiment, we added Gaussian noise with zeromean
to the 2D shapes of the mug while varying standard
deviation σ of the noise. We repeated the 3D reconstruction
process three times, and took the average of the RMS error
between the ground truth and the reconstructed 3D shapes for
each σ . The average RMS error increased almost linearly
over σ when all feature points were corrupted by Gaussian
noise (Fig. 7.).
CONCLUSION
In this study, we reconstructed a non-rigid 3D face shape
model from noisy 2D face shapes using low-rank constraints
and a GA. The 2D face shapes were obtained by manually
marking feature points on 2D pictures of faces with various
facial expressions and head poses. The proposed algorithm
does not require either a 2D model-based tracking algorithm
or an image streams. The 3D shapes reconstructed by the
proposed method had accurate depth information even in the
presence of noise. The resulting 3D shapes can be used in
many areas including animation, motion capture and 3D
object tracking.