06-05-2011, 10:13 AM
ABSTRACT
We present an approach to estimate the poses of human headsin natural scenes. The essential features for estimating the headpose are the positions of the prominent facial features relativeto the position of the head. We have developed a highdimensional,randomly sparse representation of a human faceusing a simpli£ed facial feature model. The representationtransforms a raw face image into a vector representing howwell the image matches large number of randomly-posed andshaped head models. This transformation is designed to collectsalient features of the face image that is useful to estimate thepose, while suppressing any irrelevant variations of face appearance.The relation between the sparse representation andthe pose is learned using the SVR (Support Vector Regression).The sparse representation combined with SVR is shown to estimatethe pose more quickly and accurately than SVR appliedto raw images.
1. INTRODUCTION
One of the main hurdles in face recognition is obtainingrobustness to variations in facial pose. Many existing facerecognitionsystems yield good results when comparing facesin frontal pose, but their performance drops off markedly asone of the faces moves toward pro£le [8]. Recently, however,some systems have been proposed which explicitly compensatefor facial pose, and yield vastly better results [1, 2]. Thesesystems require an estimate of the facial pose in each image asinput, which motivates us to examine the problem of obtainingsuch an estimate automatically. Ultimately, when given animage of a single face, we will need to estimate all six poseparameters: x and y location, scale, yaw (rotation around theneck, from left pro£le to right pro£le), pitch (rotation up anddown), and roll (rotation in the image plane). A seventh parameter,focal-length of the camera lense, may or may not berequired. At present, however, we concentrate only on £ndingthe two rotations that are out of the image plane . yaw andpitch . in images of faces that have been manually rotated toupright and scaled and translated to a canonical size and position.Although we present results on the PIE database [9]for comparison with other systems, our main emphasis is onunsystematic, natural images, with maximal variation in pose,expression, lighting, background, image quality, etc.Quite a few methods for facial pose estimation have beenproposed [3, 5, 4]. Of particular relevance to our new methodare [5] and [4]. In [5], images are projected into a linear subspaceobtained by applying PCA to images of faces in differentposes. This reduces the dimensionality of the data whilemaintaining much of the image variation due to pose. In [4],Support-Vector Regression (SVR) was applied to map imagesinto pose estimations.The method we propose here is based on two observations.First, intuitively, facial pose can be estimated by looking at thelocations of facial features in the image. Second, less intuitively,we can estimate the locations of those features by simplylooking for them in a random collection of places. Thislatter observation comes from work done on face tracking usingparticle systems [6]. These observations lead us to analyzeeach image by correlating it with a set of feature detectors computedfor a prespeci£ed, but randomly-generated, set of facialshapes and poses. The resulting vector is a sparse representationof the face and its pose. We train an SVR system to mapthese vectors into yaw and pitch angles.Thus, as in [5], we reduce the dimensionality of the problemwith a linear projection. However, our projection is basedon a priori knowledge of facial features, rather than statisticalanalysis of facial images. Like [4], we apply SVR to obtain ourangle estimates. However we do not apply it to the raw image.Our sparse representation of facial images is described inmore detail in Section 2, and the training of our SVR is describedin Section 3. This is followed, in Section 4, by a descriptionof experiments we have performed on natural imagesand on the PIE database. The results of the experiments (Section5) indicate that our sparse representation captures enoughinformation to yield good pose estimates, and improves bothperformance and speed over application of SVR to raw pixels.
2. SPARSE REPRESENTATION OF FACIAL IMAGES
We have designed a sparse representation of human face,which captures the unique signatures of a human face effectively,while facilitating the estimation of the head position andpose. The representation is a collection of projections to anumber of randomly generated possible con£guration of thehuman face. Each projection corresponds to a pose of the headalong with a con£guration of its facial features.
Download full report
http://www.umiacs.umd.edu/~hankyu/pubs/ICIP_pose.pdf
We present an approach to estimate the poses of human headsin natural scenes. The essential features for estimating the headpose are the positions of the prominent facial features relativeto the position of the head. We have developed a highdimensional,randomly sparse representation of a human faceusing a simpli£ed facial feature model. The representationtransforms a raw face image into a vector representing howwell the image matches large number of randomly-posed andshaped head models. This transformation is designed to collectsalient features of the face image that is useful to estimate thepose, while suppressing any irrelevant variations of face appearance.The relation between the sparse representation andthe pose is learned using the SVR (Support Vector Regression).The sparse representation combined with SVR is shown to estimatethe pose more quickly and accurately than SVR appliedto raw images.
1. INTRODUCTION
One of the main hurdles in face recognition is obtainingrobustness to variations in facial pose. Many existing facerecognitionsystems yield good results when comparing facesin frontal pose, but their performance drops off markedly asone of the faces moves toward pro£le [8]. Recently, however,some systems have been proposed which explicitly compensatefor facial pose, and yield vastly better results [1, 2]. Thesesystems require an estimate of the facial pose in each image asinput, which motivates us to examine the problem of obtainingsuch an estimate automatically. Ultimately, when given animage of a single face, we will need to estimate all six poseparameters: x and y location, scale, yaw (rotation around theneck, from left pro£le to right pro£le), pitch (rotation up anddown), and roll (rotation in the image plane). A seventh parameter,focal-length of the camera lense, may or may not berequired. At present, however, we concentrate only on £ndingthe two rotations that are out of the image plane . yaw andpitch . in images of faces that have been manually rotated toupright and scaled and translated to a canonical size and position.Although we present results on the PIE database [9]for comparison with other systems, our main emphasis is onunsystematic, natural images, with maximal variation in pose,expression, lighting, background, image quality, etc.Quite a few methods for facial pose estimation have beenproposed [3, 5, 4]. Of particular relevance to our new methodare [5] and [4]. In [5], images are projected into a linear subspaceobtained by applying PCA to images of faces in differentposes. This reduces the dimensionality of the data whilemaintaining much of the image variation due to pose. In [4],Support-Vector Regression (SVR) was applied to map imagesinto pose estimations.The method we propose here is based on two observations.First, intuitively, facial pose can be estimated by looking at thelocations of facial features in the image. Second, less intuitively,we can estimate the locations of those features by simplylooking for them in a random collection of places. Thislatter observation comes from work done on face tracking usingparticle systems [6]. These observations lead us to analyzeeach image by correlating it with a set of feature detectors computedfor a prespeci£ed, but randomly-generated, set of facialshapes and poses. The resulting vector is a sparse representationof the face and its pose. We train an SVR system to mapthese vectors into yaw and pitch angles.Thus, as in [5], we reduce the dimensionality of the problemwith a linear projection. However, our projection is basedon a priori knowledge of facial features, rather than statisticalanalysis of facial images. Like [4], we apply SVR to obtain ourangle estimates. However we do not apply it to the raw image.Our sparse representation of facial images is described inmore detail in Section 2, and the training of our SVR is describedin Section 3. This is followed, in Section 4, by a descriptionof experiments we have performed on natural imagesand on the PIE database. The results of the experiments (Section5) indicate that our sparse representation captures enoughinformation to yield good pose estimates, and improves bothperformance and speed over application of SVR to raw pixels.
2. SPARSE REPRESENTATION OF FACIAL IMAGES
We have designed a sparse representation of human face,which captures the unique signatures of a human face effectively,while facilitating the estimation of the head position andpose. The representation is a collection of projections to anumber of randomly generated possible con£guration of thehuman face. Each projection corresponds to a pose of the headalong with a con£guration of its facial features.
Download full report
http://www.umiacs.umd.edu/~hankyu/pubs/ICIP_pose.pdf