15-12-2012, 02:19 PM
Gaussian Mixture Models
1Gaussian Mixture.pdf (Size: 81.07 KB / Downloads: 152)
Definition
A Gaussian Mixture Model (GMM) is a parametric probability density function represented as a weighted sum of Gaussian
component densities. GMMs are commonly used as a parametricmodel of the probability distribution of continuousmeasurements
or features in a biometric system, such as vocal-tract related spectral features in a speaker recognition system. GMM
parameters are estimated from training data using the iterative Expectation-Maximization (EM) algorithm or Maximum A
Posteriori (MAP) estimation from a well-trained prior model.
Maximum Likelihood Parameter Estimation
Given training vectors and a GMM configuration, we wish to estimate the parameters of the GMM, , which in some
sense best matches the distribution of the training feature vectors. There are several techniques available for estimating the
parameters of a GMM [4]. By far the most popular and well-established method is maximum likelihood (ML) estimation.
The aim of ML estimation is to find the model parameters which maximize the likelihood of the GMM given the training
data.
Maximum A Posteriori (MAP) Parameter Estimation
In addition to estimating GMM parameters via the EM algorithm, the parameters may also be estimated using Maximum
A Posteriori (MAP) estimation. MAP estimation is used, for example, in speaker recognition applications to derive speaker
model by adapting from a universal background model (UBM) [6]. It is also used in other pattern recognition tasks where
limited labeled training data is used to adapt a prior, general model.
Like the EMalgorithm, theMAP estimation is a two step estimation process. The first step is identical to the “Expectation”
step of the EM algorithm, where estimates of the sufficient statistics2 of the training data are computed for each mixture in
the prior model. Unlike the second step of the EM algorithm, for adaptation these “new” sufficient statistic estimates are then
combinedwith the “old” sufficient statistics fromthe prior mixture parameters using a data-dependentmixing coefficient. The
data-dependent mixing coefficient is designed so that mixtures with high counts of new data rely more on the new sufficient
statistics for final parameter estimation and mixtures with low counts of new data rely more on the old sufficient statistics for
final parameter estimation.