12-03-2012, 02:49 PM
Principal Component Analysis
Lecture-pca.ppt (Size: 505.5 KB / Downloads: 39)
Introduced by Pearson (1901) and Hotelling (1933) to describe the variation in a set of multivariate data in terms of a set of uncorrelated variables
We typically have a data matrix of n observations on p correlated variables x1,x2,…xp
PCA looks for a transformation of the xi into p new variables yi that are uncorrelated
Transformation
What is a reasonable choice for the ?
Remember: We wanted a transformation that maximizes „information“
That means: captures „Variance in the data“
Some Features on Cov(X)
Cov(X) is a symmetric pxp matrix
The diagonal terms of Cov(X) are the variance of variables across observations.
The off-diagonal terms of Cov(X) are the covariance between variable vectors
Cov(X) captures the correlations between all possible pairs of measurements
In the diagonal terms, by assumption, large values correspond to interesting dynamics
In the off diagonal terms large values correspond to high redundancy
The principal Components of X are the Eigenvectors of Cov(X)
Assume, we can „manipulate“ X a bit: Lets call this Y
Y should be manipulated in a way that it is a bit more optimal than X was
What does optimal mean?