Essentials of Pattern Recognition An Accessible Approach

Part II introduces domain-independent feature extraction methods, and this chapter presents principal component analysis (PCA). We start from its motivation, using an example. Then we gradually discover and develop the PCA algorithm: starting from zero dimensions, then one dimension, and finally the complete algorithm. We analyze its errors in ideal and practical conditions, and establish the equivalence between maximum variance and minimum reconstruction error. Two important issues are also discussed: when we can use PCA, and the relationship between PCA and SVD (singular value decomposition).