Pca Paras Joshi.docx

PCA 1. Principal Component Analysis. 2. The aim is to reduce the dimensions of the predictor variables. 3. Suppose the dataset has n features, the aim is to reduce the number of features to k such that k<=n.

From 2 dimensions to 1 dimension

4. Mathematically we try to project the n dimensional data to a linear subspace of k dimensions. 5. Before applying PCA:  Perform mean normalization of all features so that their mean is reduced to zero(mandatory).  Do feature scaling by dividing the normalized features by either range or standard deviation of the corresponding feature. 6. Algorithm: Compute the covariance matrix.  Compute Eigen Vectors of the covariance matrix.  Find the Ureduce matrix which gives the n dimensions on which entire data can be projected. Select the first k dimensions.  Now any example zi=(Ureduce)T.xi. 7. Reconstruction from compressed representation:Xiapprox= (Ureduce).zi

8. To select best possible k ensure:Variance with k dimensions >=0.99 Variance with n dimensions

9. Applications of PCA: Reduces memory/disk space required to store data.  Speeds up the learning algorithm.  Makes Visualization easier. 10. Use of PCA to prevent overfitting is bad use of PCA, since some data is lost in the process, use regularization instead.