Principal Component Analysis (PCA) Singular （主成分分析(PCA)单数）.pdfVIP

Outline 鈥 Principal Component Analysis (PCA) 鈥 Singular Value Decomposition (SVD) 鈥 Multi-Dimensional Scaling (MDS) 鈥 Non-linear extensions: 鈥 Kernel PCA 鈥 Isomap Iyad Batal PCA 鈥 PCA: Principle Component Analysis (closely related to SVD). 鈥 PCA finds a linear projection of high dimensional data into a lower dimensional subspace such as: o The variance retained is maximized. o The least square reconstruction error is minimized. Iyad Batal Some PCA/SVD applications 飪 LSI: Latent Semantic Indexing. 飪 Kleinberg/Hits algorithm (compute hubs and authority scores for nodes). 飪 Google/PageRank algorithm (random walk with restart). 飪 Image compression (eigen faces) 飪 Data visualization (by projecting the data on 2D). Iyad Batal PCA PCA steps: transform an 饊亖 脳 饊亼 matrix 肀 into an 饊亖 脳 饊仛 matrix 肀孃 鈥 Centralized the data (subtract the mean). 1 饊亣 鈥 Calculate the 饊亼 脳 饊亼 covariance matrix: C = 肀 肀 (different 饊亖鈭? notation from tutorial!!!) o 饊€俄睎,肀 = 1 饊亖 肀嬸€仦,肀 . 肀嬸€仦,肀 饊亖鈭? 饊仦=1 o 饊€俄睎,肀 (diagonal) is the variance of variable i. o 饊€俄睎,肀 (off-diagonal) is the covariance between variables i and j. 鈥 Calculate the eigenvectors of the covariance matrix (orthonormal). 鈥 Select m eigenvectors that correspond to the largest m eigenvalues to be the new basis. Iyad Batal Eigenvectors 鈥 If A is a square matrix, a non-zero vector v is an eigenvector of A if there is a scalar 位 (eigenvalue) such that 饊€答€仯 = 位饊仯