An Introduction of Independent Component Analysis (ICA)：独立分量分析（ICA）的介绍.ppt

An Introduction of Independent Component Analysis (ICA) Xiaoling Wang Jan. 28, 2003 What Is ICA? Application: blind source separation (BSS) and deconvolution Motivation: “cocktail party problem” Assumption: two people speaking simultaneously, two microphones in different locations Principles of ICA Algorithm Assumption: sources are statistically independent Goal: it seeks a transformation to coordinates in which the data are maximally statistically independent Definition: Hierarchy of ICA Models Independence of Sources Independence: the pdf of sources can be factorized Nongaussian is independent Seek the separation matrix W which maximize the nongaussianity of the estimated sources Measures of Nongaussianity Kurtosis (4th order cumulant): Subgaussian: negative kurtosis Supergaussian: positive kurtosis Negentropy: Measures of Nongaussianity (Cont.) Mutual information: FastICA Algorithm Basic form: Choose an initial (e.g. Random) weight vector Let Let If not converged, go back to step 2 For several units: decorrelation Let Let Nonlinear ICA Model: Existence and uniqueness of solutions There always exists an infinity of solutions if the space of the nonlinear mixing functions is not limited Post-nonlinear problem Algorithms for Nonlinear ICA Burel’s approach: neural solution, known nonlinearities on unknown parameters Krob Benidir: high order moments, polynomial mixtures Pajunen et al.: SOMs, locally factorable pdf Pajunen et al.: GTM(generative topographic mapping), output distribution matches the known source distributions Post nonlinear mixtures: Taleb Jutten: adaptive componentwise separation Yang et al.: two-layer neural network Puntonet et al.: nonlinearities are a power function, geometrical considerations * * Mixing process Demixing process – mixing matrix, – separation matrix Nonlinear mixing Linear mixing Classical ICA Flexible Source model Infomax Non-stationary sources Non-stationary mixing No noise Factor Analysis R diagonal Gaussian sou