algorithms for non-negative matrix factorization.pdfVIP

Algorithms for Non-negative Matrix Factorization Daniel D. Lee* H. Sebastian Seung*t *BelJ Laboratories tDept. of Brain and Cog. Sci. Lucent Technologies Massachusetts Institute of Technology Murray Hill, NJ 07974 Cambridge, MA 02138 Abstract Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition for multivariate data . Two different multi- plicative algorithms for NMF are analyzed. They differ only slightly in the multiplicative factor used in the update rules. One algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence. The monotonic convergence of both algorithms can be proven using an auxiliary func- tion analogous to that used for proving convergence of the Expectation- Maximization algorithm. The algorithms can also be interpreted as diag- onally rescaled gradient descent, where the rescaling factor is optimally chosen to ensure convergence. 1 Introduction Unsupervised learning algorithms such as principal components analysis and vector quan- tization can be understood as factorizing a data matrix subject to different constraints. De- pending upon the constraints utilized, the resulting factors can be shown to have very dif- ferent representational properties. Principal components analysis enforces only a weak or- thogonality constraint, resulting in a very distributed representation that uses cancellations to generate variability [1, 2]. On the other hand , vector quantization uses a hard winner- take-all constraint that results in clustering the data into mutually exclusi