机器学习十大算法：EM.pdf

Chapter 5 EM Geoffrey J. McLachlan and Shu-Kay Ng Contents 5.1 Introduction 93 5.2 Algorithm Description 95 5.3 Software Implementation 96 5.4 Illustrative Examples 97 5.4.1 Example 5.1: Multivariate Normal Mixtures 97 5.4.2 Example 5.2: Mixtures of Factor Analyzers 100 5.5 Advanced Topics 103 5.6 Exercises 105 References 113 Abstract The expectation-maximization (EM) algorithm is a broadly applicable approach to the iterative computation of maximum likelihood (ML) estimates, useful in a variety of incomplete-data problems. In particular, the EM algorithm simplifies considerably the problem of fitting finite mixture models by ML, where mixture models are used to model heterogeneity in cluster analysis and pattern recognition contexts. The EM algorithm has a number of appealing properties, including its numerical stability, simplicity of implementation, and reliable global convergence. There are also extensions of the EM algorithm to tackle complex problems in various data mining applications. It is, however, highly desirable if its simplicity and stability can be preserved. 5.1 Introduction The expectation-maximization (EM) algorithm has been of considerable interest in recent years in the development of algorithms in various application areas such as data mining, machine learning, and pattern recognition [20, 27, 28]. The seminal paper of Dempster et al. [8] on the EM algorithm greatly stimulated interest in the use of finite mixture distributions to model heterogeneous data. This is because the fitting of 93 © 2009 by Taylor Francis Group, LLC 94 EM mixture models by maximum likelihood (ML) is a classic example of a problem that is simpl