机器学习十大算法：AdaBoost.pdf

Chapter 7 AdaBoost Zhi-Hua Zhou and Yang Yu Contents 7.1 Introduction 127 7.2 The Algorithm 128 7.2.1 Notations 128 7.2.2 A General Boosting Procedure 129 7.2.3 The AdaBoost Algorithm 130 7.3 Illustrative Examples 133 7.3.1 Solving XOR Problem 133 7.3.2 Performance on Real Data 134 7.4 Real Application 136 7.5 Advanced Topics 138 7.5.1 Theoretical Issues 138 7.5.2 Multiclass AdaBoost 142 7.5.3 Other Advanced Topics 145 7.6 Software Implementations 145 7.7 Exercises 146 References 147 7.1 Introduction Generalization ability, which characterizes how well the result learned from a given training data set can be applied to unseen new data, is the most central concept in machine learning. Researchers have devoted tremendous efforts to the pursuit of tech- niques that could lead to a learning system with strong generalization ability. One of the most successful paradigms is ensemble learning [32]. In contrast to ordinary machine learning approaches which try to generate one learner from training data, ensemble methods try to construct a set of base learners and combine them. Base learners are usually generated from training data by a base learning algorithm which can be a decision tree, a neural network, or other kinds of machine learning algorithms. Just like “many hands make light work,” the generalization ability of an ensemble is usually significantly better than that of a single learner. Actually, ensemble meth- ods are appealing mainly because they are able to boost weak learners , which are 127 © 2009 by Taylor Francis Group, LLC 128 AdaBoost slight