Large Margin Classification Using the Perceptron Algorithm-英文文献.pdf

Large Margin Classification Using the Perceptron Algorithm Yoav Freund Robert E. Schapire ATT Labs 180 Park Avenue Florham Park, NJ 07932-0971 USA fyoav, schapireg@ December 14, 1998 Abstract We introduce and analyze a new algorithm for linear classification which combines Rosen- blatts perceptron algorithm with Helmbold and Warmuths leave-one-out method. Like Vap- niks maximal-margin classifier, our algorithm takes advantage of data that are linearly sep- arable with large margins. Compared to Vapniks algorithm, however, ours is much simpler to implement, and much more efficient in terms of computation time. We also show that our algorithm can be efficiently used in very high dimensional spaces using kernel functions. We performed some experiments using our algorithm, and some variants of it, for classifying im- ages of handwritten digits. The performance of our algorithm is close to, but not as good as, the performance of maximal-margin classifiers on the same probl em, while saving significantly on computation time and programming effort. 1 Introduction One of the most influential developments in the theory of machine learning in the last few years is Vapniks work on support vector machines (SVM) [18]. Vapniks analysis suggests the follow- ing simple method for learning complex binary classifiers. First, use some fixed mapping to map the instances into some very high dimensional space in which the two classes are linearly separable. Then use quadratic programming to find the vector that classifies all the data correctly an