Incremental and decremental support vector machine learning.pdf英文资料.pdfVIP

Incremental and Decremental Support Vector Machine Learning Gert Cauwenberghs Tomaso Poggio CLSP, ECE Dept. CBCL, BCS Dept. Johns Hopkins University Massachusetts Institute of Technology Baltimore, MD 21218 Cambridge, MA 02142 gert@ tp@ Abstract An on-line recursive algorithm for training support vector machines, one vector at a time, is presented. Adiabatic increments retain the Kuhn- Tucker conditions on all previously seen training data, in a number of steps each computed analytically. The incremental procedure is re- versible, and decremental “unlearning” offers an efficient method to ex- actly evaluate leave-one-out generalization performance. Interpretation of decremental unlearning in feature space sheds light on the relationship between generalization and geometry of the data. 1 Introduction Training a support vector machine (SVM) requires solving a quadratic programming (QP) problem in a number of coefficients equal to the number of training examples. For very large datasets, standard numeric techniques for QP become infeasible. Practical techniques decompose the problem into manageable subproblems over part of the data [7, 5] or, in the limit, perform iterative pairwise [8] or component-wise [3] optimization. A disadvantage of these techniques is that they may give an approximate solution, and may require many passes through the dataset to reach a reasonable level of convergence. An on-line alterna- tive, that formulates the (exact) solution for
 training data in terms of that for
data and one new data point, is presented here. T