Online Learning with Kernels：（在线学习与内核）.pdfVIP

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 100, NO. 10, OCTOBER 2010 1 Online Learning with Kernels Jyrki Kivinen, Alexander J. Smola, and Robert C. Williamson, Member, IEEE Abstract— Kernel based algorithms such as support vector with the number of observations. Depending on the loss func- machines have achieved considerable success in various problems tion used [4], this will happen in practice in most cases. Thus in the batch setting where all of the training data is available in the complexity of the estimator used in prediction increases advance. Support vector machines combine the so-called kernel trick with the large margin idea. There has been little use of these linearly over time (in some restricted situations this can be methods in an online setting suitable for real-time applications. reduced to logarithmical cost [5] or constant cost [6], yet with In this paper we consider online learning in a Reproducing linear storage requirements). Clearly this is not satisfactory for Kernel Hilbert Space. By considering classical stochastic gradient genuine online applications. descent within a feature space, and the use of some straight- Third, the training time of batch and/or incremental update forward tricks, we develop simple and computationally efficient algorithms for a wide range of problems such as classification, algorithms typically increases superlinearly with the number regression, and novelty detection. of observations. Incremental update algorithms [7] attempt In addition to allowing the exploitation of the kernel trick to overcome