最新On the Noise Model of Support Vector Machines.pdfVIP

On the Noise Model of Support Vector Machines Regression Massimiliano Pontil, Sayan Mukherjee, and Federico Girosi Center for Biological and Computational Learning, MIT 45 Carleton Street E25-201, Cambridge, MA 02142, USA {pontil,sayan,girosi}@ Abstract. Support Vector Machines Regression (SVMR) is a learn- ing technique where the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called the ǫ-Insensitive Loss Function (ILF), which is similar to loss functions used in the field of robust statistics. The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of the ILF is not clear. In this paper the use of the ILF is justified under the assumption that the noise is additive and Gaussian, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori ap- proach. It applies not only to the ILF, but to a much broader class of loss functions. 1 Introduction Support Vector Machines Regression (SVMR) [8, 9] has a foundation in the framework of statistical learning theory and classical regularization theory for function approximation [10, 1]. The main difference between SVMR and classical regularization is the use of the ǫ-Insensitive Loss Function (ILF) to measure the empirical error. The quadratic loss function c