核岭回归(KRR)简介.pdf

Kernel ridge Regression Max Welling Department of Computer Science University of Toronto 10 King’s College Road Toronto, M5S 3G5 Canada welling@cs.toronto.edu Abstract This is a note to explain kernel ridge regression. 1 Ridge Regression Possibly the most elementary algorithm that can be kernelized is ridge regression. Here our task is to find a linear function that models the dependencies between covariates and response variables , both continuous. The classical way to do that is to minimize the quadratic cost, (1) However, if we are going to work in feature space, where we replace , there is an clear danger that we overfit. Hence we need to regularize. This is an important topic that will return in future classes. A simple yet effective way to regularize is to penalize the norm of . This is sometimes called “weight-decay”. It remains to be determined how to choose . The most used algorithm is to use cross validation or leave-one-out estimates. The total cost function hence becomes, (2)