prml模式识别机器学习第6章内核方法1训练数据保留还丢弃.pdfVIP

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 6: Kernel Methods Kernel Methods 1:Training Data, Keep or Discard? ◼ parameter-based methods ◼ Parameter vector w, discard training data t ◼ linear parametric models for regression and classification ◼ neura works (nonlinear models ) ◼ memory-based methods(Non-parametric) ◼ the training data points(or a subset) ◼ the Parzen probability density model ◼ nearest neighbours Kernel Methods 2 ◼ Kernel function ◼ dual representation ◼ predictions are based on linear combinations of a kernel function ◼ kernel function are evaluated at the training data points ◼ a fixed nonlinear feature space map φ(x) Kernel Methods 3 ◼ The simplest example ◼ φ(x) = x T ◼ k(x, x) = x x ◼ the linear kernel ◼ kernel trick ◼ re ce scalar product with some other choice of kernel ◼ a nonlinear variant of PCA ◼ nearest-neighbour classifiers ◼ the kernel Fisher discriminant Kernel Methods 4 ◼ Some Property ◼ symmetric function: k(x, x’) = k(x’,x) ◼ stationary kernel: k(x, x’) = k(x −x’) ◼ homogeneous kernel: k(x, x’) = k(||x −x’||) linear regression model review ◼ linear regression model Dual Representations 1 ◼ a regularized sum-of-squares error function(parameter is w) parameter is w the design matrix from parameter w to an Dual Representations 2 ◼ a regularized sum-of-squares error function(parameter is a) Gr trix set the gradient to zero: Dual Representations 3 ◼ prediction for a new input x linear regression model inverting N ×N matrix instead of