SVM 机器学习.pptVIP

Non-linear SVMs Mathematically Dual problem formulation: The solution is: Optimization techniques for finding αi’s remain the same! Find α1…αN such that Q(α) =Σαi - ?ΣΣαiαjyiyj K(xi, xj) is maximized and (1) Σαiyi = 0 (2) 0 ≤ αi ≤ C for all αi f(x) = ΣαiyiK(xi, xj)+ b Notes Note: In the dual representation we used the Gram matrix to express the solution. Kernel Trick: Replace : kernel If we use algorithms that only depend on the Gram-matrix, G, then we never have to know (compute) the actual features This is the crucial point of kernel methods Notes All information is tunneled through the Gram-matrix information bottleneck. The real art is to pick an appropriate kernel. e.g. take the RBF kernel: if c is very small: G=I (all data are dissimilar): over-fitting if c is very large: G=1 (all data are very similar): under-fitting Support Vector Machines Have an intuitive impression of SVMs Major concepts Linear classifiers Kernel tricks A short introduction to SVM regression Some important ingredients of SVMs Support Vectors Loss function The selection of Hyperparameters SVMs packages Support Vector Regression Hard Margin soft Margin Dual problem ALEX J. SMOLA and BERNHARD SCHOLKOPF, “A tutorial on support vector regression”. 2004 Support Vector Machines Have an intuitive impression of SVMs Major concepts Linear classifiers Kernel tricks A short introduction to SVM regression Some important ingredients of SVMs Support Vectors Loss function The selection of Hyperparameters SVMs packages Support Vectors for Classification x x subject to Samples whose αi≠0 e subject to Support Vectors for Regression Samples whose (αi- αi* )≠0 Support Vector Machines Have an intuitive impression of SVMs Major concepts Linear classifiers Kernel tricks A short introduction to SVM regression Some important ingredients of SVMs Support Vectors Loss function The selection of Hyperparameters SVMs packages Loss Function REGRESSION CLASSIFICATION LS-SVM Support Vector Ma