Support Vector Machine and Its Applications.pptVIP

Support Vector Machine Its Applications Mingyue Tan The University of British Columbia Nov 26, 2004 Overview Intro. to Support Vector Machines (SVM) Properties of SVM Applications Gene Expression Data Classification Text Categorization if time permits Discussion 从几何上说，支持向量机就是要在r 维空间中寻找到最佳决策面，该决策面能最好地区分正例和反例，使正例与反例之间的分类间隔最大。SVM 的基本思想可以用正例和反例线性可分的情况来说明，如图3 中二维线性可分情况所示。图中，实心点和空心点代表两类样本，H为分类线，H1、H2 分别为过各类中离分类线最近的样本且平行于分类线的直线，它们之间的距离叫做分类间隔（margin）。可以选择不同的平行线集来区分正例与反例，而支持向量机选择使平行线区间间隔最大的一组作为最佳决策面。最佳决策面上的样本称为支持向量。 Linear SVM Mathematically What we know: w . x+ + b = +1 w . x- + b = -1 w . (x+-x-) = 2 Lagrange multipliers and KKT theorem Dual Problem(1) Dual Problem(2) Advantages of Dual Form with simpler constraints, and convex quadratic program algorithms could be applied; the dimension of input space is replaced by the number of input patterns; both the final decision function and the function be maximized are expressed in dot products, which could be computed by a kernel in high dimension space. Soft Margin Classification Mercer Theorem: Nonlinear SVM - Overview Properties of SVM Flexibility in choosing a similarity function Sparseness of solution when dealing with large data sets - only support vectors are used to specify the separating hyperplane Ability to handle large feature spaces - complexity does not depend on the dimensionality of the feature space Overfitting can be controlled by soft margin approach Nice math property: a simple convex optimization problem which is guaranteed to converge to a single global solution Feature Selection SVM Applications SVM has been used successfully in many real-world problems - text (and hypertext) categorization - image classification - bioinformatics (Protein classification, Cancer classification) - hand-written character recognition Application 1: Cancer Classification High Dimensional - p1000; n100 Imbalanced - less positive samples