Chapter4MultilayerPerceptrons-医学影像处理实验室MIPL.ppt

Chapter 6Support Vector Machines Introduction SVM可被用來解pattern classification和非線性迴歸的問題。 基本上，SVM 是一個具有許多良好特性的linear machine The main idea of a support vector machine is to construct a hyperplane as the decision surface in such a way that the margin of separation between positive and negative examples is maximized. The support vector machine is an approximate implementation of the method of structural risk minimization. SVM在可分割的pattern中，Eq(2.101)的第一項為零，並最小化第二項。 SVM 具有獨特的特性，SVM未結合problem-domain knowledge，但能提供良好的 generalization performance 。 建構SVM演算法的核心是support vector xi和某輸入向量x間的inner-product kernel。 根據inner-product kernel的產生方式，可建構不同的非線性決策表面的學習機器特性。 Background Background (cont.) Optimal Hyperplane for Linearly Separable Patterns 考慮training sample {(xi, di)}Ni=1，其desire response為可被線性分離的+1或-1。 則此可分離的hyperplane定義成其中，x為input vector，w為可調整的weight vector，b為bias，因此可將此分離的問題描述成 Optimal Hyperplane for Linearly Separable Patterns 若給予權重向量w和偏差值b，則介於Eq(6.1)所定義的hyperplane和最接近的資料點間的距離稱為margin of separation，以符號r表示。 SVM的目標是找一特定的hyperplane，使margin of separation r最大化。 The decision surface is referred to as the optimal hyperplane. 令w0和b0分別表示最佳的權重向量和bias。 則用來表示輸入空間的多維度線性決策表面的optimal hyperplane可表示成 Optimal Hyperplane for Linearly Separable Patterns The algebraic measure of the distance from x to the optimal hyperplane is defined as a discriminant function 因為Eq(6.3) ，所以令g(xp)=0or Optimal Hyperplane for Linearly Separable Patterns 我們的目的是給予訓練集T ={(xi,di)}找出optimal hyperplane的參數wo和bo 根據圖6.2可知(wo,bo)需滿足下列條件 The particular data points (xi,di) for which the first or second line of Eq. (6.6) is satisfied with the quality sign are called support vectors. The support vectors are those data points that lie closest to the decision surface and are therefore the most difficult to classify. Optimal Hyperplane for Linearly Separable Patterns 考慮一support vector x(s) for which d(s)=+1，根據定義可得 根據Eq(6.5) ，The algebraic distance from support vector to optimal hyper plane is Margin of separation b