PRML_ch4_sec1_2技术分析.pptVIP

Ch 4. Linear Models for Classification (1/2)Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Summarized and revised by Hee-Woong Lim Contents 4.1. Discriminant Functions 4.2. Probabilistic Generative Models Classification Models Linear classification model (D-1)-dimensional hyperplane for D-dimensional input space 1-of-K coding scheme for K2 classes, such as t = (0, 1, 0, 0, 0)T Discriminant function Directly assigns each vector x to a specific class. ex. Fishers linear discriminant Approaches using conditional probability Separation of inference and decision states Two approaches Direct modeling of the posterior probability Generative approach Modeling likelihood and prior probability to calculate the posterior probability Capable of generating samples Discriminant Functions-Two Classes Classification by hyperplanesor Discriminant Functions-Multiple Classes One-versus-the-rest classifier K-1 classifiers for a K-class discriminant Ambiguous when more than two classifiers say ‘yes’. One-versus-one classifier K(K-1)/2 binary discriminant functions Majority voting ? ambiguousness with equal scores Discriminant Functions-Multiple Classes (Cont’d) K-class discriminant comprising K linear functions Assigns x to the corresponding class having the maximum output. The decision regions are always singly connected and convex. Approaches for Learning Parametersfor Linear Discriminant Functions Least square method Fisher’s linear discriminant Relation to least squares Multiple classes Perceptron algorithm Least Square Method Minimization of the sum-of-squares error (SSE) 1-of-K binary coding scheme for the target vector t. For a training data set, {xn, tn} where n = 1,…,N.The sum of squares error function is… Minimizing SSE gives Least Square Method (Cont’d)-Limit and Disadvantage The least-squares solutions yields y(x) whose elements sum to 1, but do not ensure the outputs to be in the range [0,1]. Vulnerable to outliers Because SSE function p