DataMi的nig_lectures10_11_classification.pptVIP

ICS 278: Data MiningLectures 10,11: Classification Algorithms Padhraic Smyth Department of Information and Computer Science University of California, Irvine Notation Variables X, C….. with values x, y (lower case) Vectors indicated by X Components of X indicated by Xj with values xj “Matrix” data set D with n rows and p columns jth column contains values for variable Xj ith row contains a vector of measurements on object i, indicated by x(i) The jth measurement value for the ith object is xj(i) Unknown parameter for a model = q Can also use other Greek letters, like a, b, d, g ew Vector of parameters = q Classification Predictive modeling: predict Y given X Y is real-valued = regression Y is categorical = classification Often use C rather than Y to indicate the “class variable” Classification Many applications: speech recognition, document classification, OCR, loan approval, face recognition, etc Classification v. Regression Similar in many ways… both learn a mapping from X to C or Y Both sensitive to dimensionality of X Generalization to new data is important in both Test error versus model complexity Many models can be used for either classification or regression, e.g., trees, neural networks Most important differences Categorical Y versus real-valued Y Different score functions E.g., classification error versus squared error Decision Region Terminlogy Probabilistic view of Classification Notation: let there be K classes c1,…..cK Class marginals: p(ck) = probability of class k Class-conditional probabilities p( x | ck ) = probability of x given ck , k = 1,…K Posterior class probabilities (by Bayes rule) p( ck | x ) = p( x | ck ) p(ck) / p(x) , k = 1,…K  where p(x) = S p( x | cj ) p(cj) In theory this is all we need….in practice this may not be best approach. Example of Probabilistic Classification Example of Probabilistic Classification Example of Probabilistic Cl