数据挖掘_chap5_alternative_classification.ppt

General Structure of ANN Training ANN means learning the weights of the neurons Algorithm for learning ANN Initialize the weights (w0, w1, …, wk) Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples Objective function: Find the weights wi’s that minimize the above objective function e.g., backpropagation algorithm (see lecture notes) Support Vector Machines Find a linear hyperplane (decision boundary) that will separate the data Support Vector Machines One Possible Solution Support Vector Machines Another possible solution Support Vector Machines Other possible solutions Support Vector Machines Which one is better? B1 or B2? How do you define better? Support Vector Machines Find hyperplane maximizes the margin = B1 is better than B2 Support Vector Machines Support Vector Machines We want to maximize: Which is equivalent to minimizing: But subjected to the following constraints: This is a constrained optimization problem Numerical approaches to solve it (e.g., quadratic programming) Support Vector Machines What if the problem is not linearly separable? Support Vector Machines What if the problem is not linearly separable? Introduce slack variables Need to minimize: Subject to: Nonlinear Support Vector Machines What if decision boundary is not linear? Nonlinear Support Vector Machines Transform data into higher dimensional space Ensemble Methods Construct a set of classifiers from the training data Predict class label of previously unseen records by aggregating predictions made by multiple classifiers General Idea Why does it work? Suppose there are 25 base classifiers Each classifier has error rate, ? = 0.35 Assume classifiers are independent Probability that the ensemble classifier makes a wrong prediction: Examples of Ensemble Methods How to generate an ensemble of classifiers? Bagging Boosting Bagging Sampling with replacement Build classifier on each bootstrap sample Each sample has prob