Random Forests - Universit#233; dOttawa - University of Ottawa随机森林大学与# 233；dottawa -渥太华大学.pptVIP

Random Forests Paper presentation for CSI5388 PENGCHENG XI Mar. 23, 2005 Reference Leo Breiman, Random Forests, Machine Learning, 45, 5-32, 2001 Leo Breiman (Professor Emeritus at UCB) is a member of the National Academy of Sciences Abstract Random forests (RF) are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost, and are more robust with respect to noise. Introduction Improvements in classification accuracy have resulted from growing an ensemble of trees and letting them vote for the most popular class. To grow these ensembles, often random vectors are generated that govern the growth of each tree in the ensemble. Several examples: bagging (Breiman, 1996), random split selection (Dietterich, 1998), random subspace (Ho, 1998), written character recognition (Amit and Geman, 1997) Introduction (Cont.) Introduction (Cont.) After a large number of trees is generated, they vote for the most popular class. We call these procedures random forests. Characterizing the accuracy of RF Margin function: which measures the extent to which the average number of votes at X,Y for the right class exceeds the average vote for any other class. The larger the margin, the more confidence in the classification. Generalization error: Characterizing… (Cont.) Margin function for a random forest: strength of the set of classifiers is suppose is the mean value of correlation the smaller, the better Using rando