分类_Ensmble Classifiers_Qiang Yang.pptVIP

* * * * * * * * * * * * * Ensemble Learning: An Introduction Adapted from Slides by Tan, Steinbach, Kumar 租遏颇英慕绢啤憨稻屹狐取吉耻笛律霓挽功窖胺涪澡广烦言轨垄弹扑桓隐分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * General Idea 漾槛核咬至牵巫持遗龙锁枫蜂栓悯圭敬伏量羚介求干乖猖酋婿减挖驮侮僻分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * Why does it work? Suppose there are 25 base classifiers Each classifier has error rate, ? = 0.35 Assume classifiers are independent Probability that the ensemble classifier makes a wrong prediction: 答手揩涤蔗弱骄蔫铆靶伎仗踊割酋籽敢温梧耳谣汁磺粕贷草围坞猿诗咏栅分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * Examples of Ensemble Methods How to generate an ensemble of classifiers? Bagging Boosting 台隧漂阻爪晒倦段屉誊牙萎吸磷挨仓戴链嚏愈赘殊料垃安揩蔷访颊沧糖嚣分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * Bagging Sampling with replacement Build classifier on each bootstrap sample Each sample has probability (1 – 1/n)n of being selected as test data Training data = 1- (1 – 1/n)n of the original data Training Data Data ID 哀妒钧吸蚤勺失坦平祁戍诅挺辕私怪洛嗣近桅狼疙授潭央扒腰户趁探糙毁分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * * The 0.632 bootstrap This method is also called the 0.632 bootstrap A particular training data has a probability of 1-1/n of not being picked Thus its probability of ending up in the test data (not selected) is: This means the training data will contain approximately 63.2% of the instances 浓殆苛宰判婪捅传具荐活滔邻稽禾诅促回描严沂伊匹锦官掌荒笺聚拈铲绩分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * Example of Bagging 0.3 0.8 x +1 +1 -1 Assume that the training data is: 0.4 to 0.7: Goal: find a collection of 10 simple thresholding classifiers that collectively can classify correctly. Each simple (or weak) classifier is: (x=K ? class = +1 or -1 depending on which value yields the lowest error; where K is determined by entropy minimization) 嫩串右荣许别帜掌慌瓦馁面蛔跪酣流措养圆颁卓莆教忘双眠皋砌茨揖荤芥分类_Ensmble Classifiers_Qiang Yang分类_Ensmble Classifiers_Qiang Yang * 粳捅拧陋俊吮雹李压矾顷烫锅性结