The Case Against Accuracy Estimation for Comparing Induction Algorithms：（针对精度估计比较归纳算法）.pdfVIP

The Case Against Accuracy Estimation for Comparing Induction Algorithms Foster Provost Tom Fawcett Ron Kohavi Bell Atlantic Science and Tech Bell Atlantic Science and Tech Silicon Graphics Inc. M/S 8U-876 400 Westchester Avenue 400 Westchester Avenue 2011 N. Shoreline Blvd. White Plains, NY 10604 White Plains, NY 10604 Mountain View, CA 94043 foster@ fawcett@ ronnyk@ Abstract analysis is the prop er metho dology to provide such verication. We then provide a thorough analysis of We analyze critically the use of classica- classier p erformance using standard machine learning tion accuracy to compare classiers on natu- algorithms and standard b enchmark data sets. The re- ral data sets, providing a thorough investiga- sults raise serious concerns ab out the use of accuracy, tion using ROC analysis, standard machine b oth for practical comparisons and for drawing scien- learning algorithms, and standard b ench- tic conclusions, even when predictive p erformance is mark data sets. The results raise serious con- the only concern. cerns ab out the use of accuracy for comparing The contribution of this pap er is two-fold. We analyze classiers and draw into question the conclu- critically a common assumption of machine learning sions that can b e drawn from such studies.