Bias of the Random Forest Out-of-Bag (OOB) Error for Certain Input Parameters英文文献资料.docVIP

Open Journal of Statistics, 2011, 1, 205-211 205 doi:10.4236/ojs.2011.13024 Published Online October 2011 (http://www.SciRP.org/journal/ojs) Bias of the Random Forest Out-of-Bag (OOB) Error for Certain Input Parameters Matthew W. Mitchell Metabolon Inc., Research Triangle Park, New Caledonia, USA E-mail: mmitchell@ Received June 8, 2011; revised July 10, 2011; accepted July 18, 2011 Abstract Random Forest is an excellent classification tool, especially in the omics sciences such as metabolomics, where the number of variables is much greater than the number of subjects, i.e., “ n? p ”. However, the choices for the arguments for the random forest implementation are very important. Simulation studies are performed to compare the effect of the input parameters on the predictive ability of the random forest. The number of variables sampled, m-try, has the largest impact on the true prediction error. It is often claimed that the out-of-bag error (OOB) is an unbiased estimate of the true prediction error. However, for the case where n? p , with the default arguments, the out-of-bag (OOB) error overestimates the true error, i.e., the random forest actually performs better than indicated by the OOB error. This bias is greatly reduced by sub- sampling without replacement and choosing the same number of observations from each group. However, even after these adjustments, there is a low amount of bias. The remaining bias occurs because when there are trees with equal predictive ability, the one that performs better on the in-bag samples will perform worse on the out-of-bag samples. Cross-validation can be performed to reduce the remaining bias. Keywords: Random Forest, Multivariate Classification, Metabolomics, Small n Large p 1. Introduction 1) for subsampling with versus without replacement (the replace parameter); 2) the proportions of observations used for the in-bag samples (the sampsize parameter); and 3) various values of