no longer confidential estimating the confidence of individual regression predictions不再保密评估个人的信心回归预测.pdfVIP

No Longer Confidential: Estimating the Confidence of Individual Regression Predictions 1 ¨ ¨ 2 1 Sebastian Briesemeister *, Jorg Rahnenfuhrer , Oliver Kohlbacher ¨ 1 Applied Bioinformatics Group, Center for Bioinformatics/Dept. of Computer Science, University of Tubingen, Germany, 2 Department of Statistics, TU Dortmund, Dortmund, Germany Abstract Quantitative predictions in computational life sciences are often based on regression models. The advent of machine learning has led to highly accurate regression models that have gained widespread acceptance. While there are statistical methods available to estimate the global performance of regression models on a test or training dataset, it is often not clear how well this performance transfers to other datasets or how reliable an individual prediction is–a fact that often reduces a user’s trust into a computational method. In analogy to the concept of an experimental error, we sketch how estimators for individual prediction errors can be used to provide confidence intervals for individual predictions. Two novel statistical methods, named CONFINE and CONFIVE, can estimate the reliability of an individual prediction based on the local properties of nearby training data. The methods can be applied equally to linear and non-linear regression methods with very little computational overhead. We compare our confidence estimators with other existing confidence and applicability domain estimators on two biologically relevant problems (MHC–peptide binding prediction and quantitative structure- activity relationship (QSAR)). Our results s