[8] Boosting Algorithms Regularization, prediction and model fitting by Peter Bühlmann and Torsten Hothorn英文资料.pdf

Statistical Science 2007, Vol. 22, No. 4, 477–505 DOI: 10.1214/07-STS242 © Institute of Mathematical Statistics, 2007 Boosting Algorithms: Regularization, Prediction and Model Fitting Peter Bühlmann and Torsten Hothorn Abstract. We present a statistical perspective on boosting. Special empha- sis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regres- sion models for survival analysis. Concepts of degrees of freedom and cor- responding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical mod- els are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fit- ting, prediction and variable selection. It is flexible, allowing for the imple- mentation of new boosting algorithms optimizing user-specified loss func- tions. Key words and phrases: Generalized linear models, generalized additive models, gradient boosting, survival analysis, variable selection, software. 1. INTRODUCTION gradient descent algorithm in function space, inspired by numerical optimization and statistical estimation. Freund and Schapire’s AdaBoost algorithm for clas-